Abstract:

In today's world, numerous individuals face significant barriers in accessing a complete education due to various disabilities. Among them are children who possess a deep love for reading but encounter difficulties without appropriate guidance. Digital learning tools, such as Immersive Reader, create reading experiences for learners of all ages and abilities that promote accessibility and comprehension. However, these technologies often possess limited flexibility, as they can only offer predefined solutions. For example, reading support features such as accompanying pictures are limited to single words, rather than arbitrary phrases.

In this work, I investigate how these emerging AI tools could be leveraged to augment the reading experiences for students with learning differences. My initial exploration began with an analysis of Immersive Reader’s functionalities, including color schemes, fonts, and its built-in dictionary. I identified limitations and areas for improvement, and as a result, I set out to address these limitations by introducing innovative modifications and refinements.

I prototyped Generative Reader, an AI-powered reading augmentation tool that allows students to select an arbitrary text or phrase, generates visual representations using Stable Diffusion or DALL·E, gets simplified explanations, as well as enables iterative Q&A using ChatGPT.

When it is used, the user would swipe to highlight a series of words, then a box would unfold like a paper to show the simplified version of the text generated by ChatGPT, and two images generated by Stable Diffusion or DALL·E which aims to explain the selected text. The user is free to click regenerate to get a further simplified text and new images to enhance their understanding. If the user has further questions, they can click on the microphone icon to ask questions which will be processed and answered by ChatGPT about the highlighted text, e.g., “how will this phrase be used”. To enhance the experience, a teacher component will be added to verify the accuracy of the generated image and text.

Keywords:

Dyslexia, Reading Support, Generative AI.