Augmenta11y

Enabling individuals with dyslexia to read faster.

What I Did

  • Project Lead
  • Requirement Gathering & Analysis
  • Competitive Research
  • High Fidelity Screens
  • Cross-platform App Development
  • Usability Testing

Tools Used

  • Sketch
  • VS Code
  • Android Studio
  • XCode

Team Members

  • Anand Chowdhary
  • Mudita Sisodia

Duration

  • August 2018 to Present

Problem

According to Dyslexia International, at least 1 in 10 people are affected by dyslexia, i.e. more than 700 million children and adults worldwide.

On the web, there are some plugins & extensions that help make websites more inclusive and accessible. For our bachelor's thesis project, my team and I wanted explore how this concept could be translated into the real world – enabling individuals with dyslexia to read textual information faster while being on the go.

Solution

Screenshots of the app
  • Augmented Reality accessibility app for individuals with dyslexia to read faster

  • Observed a 21% decrease in reading times using the app

  • Works in 7 different languages

  • Cross-platform iOS & Android app

  • 2,000+ downloads worldwide

Our Approach

Research

Requirement Gathering

Since the idea had been validated by the existence of the web-based accessibility tools already, our goal was to figure out if this need translated to real-world textual information as well. For this, we audited how accessible such information was and prepared various use-cases where our app could help. Some examples included reading station names on a subway station, comprehending a restaurant menu, and reading the news or magazines.

We made mockups of how the text would look inside the app, and showed these to people of various age groups suffering from learning disabilities and noted down their feedback during our semi-structured interviews.

Analysis

The response we received was vital in determining our target audience. It was inferred that the primary audience which would benefit from such a product were children in the age group of 8-14. This was also backed by research, which stated that children coping with learning disabilities benefit greatly from the right kind of interventions and support provided by technology. Ages 15-60 were found to have found ways to deal with reading comprehension already, and hence didn’t believe that the app helped them greatly.

We thus identified the factors affecting readability for individuals with dyslexia.

Target Audience

  • Primary Audience: Children aged 8-14.

  • Secondary Audience: Ages 60+ who don’t necessarily suffer from dyslexia, but liked the idea due to ease of readability.

Research Insights

  • There's no universal solution for improving readability (everyone has different preferences)

  • Major reading issues occur with children

  • No portable/mobile solutions exist for dyslexia

  • No dyslexia-specific accessbility apps on the market

Design

Based on our research insights, we moved on to defining design implications for our solution. We knew right off the bat that we couldn't blindly rely on existing UI patterns because we were dealing with a niche audience. This meant not relying on hidden gestures, icons, and using familiarity from other apps to inform the app's design.

Some of the implications were:

  • Simple mental model

  • Child-friendly functionality

  • Accessibility first

Owing to existing solutions, we identified a few key features that were a must for this solution:

  • Personalized Settings & Onboarding Experience: Since learning disabilities are subjective, it’s important to have an on-boarding experience which understands user preferences and provides easy customization.

  • Reader Mode: Tapping a block of text opens a new screen with the text coupled with a text-to-speech mode.

  • History Mode: All tapped text blocks are saved for easy retrieval.

Tech Stack

Due to the accelerated timeline defined by my university, we didn't get a lot of time to mockup these screens, and thus decided to develop the MVP directly as an app. Having experience with Web Development and JavaScript individually, we opted to use React Native, a cross platform app development solution maintained by Facebook. Using open source libraries and some trained machine learning models, we developed the prototype application’s basic features.

Iteration 1

The first iteration of the app served as an alpha-build and was launched in October 2018. This version was showcased at Dutch Design Week 2018, where we got some valuable feedback about the overlapping bounding text boxes, user personalization preferences, and some experience pain points. This alpha served as a proof-of-concept and basis for the first round of testing.

In hindsight, one of my learnings was that we definitely should have thought more about the visual design of the app before starting to code. It would have saved us a lot of time later when discussing the overall user experience of the app.

Screens

Camera Screen
Reader Mode
Settings Screen
History Screen

Feedback

Based on the feedback and its analysis, we re-aligned our design goals.

Here are some of the user quotes we obtained from the first round of feedback.

Home Screen:

  • “How does the app work? I don’t get it”: Lack of a tutorial/onboarding experience

  • “What are the default font settings? Why don’t you let me pick before I start using the app?”: No universal solution, must let users pick

  • “I like how simple the interface is”: Reinforced our design implication of not relying on icons/hidden gestures

Reader Screen:

  • “This mode helps me read without the camera distractions”: Positive feedback for having an isolated reading experience

  • “I can't understand the text-to-speech”": Offer more accents, slower tempo options for read aloud mode

History Screen:

  • “Why can’t I clear my history?”: Must provide an option to delete recorded pieces of text

  • “Why can’t I go to the Reader from the History screen?”: Tapping on a text box should take them to the Reader mode

  • “I see the same item in my history multiple times”: Checking and removing duplicate entries

Settings Screen:

  • “These font names are confusing”: Instead of font names being the focus, it should be their styles.

  • “Why are the labels not in my selected font?”: The entire app's UI should change into the selected font instead of just the AR elements.

Dutch Design Week 2018

Iteration 2

We prioritized between the feedback we received and decided to implement it in phases. The most important features were pushed out in Iteration 2.

New Onboarding

The on-boarding helps users set their preferences in terms of the app's language, line height, letter spacing, font styles, and background colors. These are also customizable anytime inside the app. We also introduced internationalization and localization inside the app – making it compatible with 7 different languages.

Adding translations

The new Camera screen was capable of detecting the language of the text the user had scanned, and checking it with their preferred language. If the two didn't match, a new ‘Translations’ available button helped them read better.

Minor Improvements

We also added a new guided walkthrough feature to help users understand the various features of the app.

Guided Walkthrough based on v1 feedback
Improved settings modal and extended reader mode functionality

Testing and Evaluation

We tied up with health organizations from various cities in India to validate our research. The test subjects with a group of 39 students aged 8–14 years. The students were provided with two passages appropriate for each age group. The task given was to read one paragraph normally, and the other using the app. To ensure one paragraph was not inherently easier to read than the other, we alternated between the paragraph read using the app. The time taken by the students to read each passage, and their text styling preference recorded.

Students during testing & evaluation sessions

Outcomes

We saw a 21.2% reduction in the amount of time taken to read text using Augmenta11y. It was also observed that 85.7% of the students found the OpenDyslexic font helpful and 76.9% of them preferred to have a yellow background to the text.

Visualizing & analyzing the test data

We also presented a poster at India HCI 2018 – an ACM SIGCHI conference and will be publishing this research as a paper at the 9th International Conference on Computer Communication and Informatics (ICCCI 2019) in the HCI track.

Here's me at India HCI 2018 👋

In March 2019, after five months in beta – Augmenta11y launched worldwide officially!

Download it here for Android & iOS devices.

Next Steps

Even though my thesis got over in May, this was a project that was very close to my heart. Thus, I decided to continue working on it. In November 2019, I had the pleasure of interacting with Apple's Developer Relations team, specifically some UI/UX experts and software engineers who gave me a lot of valuable feedback on the app.

After analysing their points of improvement plus customer requests for more features, I'm currently working on developing the third iteration of the app. Here are the prototyped high-fidelity UI mockups of the upcoming redesign:

Camera Screen
Nudge to interact
Highlighted text
Bottom Sheet Expanded
Text Settings
Camera Settings

The main objectives of this re-design were:

  • Interface Design Improvements

  • Standardising according to platform design guidelines

  • Improving navigation & way-finding

This was done by introducing a straight-forward bottom navigation, subtly highlighting text boxes in the AR experience instead of overusing the yellow, and making a new settings screen instead of a modal.

Learnings

  • Building empathy for a user population I didn’t know a lot about has helped me grow immensely as a designer

  • Being where your users are always provides much greater context than just reading about them

  • Knowledge of technical constraints while designing and developing a product end-to-end helps in setting realistic product milestones

In The Press