George Kerscher, Chief Innovations Officer at the DAISY Consortium, gives an overview of the newly updated Fundamental Accessibility Tests for Reading Systems, which will launch the January 2019 round of testing.
Over the last four years or so, the DAISY Consortium has been driving the testing of EPUB reading systems and apps. It is quite a challenge, but I think we have a pretty good methodology in place. This short article will give an overview of how the testing works and will provide an analysis of the value it delivers to various communities. Of special interest will be the newly updated Fundamental Accessibility Tests for Reading Systems.
The Methodology
The unique approach lies in the Fundamental Accessibility Tests for Reading Systems, which is an EPUB 3 publication. It is an all-in-one testing system containing uniquely identified pass or fail tasks that a person with a disability performs. The results, which indicate whether the tasks are supported or not, are recorded on epubtest.org. We know this test title is perfect from an EPUB conformance perspective, so if we experience difficulties we don’t need to wonder if there was a mistake in the EPUB. We can therefore confine our testing to the reading system/app and the assistive technology that is being used.
Three disability groups are targeted:
- Persons who are blind and use screen readers
- Persons who have low vision and use visual adjustments to make the title easier to see
- Persons with learning differences such as dyslexia who rely on features such as increasing the line spacing or having the text read to them while the text is highlighted
An Example of a Test That a Person Who Is Blind Would Perform
A blind person like me uses a reading app with a screen reader. This reads the app menus and book contents using TTS (text to speech). Screen readers are available for computers, smartphones, and tablets, but not all apps work with them, including reading apps. This is why we test reading apps to make sure the fundamental accessibility features are supported.
The item below is a single test in the “Non-Visual” Reading chapter. Note, there are three parts:
- It has the unique identifier for reporting
- It gives the instructions for the test
- It provides guidance as to what passes or fails
Reading-010 Initiate “read from here”
Focus on the beginning of a paragraph or a sentence and initiate reading. Reading should continue until interrupted by the user or at the end of the current document, e.g., current chapter.
The text being read should preferably remain in view, but this is not a requirement to pass the test.
Indicate Pass or Fail.
An Example of a Test That a Person with a Specific Learning Difference Would Perform
Individuals with reading differences such as dyslexia rely on personalization, and this can be critical as a learning strategy. For example, documents are printed onto colored paper using preferred fonts and spacing to make it easier to read and understand information. This personalization is now available in reading apps, and our tests for visual adjustments examine these capabilities.
Another essential strategy is reading the text on the screen while it is simultaneously read aloud. This dual mode of reading has been shown to improve comprehension and strengthen reading skills. The read-aloud feature is so important than many apps now include it. Read-aloud also gets its own section in Fundamental Accessibility Tests.
ReadAloud-610 Text is emphasized as it is spoken by read aloud
Activate the Read Aloud feature in the reading system.
When the Read Aloud is activated, the text being spoken should be emphasized visually using a contrasting color or any other means. As the reading continues, the emphasis should also move to the next section of text to be read.
Indicate Pass or Fail.
Who Benefits from all this Testing?
- Developers of Reading Apps: We have many developers who rely on our testing to improve their applications. We are happy to work directly with them to create a great reading experience for people who use their vision, audio, or braille (or as we say, with eyes, ears, and fingers!). We do not want to embarrass companies and we will withhold results while they improve their product. However, eventually we have a duty to other communities to publish our results. Fortunately, we are having great success with many developers of apps that are widely used, and they even ingest the title and make it available for use in their system; this has the added benefit of testing some of the advanced annotation capabilities.
- Education Organizations: Universities and K-12 schools use the information and results that we publish to weed out applications that do not work for students with disabilities. In the US and other countries, it is a legal requirement to purchase accessible software and content. Once the information about the accessibility of specific products is known and made public, it makes it much easier for an educator to select an accessible product over one that could get them in trouble.
- Assistive Technology (AT) Developers: Because we test using AT such as JAWS, NVDA, VoiceOver, Talkback, and Narrator, we often find that the reading app appears to be behaving properly but the AT is preventing a test from passing. Because we can pinpoint the problem and reproduce it broadly, we provide the information that the AT developer needs to fix the problem on their side.
- Publishers: Publishers take note of the results. Many times, publishers provide guidance to customers about the applications that work for persons with disabilities and that support the accessibility features they are adding to their titles.
- Libraries: DAISY Consortium libraries all over the world look at the results to get guidance for use in their services. We are seeing an increase in mainstream public library services being evaluated for their accessibility, and this will include the apps they use.
- Persons with Disabilities: Of course, persons with disabilities themselves use the results and the guidance we publish to help them to decide which reading app to use. It really helps to get guidance because learning a new reading system can take a lot of time, and starting with one you know will work with your hardware and software configuration is a major benefit.
The Crowd Behind the Testing
We have a two-tiered method for recruiting testers. A core set of experts, including persons with disabilities, drive the development of the test titles and maintain the website and the results. Most of these people have in their job description a work item to do work with accessibility testing at epubtest.org. Some come from publishing houses or reading system companies. Some are accessibility consultants who are hired by tech companies to test specific reading apps, and these folks often team up with volunteers to confirm their results are accurate. Some of the testers come from the education sector and are skilled with a wide range of AT used on their campus. Most people come from DAISY Consortium member organizations.
We have a second tier of crowd source volunteers. We started this in early 2018 and have been thrilled with the response. We now have more than 70 people signed up who contribute to the published results and we expect the number to increase. For those who love reading, a lot of tips and tricks to have a better reading experience with specific reading app and AT combinations are shared in this group.
We know that apps change frequently, and so does the AT. This usually results in better scores, but sometimes a change to the operating system, reading app, or AT makes things worse. The number of permutations that one can have is staggering, and this is why we love our crowd of testers.
What Happens in January 2019?
When we replace the old “Fundamental Accessible Reading System Tests” with the new one, the results on epubtest.org will change. Because there are new tests, the scores will change, and the site will show that the test needs updating. The last time we did this, the developers freaked out because they want to show off the accessibility work they had done. This time, we gave them advanced warning that a new title was coming out. We also plan to really “hit it” with the testing and update all the current apps ASAP. I certainly hope that by the end of January we will have most of the testing completed.
Resources
EPUB Test website: http://www.epubtest.org—where you can get the new test title from the test suite and you can also see the accessibility test grid.
Inclusive Publishing website: http://www.inclusivepublishing.org—for the Reading App Roundup with specific recommendations.
This article was written for Inclusive Publishing and for BISG where it has been cross-posted.
George Kerscher, PhD, began his IT innovations in 1987 and coined the term “print disabled.” George is dedicated to developing technologies that make information not only accessible, but also fully functional in the hands of persons who are blind or who have a print disability. He believes properly designed digitally published materials and web pages can make information accessible to all people. George is an advocate for semantically rich content that can be used effectively by everybody.
As Chief Innovations Officer of the DAISY Consortium, Senior Advisor, Global Literacy, to Benetech, and member of Publishing Groups in the W3C, Kerscher is a recognized international leader in document access. In addition, he chairs the DAISY/NISO Standards committee and the Steering Council of the Web Accessibility Initiative and serves on the advisory board of the Institute of Museum and Library Services.
