The Bootstrap Blog

Accessibility (Part 4): User Testing

You can read our other entries on the subject: UI, Describing Images and The Definitions Area

"Blind kids just aren't going to make their own videogames if they can't see them."
--Anonymous

At Bootstrap, even our introductory Bootstrap:Algebra course has students program their own videogames. When it comes to differently-abled students, we want to take their affordances and constraints into account when it comes to the curriculum. What are the implications of graphical output when working with visually-impaired students? This past April, we had the opportunity to start looking at this very question.

We were invited to participate in the Alabama STEM Wars event, hosted by Daniela Marghitu's team at Auburn University. STEM Wars is a cross-discipline event aimed at exposing differently-abled students to various STEM activities, and Dr. Marghitu asked us to run the Computer Science track at the event. Most of the students at this particular STEM Wars were visually impaired, which made it a perfect chance to start looking at the use of graphical output with this population, and to put our new software to the test.

The event was incredible! Several students wore Star Wars costumes, and many of the teachers and facilitators did the same. After a fantastic welcome presentation, the students split up into three groups, which rotated through brief events in Computer Science, Engineering, and Biology. Most of the students were between the ages of 12-16, and none of them had ever programmed a computer before.

They only had a short time to learn any programming, but every student learned how to use our editor and learned enough Racket to write simple programs on the computer. Many of the students got really into it, and quite a few said that programming was their favorite event of the day. At the same time, we at Bootstrap learned a lot from the experience, and we think it's valuable to share our takeaways with the community.

Lessons Learned

Students are not familiar with screen-readers. We incorrectly assumed that many students would be comfortable with the basic keyboard shortcuts and spoken feedback used by screenreaders. CS teachers working with visually-impaired students should not make this same mistake! This has curricular implications, since it suggests at least some classroom time should be spent building up this knowledge for students.
There's a threshold for "maximum length of code", below which typing is better than blocks. Kids overwhelmingly preferred text-based programming for short expressions. Text is a really easy interface when you're not writing anything complicated. Visually-impaired students found the overhead of block navigation unecessary. There's reason to believe that this threshold is different for each student, which underscores a need for an environment that can switch between both.
Visually-impaired students enjoyed making images. We hypothesized that making images would be either uninteresting or downright frustrating for students who couldn't see them. This hypothesis turned out to be dead wrong. A visually-impaired teacher in the room with me, and he wasn't surprised at all. "Even when [visually-impaired] kids are included in a mainstream class", he said, "they're often given 'blindwork' which is different from the work given to sighted kids, and they feel that." The opportunity to write programs that would typically be intended for sighted users was exhilarating, not to mention the pleasure of being able to formally describe things that the students can't easily see themselves.

Equity doesn't end at the tool

Making your tool usable by differently-abled students is a tall order, but there's a lot to be learned from this experience. First, block-based editors aren't always a win! Depending on the student and the code being written/read, a text-based editor might be preferable. Second, curriculum writers should not take usability aids for granted. CS courses that aim for accessiblity should include materials and scaffolds that help introduce these devices - just as they already include materials for familiarizng sighted users with the UI of Eclipse, Scratch, or WeScheme. And finally, curriculum developers should be cognizant of the tradeoffs between making special accommodations for differently-abled students. There's a balance between totally other-izing and making no accommodations at all, and that balance point may be surprising, subtle, or even surprisingly subtle.

We are grateful to the National Science Foundation, the ESA Foundation, and Vint Cerf for funding this work and to Paul Carduner, Sina Bahram, and the AccessCSforAll Consortium for their feedback, advice, and engineering assistance. We are also grateful to Dr. Marghitu, who is a co-PI on the South East Alliance for Persons with Disabilities in STEM (SEAPD-STEM).

Posted June 16th, 2017