‘Hacking’ Assistive Technology: Creating Personal AI Tools for Access

Hybrid Event: 3725 BBB / Zoom  Passcode:183426

Abstract: This dissertation presents motivation for and approaches to ‘hack’ assistive software: to move beyond one-size-fits all assistive technology design, and towards enabling anyone to create and customize AI tools for accessibility that meet their unique needs. AI-based assistive technologies have been increasingly adopted by blind people to gain visual access in scenarios such as reading printed text, identifying objects, and navigating. These services tend to be designed in ways that maximize their broad applicability and usability, and they are intended for simple and common use cases, assuming universal needs. However, this leads to inflexibility, and there remains a long-tail of diverse needs that AI assistive technology cannot account for. My dissertation argues that in addition to ‘lowering the floor’ and making AI tools that are usable for everyone, we also need to ‘raise the ceiling’ and create systems that enable people to do more. Building on a foundation of ‘making’ and DIY cultures, I bring those philosophies into the assistive software space, and develop new approaches for people to make and adapt software to their needs.

I first present work that studies the challenges with existing assistive applications, finding that blind people already put a significant amount of effort into working around gaps in these tools. They engage in design and making to do so, through three strategies which I term hacking, switching, and combining. I demonstrate that there is a significant gap between the assistive technology solutions that people desire to create, and the approaches that can support them in doing so. I then present two tools that leverage end-user programming approaches in order to close this gap. First, I present ProgramAlly, an end-user programming application for creating customized filters for visual information, using existing models as building blocks that can be rearranged. Next, I present AllyExtensions, a system leveraging mobile automation tools to augment already deployed applications with new features in a flexible way. Through designing and evaluating these systems, I demonstrate end-user creation processes that are approachable, accessible, and expressive for blind people, as in ProgramAlly, and that can also be applied practically to real-world assistive technology workflows, as in AllyExtensions. Overall, this dissertation presents motivation and methods for the democratization of assistive technology creation, and supports people in having greater control over AI-based technologies in their lives.