Implementing an Accessible Conversational User Interface: Applying Feedback from University Students and Disability Support Advisors
Francisco Iniesto, Tim Coughlan, Kate Lister · 2021 · Proceedings of the 18th International Web for All Conference (W4A) · doi:10.1145/3430263.3452431
Summary
This paper presents results from a beta trial of the ADMINS (Assistants for the Disclosure and Management of Information about Needs and Support) virtual assistant, developed at The Open University (UK) to help students disclose disabilities and receive guidance about available support. The OU supports over 20,000 students with declared disabilities, and the existing process using online forms and advisor conversations is challenging and time-consuming for both students and the institution. ADMINS replaces static forms with a conversational user interface (CUI) that supports both text and speech interaction, allowing students to describe their needs through dialogue rather than filling in structured forms. The project follows a participatory design approach, employing student consultants with diverse accessibility needs as expert stakeholders alongside disability support advisors. The trial used a mixed-methods methodology including free interaction and direct observation of students using the assistant, pre- and post-activity questionnaires (SUS and SUISQ-R), and semi-structured interviews. Twenty-two students completed the trial, representing a wide range of disabilities including long-term medical conditions, mental health conditions, fatigue or pain, specific learning difficulties like dyslexia, restricted mobility, autistic spectrum conditions, impaired speech, and visual impairment. Three disability support advisors also participated. A Person-Centred Planning approach allowed students to choose their preferred interaction modality and device.
Key findings
The SUS score was 72.3, classified as good (B) usability and above the average benchmark of 68. The SUISQ-R overall score was 4.67 out of 7, indicating a fairly good evaluation. User goal orientation (4.93) and customer service behaviour (5.64) scored well, confirming the assistant was correctly identified as supporting disability disclosure and was perceived as polite, courteous, and friendly. However, speech characteristics (4.23) and verbosity (3.89) scored lower — messages were too repetitive, overly talkative, and provided more detail than needed. Qualitative feedback revealed significant accessibility barriers: on iPad, text boxes lost focus making typing difficult; the rapid buffering of text caused the screen to scroll, forcing users to scroll back up to read questions; language was too technical for users with dyslexia ("assistive technology" definitions were too long and complex); the speech version sometimes picked up its own voice creating loops; and there were functional differences between text and speech versions. The trial generated 163 log instances for implementation improvements. Three key design principles emerged: conceptual design (manage expectations, clarify limitations, allow personalisation), conversational design (maintain engagement, avoid conversation breakdowns), and personality design (ensure empathy in language, maintain gender neutrality, adjust pace).
Relevance
This research tackles a critical real-world problem: the disability disclosure process in higher education is often inaccessible to the very students it is meant to serve. The participatory design approach — with disabled students as paid consultants, not just test subjects — is a model for inclusive technology development. The findings offer practical lessons for anyone building accessible conversational interfaces: language must be simple enough for users with cognitive and learning disabilities, not just technically compliant; speech interfaces need robust handling to avoid picking up their own output; text display speed and scrolling behaviour directly impact readability; and the same CUI must work comparably across text and speech modalities. The identified design principles around conceptual, conversational, and personality design provide a useful framework for accessible CUI development more broadly. The work also highlights that traditional usability metrics like SUS may miss accessibility-specific barriers that qualitative feedback reveals.
Tags: conversational user interfaces · disability disclosure · higher education · participatory design · chatbots · virtual assistants · user experience · accessibility testing