OnScreenDualScribe with Point-and-Click Interface: A Viable Computer Interaction Alternative based on a Virtual Modified Numerical Keypad
Kavita Krishnaswamy, Patricia Ordóñez, Phillip Beckerle, Stephan Rinderknecht, Torsten Felzer · 2016 · Proceedings of the 18th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '16) · doi:10.1145/2982142.2982184
Summary
This paper presents a first-person evaluation of the OnScreenDualScribe (OSDS) Point-and-Click Interface, a virtual keyboard designed for people who cannot use a standard keyboard or mouse effectively. The first author, who has Spinal Muscular Atrophy (SMA), can only interact via a few fingers in one hand using a trackball mouse, with lateral movement being easier than vertical. She fatigues after 1-2 hours of virtual keyboard use and requires a caregiver to place her hand on the device and massage her fingers for relief. The OSDS replaces the standard QWERTY layout with a compact alphanumeric grid organized into numbered rows, where users select letters by clicking row numbers on an avatar panel rather than targeting individual keys across a large keyboard. The system features word prediction with a 100,000+ word dictionary that narrows candidates as row selections are made — for example, pressing [1][5] followed by [3] populates predictions with words beginning with those letter combinations. The Point-and-Click Interface is an extension of a previous version that required a physical DualPad numeric keypad, making it accessible to users like the first author who cannot exert sufficient pressure on physical keys. It also includes dwell-time clicking for users who cannot click a mouse button, where hovering over a target for a configurable duration triggers a click automatically.
Key findings
The first author tested OSDS over seven days (with rest breaks on days 4 and 6), comparing it against her two primary text entry methods: Dragon NaturallySpeaking (speech recognition) and SofType (an on-screen QWERTY keyboard). Dragon NaturallySpeaking was fastest at 31.3 wpm average with 0.3 average errors. SofType averaged 9.4 wpm with 1.5 errors. OSDS started at 2.69 wpm on day 1 and doubled to approximately 5 wpm by day 7, with an overall average of 2.0 wpm and 3.7 errors. Despite being slower, OSDS showed clear improvement trajectory. The user provided 20 detailed observations covering customization gaps (no font size control, no layout customization, inability to turn off sound effects or dwell mode once enabled), usability issues (application crashes on certain inputs, interface components positioned in hard-to-reach screen locations, only rows 1-5 displayed despite having 8 rows), and positive features (word prediction is powerful and helpful, row selection is an efficient concept, the ability to type Greek and mathematical symbols makes it suitable for programming). The user noted the steep learning curve but valued OSDS as a viable alternative when speech recognition is inappropriate (e.g., for LaTeX or programming) and when SofType's larger key targets cause excessive fatigue.
Relevance
This paper is valuable as a detailed autoethnographic account of assistive technology adoption by a person with severe motor disabilities. It demonstrates that text entry speed alone is insufficient for evaluating input methods — the user's preference depends on context (speech for general writing, OSDS for programming, SofType for mixed use) and on physical factors like fatigue and range of motion. The 20 improvement suggestions provide a concrete roadmap for how research prototypes fall short of real-world usability: missing customization options, bugs that appear only in sustained use, and interface layouts that do not account for limited range of motion. For accessibility practitioners, this highlights that motor accessibility is not a single problem with a single solution — users with conditions like SMA need multiple complementary tools and the ability to switch between them based on task demands. The collaborative authorship model, with the primary user as lead author and the tool developer as co-author, exemplifies how participatory research should work in assistive technology development.
Tags: text entry · motor disability · virtual keyboard · assistive technology · word prediction · spinal muscular atrophy · input methods · alternative input