Understanding Pointing Problems in Real World Computing Environments
Amy Hurst, Jennifer Mankoff, Scott E. Hudson · 2008 · Proceedings of the 10th International ACM SIGACCESS Conference on Computers and Accessibility (Assets '08) · doi:10.1145/1414471.1414481
Summary
This paper develops techniques for collecting and analysing pointing performance data during real-world computer use, rather than solely in laboratory settings, to understand how pointing behaviour varies over time for individuals with motor impairments. Two computers were installed at Pittsburgh's United Cerebral Palsy Center where six participants (mean age 40.5, conditions including cerebral palsy, upper extremity impairment, arthritis, memory impairment, and intermittent tremor) could use them freely during their spare time over a four-month period. The software extended the DART system to log all pointing, keyboard, and window events, plus capture 200x200 pixel screenshots around each click for later analysis. The authors address a fundamental challenge of real-world data: not knowing user intent. They solve this through a hybrid approach combining event log analysis with difference imaging — comparing screenshots before and during clicks to determine whether a click produced a visible change (successful) or not (accidental). Six pointing performance metrics were analysed: too many buttons (simultaneously pressing left and right), accidental clicks, double click speed, slips (cursor movement during a click), direction changes in X and Y axes, and excess distance travelled (ratio of actual path to straight-line distance).
Key findings
The central finding is high variability in pointing performance both within and across sessions for all participants. Levene's test confirmed unequal variance across all measures (p<0.001). Double click speed averaged 1315.85ms (SD=303.88) — more than twice the Windows default limit of 500ms, with individual means ranging from 1090ms to 2616ms. The number of clicks needed to open a desktop icon ranged from 2 to 20. Accidental clicks accounted for 17.5% of all clicks in the two most-used games, with 66.72% of accidental clicks involving a slip during the click. Slip distances varied from 0 to over 20 pixels across sessions (overall mean=5.9, SD=19.45). Excess distance ratios ranged from near-optimal 2.0 to as high as 15.86 for the same participant across different sessions. Critically, baseline laboratory performance often differed dramatically from real-world use — for example, one participant showed no overlapping button presses during baseline but exhibited them repeatedly during real-world sessions. Cross-correlation analysis revealed that in bad sessions, multiple types of difficulties tended to co-occur: slip variance correlated with Y direction change variance (r=0.90, p<0.05) and excess distance variance (r=0.82, p<0.05). Most participants were unaware of available accessibility settings, and one expressed fear of changing settings because he might not be able to revert them.
Relevance
This paper makes a compelling methodological argument that has broad implications for assistive technology practice: configuring accessibility settings based on a single laboratory assessment is fundamentally inadequate because pointing performance varies dramatically and unpredictably over time. The variability stems from factors like medication effects, disease progression, fatigue, and daily fluctuations in motor control — none of which are captured in a one-time lab session. For practitioners, this means that assistive technology assessments should be ongoing and ideally incorporate real-world usage data. The finding that most participants did not know about available accessibility configurations, and one was afraid to try them, highlights a critical gap in assistive technology delivery. The paper's vision of software that continuously monitors pointing performance and automatically adapts settings anticipated modern adaptive interface research. The specific error categories documented (slips, accidental clicks, double click timing, direction changes) provide a practical taxonomy for understanding and measuring motor accessibility barriers.
Tags: motor disability · pointing · input methods · assistive technology · cerebral palsy · adaptive systems · research methodology · in-the-wild study · mouse accessibility