Quantifying Touch: New Metrics for Characterizing What Happens During a Touch
Junhan Kong, Mingyuan Zhong, James Fogarty, Jacob O. Wobbrock · 2022 · Proceedings of the 24th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '22) · doi:10.1145/3517428.3544804
Summary
This paper formalizes 15 target-agnostic metrics for characterizing what happens during a touch interaction on a touchscreen, moving beyond the traditional approach of treating touches as simple (x, y) coordinate events. The key insight is that touches are not atomic — they unfold over time and space as a finger lands, potentially slides, changes contact area and orientation, and lifts. This temporal unfolding is especially pronounced for users with limited fine motor function (LFMF), for whom performing quick, accurate touches can be difficult. The 15 metrics are organized into three categories. Location-and-time metrics include touch direction (overall angle of movement), variability (total distance traveled, i.e., "jitter"), drift (displacement from finger-down to finger-up), duration (time the finger is on screen), and extent (maximum distance between any two touch centroids). Area metrics include absolute and signed area change (whether the contact patch grew or shrank), area variability (cumulative area changes), area deviation (standard deviation of area), and area extent (range of areas). Angle metrics include absolute and signed angle change (rotation of the touch oval), angle variability, angle deviation, and angle extent. Each metric treats a touch as a time series of ovals rather than a single point, capturing the shape, orientation, and size of the finger's contact area at each sampling frame. The authors also propose three policies for handling inadvertent multi-finger touches: first-down (use whichever finger lands first), longest-lived (use the finger that persists longest), and sum-of-all (aggregate all finger traces). The metrics are computed from a Microsoft Perceptive Pixel (PPI) 55-inch tabletop display capable of reporting complete touch-oval information including centroid position, major/minor axis lengths, and orientation angle.
Key findings
The metrics were exercised in a study with 27 participants — 15 with limited fine motor function (LFMF, average age 63.8, conditions including essential tremor, arthritis, Charcot-Marie-Tooth disease, spinal cord injury, and traumatic brain injury) and 12 without LFMF (average age 34.8). Participants completed 200 crosshair-tapping trials each, generating 5,400 total touch trials. Participants were grouped into three impairment levels: None (11), Moderate (10), and Severe (6). The Severe group showed significantly higher touch variability (p < .01), drift (p < .01), duration (p < .01), and extent (p < .01) than both other groups, confirming the metrics' ability to detect fine-motor challenges. Crucially, different fine motor challenges correlated with different subsets of metrics — for example, tremor correlated with variability, drift, duration, extent, and area change; stiffness correlated with direction, variability, drift, and extent; pain correlated with variability, drift, and area change. This granularity means the metrics can characterize not just that someone has motor difficulties, but potentially what kind. The three multi-finger policies yielded similar overall conclusions, with the longest-lived finger matching the first-down finger in 77.2% of multi-touch cases, while the sum-of-all policy magnified group differences. All 15 metrics are target-agnostic, meaning they can be computed at runtime without knowing what interface element the user was trying to touch — a critical property for real-world deployment.
Relevance
This work provides a foundational measurement framework for anyone building adaptive or accessible touch interfaces. The target-agnostic nature of the metrics means they could be deployed in shipping software to detect motor challenges at runtime and dynamically adapt the interface — enlarging targets, increasing touch tolerance, or adjusting confirmation delays based on observed touch behaviors. For accessibility researchers, the metrics offer a standardized, mathematically formalized vocabulary for describing touch performance that goes far deeper than accuracy and speed alone, enabling more nuanced understanding of how different motor impairments manifest in touch interactions. The correlation between specific fine motor challenges and specific metrics opens possibilities for health applications — potentially tracking tremor progression, medication effects, or fatigue over time through touch behavior analysis. For practitioners designing touch-based systems, the key insight is that treating a touch as a single point discards rich information about user ability; systems that observe the full touch process can be more responsive to diverse motor abilities, aligning with ability-based design principles.
Tags: touch input · motor accessibility · human performance · touch metrics · fine motor function · touchscreen accessibility · ability-based design