Describing people to computers

Adaptation is impossible without a description of the person

An adaptive interface is one that changes to suit its user. Every discussion of how it should change skips past a prior question: change according to what? The system needs a description of the person on the other side of the screen, in terms it can act on.

Almost all accessibility work addresses the other half. The Web Content Accessibility Guidelines describe properties a page should have. The Accessible Rich Internet Applications specification describes semantics a widget should expose. Both are descriptions of content. Neither says anything about the person, because both assume the person brings assistive technology that already knows what they need.

That assumption holds for a screen reader and breaks for nearly everything else. A game cannot infer from a screen reader that its player has three degrees of gaze accuracy and needs two and a half seconds to confirm a selection. Somebody has to write that down.

A preference is not a capability

The standard that does describe people is ISO/IEC 24751, known as AccessForAll. It became a standard in 2008 and is still the deployed one. It records a person’s Personal Needs and Preferences: a screen reader, a speech rate, a Braille display, a set of display options.

I raised two objections to it in 2009, and seventeen years later I would raise both again.

The first is that a profile saying the user requires a screen reader does not say what it is about the user that requires one. Are they totally blind? Do they have residual vision that is still useful to them? Are they sighted and using a screen reader for some other reason altogether? Those are different people, and the difference governs what an interface can attempt beyond handing its text to speech synthesis. Where there is usable vision, size and contrast and layout still do work and can still be adapted. Where there is none, they are effort spent on nothing and something else has to carry the information. The profile records the accommodation and discards the reason for it, and the reason was the part a system could have acted on.

The second is how context of use was expressed. It required a complete standalone profile for each context. A low-vision reader who magnifies on a large desktop display and switches to a screen reader on a phone has to describe themselves twice, and more than twice once environmental factors come into it. Collaborative settings were barely considered at all.

Nothing about that information is wrong. The problem is what the duplication does to it. Each profile is a record of what that person does in that situation, which makes it a statement of preference rather than of need, and it is bounded by the technology the person or whoever profiled them happens to know about. A capability that some other arrangement might have used never gets written down, because the profile only has room for the arrangement already arrived at.

Both objections come to the same thing. A description of the person would have survived a change of device, a change of surroundings and the arrival of a technique nobody had thought of. A description of their current arrangement survives none of those. The external influences and setting groups later in this piece are what separate the two: the capability is recorded once, the situation is declared apart from it, and a context is a view of the one profile rather than another copy of the person.

So the model I proposed records what a person can do, not what they have chosen. Its governing sentence has not changed: it is what the user can do, not why they cannot. Not the diagnosis, not the equipment, and not the accommodation. The capability.

None of which abolishes preference. What a person wants survives, and it should override whatever a system works out for itself. But it has to be recorded as preference, and keeping the two apart is exactly what the 2009 models did: one for what a person is able to do, another for what they would rather. Separating them is what allows the second to overrule the first without anyone having to dress it up as the first.

The profile advises, the person decides

One thing has to be said here, because it governs how everything below should be read. A capability profile is an inference about somebody, and inferences are wrong sometimes. It describes what is likely to work and it informs a sensible default, and that is the whole of its authority. It does not decide. If a person wants lower contrast than this model would suggest, or smaller text, or a channel that looks unlikely to serve them, that is their choice and a system’s work is to honour it.

I have watched a model of mine be wrong about exactly this, and the case is worth reading before trusting any profile too far. It is set out in describing what people want, along with what preference is and how it might be recorded.

What a system actually needs to know

The test for any property in this model is simple, and I only made it explicit in 2026 after finding I had drifted away from it. A property has to name a decision that a renderer, an input handler, or a content selector actually makes. If it cannot, it is a medical observation with a schema around it, however true it may be.

Applying that test found three practical questions the model could not answer at all:

The spread on the first of those is the argument for recording it. Across my worked profiles, text entry runs from three words a minute for a single-switch scanning user to thirty for someone who types with their toes. A tenfold difference, invisible to any model that records only which input device is in use.

What follows separates into four concerns. There is the schema, meaning the properties themselves and what may be known about anyone. There is one person’s values against that schema, gathered into the contexts they are used in. There is what that person would rather, which is a different question from what they can do. And there is changing a profile in response to something happening. Most of what is below is the first two.

Every capability is full, partial, or absent

This is the shape of the whole model and the thing most easily misread, including by me when I rebuilt it. Each property takes one of three values. A measurement appears against the middle value and against nothing else.

So a person who cannot perceive contrast has contrast NONE. Writing nought per cent would claim that a measurement was taken of something that is not there. This distinction sounds pedantic and is not: a nought is a number, and numbers get computed with.

This is easy to lose, and I lost it in my own work for years, because a property and its measurement sit so naturally in the same column. Focus is full, partial or absent. Focus duration is a number of minutes. Those look like two different kinds of property and they are not. They are the same kind, with the minutes attached to its middle value: full means indefinitely, none means not at all, and the minutes exist only in between.

What a measurement looks like

Minutes are easy, because a person can answer in them. Ten other properties used to carry a percentage instead, and nobody can answer in those. The replacements are ordered scales whose every point is a sentence someone can be asked. The one below is shared by all six colour and intensity channels, because those six decide the same thing between them: which hues and tones may carry meaning.

The other four read the same way. Contrast need runs maximum, strong, raised, typical. Vibration runs strong-only, typical, subtle. Position sense runs needs-watching, needs-landing-check, reliable-unseen. Steadiness runs large-only, unsteady, mostly-steady, steady. They are different lengths on purpose: a scale should be as long as its subject has distinct states, not as long as a house style says.

One point on that first scale is worth dwelling on, because it is the one a percentage could never express. A person whose channel is unreliable is not the same as one whose channel is absent. Absent is NONE, and no measurement arises. Unreliable is somebody who perceives the light perfectly well and still cannot discriminate with it, which is what red-green colour blindness actually is. Writing that as a low percentage would suggest dimness, and dimness is not the problem.

Precedence decides which questions are worth asking

Properties sit in a hierarchy of importance. The original paper attaches one sentence to the top of every table: remaining properties are only of interest for PARTIAL sight.

Read literally, and it is meant literally, that sentence does a great deal of work. A child property is worth asking about exactly when its parent is PARTIAL. A FULL parent leaves no impairment to describe. A NONE parent leaves nothing to describe either. There is no point asking a blind person for their minimum readable font size.

Two consequences follow, and I got the second of them wrong at first.

Absence propagates. A capability cannot exist beneath one that does not. The model refuses partial colour perception under absent sight, and that refusal is correct.

Fullness does not propagate. A FULL parent makes its children uninteresting, never forbidden. I first implemented this as a ceiling, where a child could never exceed its parent, and it broke immediately. Someone with tunnel vision has PARTIAL sight and may have entirely FULL colour perception. A Braille reader has FULL language and a very specific tactile script. Recording either is extra detail, not a contradiction.

The clearest case came from a profile for a person with Multiple Sclerosis. Hearing is FULL and listening duration is PARTIAL at fifteen minutes, because fatigue in Multiple Sclerosis is central rather than sensory. Under a ceiling rule that combination is incoherent. It is not incoherent. It is the condition.

The body is not a pair of hands

The 2026 rebuild kept finding the same fault in different places. Every part of the model that touched the body assumed hands, and nobody had decided it should.

Tactile sense began as contact on the skin, which is whole-body and could not say that vibration white finger takes the fingertips and leaves the rest intact. So I narrowed it to the hands, which fixed that case and broke a larger one: a person with no arms who types with their toes has excellent sensation in their toes and, under a hands-only reading, none worth recording. I had traded one inexpressible person for another.

The fix was to say where. Tactile sense is now recorded by body site, listing only the sites that differ from full, because absence already means “not of interest” everywhere else in the model.

Then a question exposed something worse. Asked whether one-handedness was expressible, I found the model did not merely fail. It stated a falsehood. A setting of “hands, no sensation” claims both hands, and a one-handed person’s remaining hand feels perfectly well. An incomplete model is a gap. A model that asserts the opposite of the truth is a defect, and I had recorded it twice as the former.

Sites now carry a side, which is a separate field rather than a doubled list, because side is orthogonal to site. It earns its place by the decision test: which side a person works with decides where controls go, where a switch is mounted, and which one-handed keyboard layout to offer.

The same de-centring applies to control. A property named for manual stability assumed hands; it is now effector stability, because a foot is an effector and so is a chin. Discrete control now names which body sites do the work. Someone who types with their toes has full discrete control and needs a different layout, not a lesser one, and calling that PARTIAL without saying why would be a falsehood as well as an insult.

Knowing a language, receiving it, and producing it are three separate facts

The original paper has a fragment of a language grouping which points at this without completing it. It gives a property for reading signed text with the parents “sight plus signLanguageSet”, and never defines signLanguageSet. Filling that gap forced the distinction into the open.

Knowing American Sign Language does not require eyes. Reading it visually does. A DeafBlind signer may have had American Sign Language as a first language since childhood and now receive it hand over hand. When I first made knowledge of a signed language depend on sight, the model refused to let that person know their own first language.

The paper already had the answer in its own structure. Language itself has no parents, while reading written text needs sight, understanding speech needs hearing, and reading sign needs sight. Knowing and receiving were always separate. I had collapsed them.

Production is a third axis, and it had only one property to itself. Someone with tremor or absent touch may read the two-handed manual alphabet on their own hand without difficulty and be quite unable to spell it onto someone else’s. The rule that emerged is short: reception depends on senses, production depends on hands.

Recording four language skills separately, listening, speaking, reading and writing, is what makes two very different people expressible. A person using English as an additional language understands better than they speak. A Deaf person may read and write English fluently with no listening at all. “Knows English” is true of both and useless about either.

Most properties decide nothing on their own

Requiring each property to name a decision was the first half of the fix. The second half corrects an overstatement in the first.

Contrast sensitivity sets no palette by itself. It does so together with six colour and intensity bands. Gaze accuracy fixes no target size until it is read alongside minimum target size and effector stability. A property usually contributes to a decision rather than making one, and a model that claimed otherwise would produce one overstated claim per property.

So a property declares either a decision it makes or a contribution it offers, naming the properties it must be read with. Of the twenty-six decisions the current model supports, seventeen are joint. Two thirds of what this model decides needs more than one property, and the tidier one-to-one version would have hidden every one of them.

That last row is worth dwelling on. Automatic speech recognition is trained on a narrow band of voices. A person whose family understands them perfectly may be unusable by voice control, and a system that infers the machine figure from the human one will offer “just talk to it” and strand them. Recording a single intelligibility number would hide exactly the case that breaks.

Templates are views, not compartments

Fifty-nine properties is too many to hand anyone at once, so the model groups them. The paper calls these Capability Templates and defines them as views of Properties that reflect grouping such as those of Tables 1 to 4, with the crucial rider that the same Property may exist in many templates.

That rider is the whole point, and it is why a template is not a category. Sight appears in both the vision template and the colour one. Key control appears in both alternative access, because a key is a key whether it is pressed by a finger or by a head switch. Kinaesthesia appears in both input and touch, because knowing where your hand is belongs to the haptic sense and to the act of aiming equally. Force those overlaps into a single tree and you have to pick a winner, and every pick throws away a real relationship.

The templates then group once more, into the four that follow Nesbitt’s sensory design spaces and the three that do not. Input, alternative access and reading are not senses. They are things a person does, and that extension was anticipated from the start: it is possible to imagine other groupings, not related to specific design spaces, with use of language one obvious candidate.

The same person, a different situation

A profile is not a fixed answer, because some of what a system needs to know is not a fact about the person at all. The paper makes the point with the tremor case: the physical stability of the screen also plays a part, so that a person with hand tremors may find that the readable size of text depends on whether the screen is placed on a Table, or is held in their hand. Same person, same tremor, different answer.

So an External Influence is a first-class thing in the model rather than a variable someone reads from the environment. It is declared, its permitted values are declared, and both an undeclared influence and an out-of-range value are refused rather than quietly defaulted. A system cannot invent a situation it was never told about, which matters because the failure would otherwise be silent and would look like the profile being wrong.

A setting whose value comes from other settings plus an influence is a derived one, which is what Shlaer-Mellor calls mathematical dependence. Two rules keep it honest. A derived value must cite the formula or algorithm that produced it, and that citation travels with the resolved value, so a font size of twenty-four points can be traced back to the rule and the inputs that produced it rather than appearing as a number the system asserts. And the dependencies must not form a loop: a cycle between derived settings is refused, because a value defined in terms of itself has no value.

Settings then gather into Setting Groups, which are the same idea as a template one layer down. A template is a view over properties in the schema. A Setting Group is a view over one person’s settings for a named situation, and it can declare which influences bear on it. The reference profile has one called seeing, described simply as at a desk with a mounted display, and another for audio-first play. The person does not change between them. What the system should ask about them does.

The unit of use is not always one person

Working through profiles for switch, breath and gaze access, I concluded that a real-time falling-block game is structurally closed to a single-switch scanning user. Scanning takes seconds per selection and the pieces do not wait.

That conclusion was narrow in a way I could not see: it assumed the unit of play is an individual. When bespoke controls cannot bridge a gap, disabled gamers routinely share controls with a gaming buddy who covers the timing or the inputs they cannot manage. Microsoft ships a Copilot mode on Xbox that makes two controllers act as one, for precisely this.

My own model had the structure and I had never asked what it was for. Its Entity is defined as either a user or a group of users, and a group’s settings are functionally dependent on its members’. What that did not anticipate is this resolution. A classroom sharing one screen needs the most accommodating setting, so if one student needs eighteen point type, everyone gets eighteen point type. A co-pilot pair is the opposite: capability is divided, so whatever either can do, the pair can do.

The asymmetry in what may be lent is the part that matters:

• Motor capability delegates cleanly. A buddy’s hands are as good as anyone’s for pressing a button on time. The game cannot tell whose finger arrived.
• Perception does not delegate in real time. A sighted friend describing a falling piece is always describing where it was. In a turn-based game the same description works perfectly, which is a fact about the game rather than about the people.
• Comprehension must not delegate at all. A buddy who decides what to do is not assisting, they are playing, and a model that called that access would be lying.

So co-piloting rescues the switch-scanning player from a real-time game and does nothing whatever for a DeafBlind one. That asymmetry is the finding, and it is the sort of claim that softens into “assistive partnership helps” and stops meaning anything unless it is stated precisely.

Lending a capability has a consequence that is easy to miss and would break a renderer. Once the assistant supplies a capability, the settings hanging beneath it describe the primary alone and no longer describe the pair, so the model marks them superseded and a renderer must not act on them. Sizing a control for the primary player’s unsteady hand is exactly wrong when it is the buddy’s hand on the button. The pair is a third thing, and reading either person’s profile on its own gets it wrong in a different direction.

Eighteen profiles, and what each one broke

The model was rebuilt against worked profiles rather than in the abstract, and every one of them found something. They are stand-ins, to be replaced or augmented with lived experience when it is available, and each records that basis in its own data so a fixture cannot quietly become a finding. They are deliberately not personas: no name, no age, no occupation and no narrative, because those invite generalising from a character to a population.

Fred is like Jim except

None of the eighteen is written out in full. Each is expressed as its difference from a single reference profile. The idea is as old as the model itself: it is possible to say ‘Fred is like Jim except…’, and starting with Jim’s profile, to create Fred’s profile describing only the differences between the users. An Instance adds, modifies, or deletes rows in the Tables, and that is exactly the transaction the code performs.

Writing profiles this way is not a storage optimisation. It is what makes them readable. The blind-since-birth exemplar is one changed line, and being one line is the finding: everything else about that person is unremarkable and the model should not pretend otherwise. The reference profile itself is seven lines, because a property only becomes worth recording when its parent is partial. Absent under a NONE and absent under a FULL both mean there is no question left to ask.

Three rules in the implementation earn their keep. An add that collides with something already present is refused, and so is a modify of something absent, because both are usually a typo rather than an intention. And a modify replaces the whole row instead of merging into it, so a capability can fall to NONE without a stale measurement surviving underneath it. That last one is the same error as writing nought per cent, arriving by a different route.

What is not implemented is the rest of it. The adaptation model also carries event triggers, instance sequences and a sequence number, which together let profiles be composed in a declared order in response to something happening. Those are designed and not coded, so a profile cannot yet be versioned through time, which is the same gap that makes progressive conditions snapshots rather than histories.

The full set, with every property and every value, is generated directly from the running models so the document and the code cannot disagree. Read the generated profile document (opens in a new window)

What else exists

The 2009 paper had one comparator. In 2026 there are several, and one of them arrives at my position independently.

Read against that field, what remains distinctive here is the three-value scale with a measurement only on the middle value, precedence as acquisition order, functional dependency on external influences, the separation of capability from capacity from preference, and a formal executable information model underneath it all. What is behind is the empirical grounding and deployment.

The survey does not hand me a scale. Cambridge solves the scaling problem on the demand side, where a task can be rated against a population, and that is a different question from asking a person what they can do. So the scales here are built rather than borrowed. The ten percentages are now five ordered scales, each sized to its own subject rather than to a house style, and every point of each one is a sentence somebody can be asked and can answer: a colour channel carries meaning reliably, or only when emphasised, or only as support, or not at all.

Replacing them had a consequence beyond tidiness. An ordinal value is not an interval one, so the distance between two neighbouring points is not a quantity and nothing may compute with it. One derived setting had been doing exactly that, scaling a font size by a formula over the stability percentage. That formula asserted a straight line between how steadily someone holds a device and how large text must be, which nobody established and which I invented along with the percentage it multiplied. It is now a declared table of four numbers with names on them, which someone who has watched a person work with a shaking hand can argue with.

References

This work

• Dodd, R., Green, S. and Pearson, E. (2009). User Capability in an Adaptive World. Proceedings of the 1st International Workshop on Multimodal Interfaces in Semantic Interaction, MSIADU ’09, Beijing. doi:10.1145/1631097.1631110
• Dodd, R., Green, S. and Pearson, E. (2008). The CISNA Model of Accessible Adaptive Hypermedia. Proceedings of the International Cross-Disciplinary Conference on Web Accessibility, W4A 2008. doi:10.1145/1368044.1368052
• Dodd, R. (2008). 20 Years On: the Dexter Model of Hypertext and its impact on web accessibility. ACM SIGACCESS Accessibility and Computing.

Standards and classifications

• ISO/IEC 24751-1:2008. Information technology. Individualized adaptability and accessibility in e-learning, education and training. Part 1: Framework and reference model. International Organization for Standardization. Confirmed as current on periodic review in 2020. iso.org/standard/41521.html
• ISO/IEC 24751-2:2008. Part 2: “Access for all” personal needs and preferences for digital delivery. iso.org/standard/43603.html
• ISO/IEC 24751-3:2008. Part 3: “Access for all” digital resource description. iso.org/standard/43604.html
• ISO/IEC 24751-4:2023. Part 4: “Access for all” framework for individualized accessibility and registry server application programming interface. iso.org/standard/82901.html
• World Health Organization (2001). International Classification of Functioning, Disability and Health. Geneva: WHO. who.int
• World Wide Web Consortium. WAI-Adapt, formerly Personalization Semantics. Web Accessibility Initiative. w3.org/WAI/adapt

Ability-based design

• Wobbrock, J.O., Kane, S.K., Gajos, K.Z., Harada, S. and Froehlich, J. (2011). Ability-Based Design: Concept, Principles and Examples. ACM Transactions on Accessible Computing 3(3), article 9. doi:10.1145/1952383.1952384
• Wobbrock, J.O., Gajos, K.Z., Kane, S.K. and Vanderheiden, G.C. (2018). Ability-Based Design. Communications of the ACM 61(6). doi:10.1145/3148051
• Nolte, A., Wobbrock, J.O., Volkmann, T. and Jochems, N. (2022). Implementing Ability-Based Design: A Systematic Approach to Conceptual User Modeling. ACM Transactions on Accessible Computing 15(4). doi:10.1145/3551646

Adaptive interface generation

• Gajos, K.Z., Weld, D.S. and Wobbrock, J.O. (2010). Automatically generating personalized user interfaces with SUPPLE. Artificial Intelligence 174(12–13), 910–950. doi:10.1016/j.artint.2010.05.005
• Gajos, K.Z., Wobbrock, J.O. and Weld, D.S. (2007). Automatically generating user interfaces adapted to users’ motor and vision capabilities. Proceedings of the ACM Symposium on User Interface Software and Technology, UIST ’07.

Inclusive design and population data

• Engineering Design Centre, University of Cambridge. Inclusive Design Toolkit. Capability scales, capability loss simulation and exclusion calculation. inclusivedesigntoolkit.com
• Engineering Design Centre, University of Cambridge. Exclusion Calculator. calc.inclusivedesigntoolkit.com

Deployment of AccessForAll

• Raising the Floor. AccessForAll, and the Global Public Inclusive Infrastructure. Includes the Cloud4All project. raisingthefloor.org

Design spaces

• Nesbitt, K.V. (2003). Designing Multi-sensory Displays for Abstract Data. PhD thesis, University of Sydney. The source of the visual, auditory and haptic design spaces the subject ontologies are scoped to.

What this model still cannot say

Recorded here rather than left to be discovered, because a limitation written down is a limitation someone can argue with.

The scale points are mine, not anybody’s lived experience. The percentages are gone and the scales that replaced them ask answerable questions, but where each of the eighteen profiles sits on those scales is still a judgement I made. In thirteen of the fourteen cases I made it by mechanically banding the percentage I had invented in the first place. That is a better shape carrying the same provenance, and it is what interviewing people replaces. The fourteenth is the interesting one: banding put the colour exemplar’s red and green channels on the same point, flattening the precise thing that profile exists to show, and I had to overrule it by hand.

Vision has no laterality. Hemiplegia commonly comes with loss of the visual field on the same side, and while a viewing rectangle can describe the remaining field, that is close rather than equivalent.

Progressive conditions are recorded as snapshots. A profile for late-stage motor neurone disease is true on the day it was taken. Versioning a profile through time is what the adaptation model’s instance sequences are for, and those are designed but not coded.

Preference model. The complementary 2009 Preference Model is not described here as we are dealing only with capacity and capability. It is described elsewhere in the Adaptation section of the site.

Body sites have no granularity below the named site. Sensation in one hand and not the other is now expressible. Sensation in the palm but not the fingertips is not.

Looking at the larger picture

A model of the user is half of an adaptive system. The other half is a model of the application, abstract enough that it can be rendered differently for different people, which is the line of work in the CISNA model of accessible adaptive hypermedia.

Modelling capability and capacity helps define user need. Interaction modalities and the metaphors they use have to be considered alongside it, to establish what capabilities a modality requires of the user before it can be used effectively. That is part of an Interaction Model, which matches possible modalities to user need.

The two meet in a demonstrator. The accessible Tetris case study is where these profiles are put to work against a real application, and where the model’s findings stop being tidy. A real-time falling-block game is closed to a single-switch scanning user unless the game itself will wait, and an audio-first demonstrator has nothing to offer a DeafBlind player at all. Both are findings rather than embarrassments, and both came from writing the profiles down.

The models, the profiles and the generated document are open source. Read the code on GitHub (opens in a new window)