Evidence

The numbers behind the technical claim, the methodology that produced them, and case studies of where Paradise gets results that single-file linters and rendered-DOM scanners miss. This page is also the place to be honest about where Paradise still falls short.

In progress. The corpus, the comparison harness, and the per-issue analysis are documented in the Paradise repo under ANALYZER_AUDIT_REPORT.md; this page is being written from there.

The 88% number

On the test corpus, Paradise reports 88% fewer false positives than the combination of axe and eslint-plugin-jsx-a11y running over the same code. The corpus is a curated set of HTML / JavaScript / CSS triples drawn from real production codebases (with permission), each annotated with ground-truth accessibility issues by experienced practitioners.

False positives are the issues that noneof the practitioners flagged but at least one tool reported. Those are the warnings that train developers to ignore accessibility tooling — every false positive is a user-trust cost. The reduction matters because it’s the gap between “tool you act on” and “tool you skim past”.

The number is honest about scope. It is computed against source-detectable issues — things that can be identified without running the application. Runtime-only concerns (focus order during dynamic content load, ARIA-live announcements, third-party widgets injected at interaction time) are not in the corpus. For those, runtime tools are the right answer; see Lineage for the autoA11y / Paradise complementarity.

Methodology

The evaluation pipeline:

  • Corpus. A curated set of HTML / JS / CSS triples, each between 200 and 5,000 lines total, drawn from real production codebases with permission. Diverse in framework choice (vanilla, React, Vue, Svelte, Angular).
  • Ground truth. Each triple is annotated by two independent accessibility practitioners; their annotations are reconciled into a single ground-truth set of issues. Inter-rater agreement is reported in the audit document.
  • Tools under test. axe (rendered-DOM), eslint-plugin-jsx-a11y (AST-pattern), and Paradise. Each tool is run against each triple in its native idiom (axe via Puppeteer; eslint via ESLint; Paradise via its Node API).
  • Counting.True positives are ground-truth issues each tool reported. False positives are issues a tool reported that weren’t in ground truth. False negatives are ground-truth issues no tool reported.

The 88% headline is the aggregate false-positive rate across the corpus. Per-corpus breakdowns and per-issue-type breakdowns live in the audit report.

Case studies

Three or four worked case studies will land here once the cleaned-up corpus is published. For now, the worked example on Architecture shows the canonical “cross-file split” case — a click handler in handlers.js, a CSS rule in styles.css, an HTML element in index.html — that single-file tools cannot reason about.

Where Paradise still misses

Honest framing: the source-level approach has structural limits. Paradise will not see:

  • Runtime-only content.Third-party widgets injected into iframes, dynamic modules whose source isn’t present at scan time, content streamed from a server in response to user interaction.
  • Real focus order during animation. Paradise reasons about declared focus behaviour; animations and transitions can change the ordering in ways that only a runtime tool can observe.
  • Live-region announcements.Paradise can detect that a live region exists; it can’t evaluate whether the announcement timing and content are appropriate for the user.
  • Visual-design regressions. Contrast ratios in dynamically-themed UIs, focus-indicator visibility against arbitrary backgrounds — Paradise checks the rules; rendering tools check the result.

Each of these is a runtime tool’s strength. The honest accessibility practice uses both: source-level analysis catches what authors can fix in source, runtime analysis catches what only manifests when the application runs. The integration model with autoA11y (Paradise’s runtime complement; full history on Lineage) is described in the Paradise repo’s AUTO_A11Y_INTEGRATION_PLAN.md.

Reading on