Skip to content
Arbor

How we test

Our audits follow WCAG-EM, the evaluation method the W3C publishes. Automated tools catch the mechanical faults. Everything else we test by hand, with a keyboard, screen readers, zoom and reflow.

Five steps

  1. 1

    Define the scope

    We agree the target, WCAG 2.2 AA, the assistive technology and browsers, and anything extra your team needs, and write it into the quote.

  2. 2

    Explore the site

    We look for the common pages, the tasks people come to finish, the kinds of page you have and the technology behind them.

  3. 3

    Pick the sample

    Every template and every key user flow, plus a few pages picked at random to catch surprises: at least two, or 10% of the main sample if that is more.

  4. 4

    Test the sample

    An automated pass with axe and Accessibility Insights comes first. Then we test every success criterion by hand, with a keyboard, screen readers, zoom, reflow, contrast tools and motion settings.

  5. 5

    Report with evidence

    Every issue gets its evidence, a severity rating and a recommended fix.

If you add agent readiness, we also score your site with our Agent Readiness Index. Its weights and pass rules are on the agent readiness page.

Our test setup

NVDA
The latest release, with Chrome and Firefox on Windows
TalkBack
With Chrome on a mid-range Android phone
Keyboard
Every task by keyboard alone, in Chrome, Firefox and Edge
Zoom and reflow
Zoom at 200% and 400%, and reflow at a width of 320 CSS pixels
Contrast
TPGi's Colour Contrast Analyser and Windows contrast themes

For the automated pass and for spot checks we also use axe DevTools, Microsoft's Accessibility Insights for Web, WAVE, ANDI from the US Social Security Administration, HeadingsMap and a text spacing bookmarklet.

How we rate severity

Every issue also lists the WCAG criteria it fails and an effort estimate: small, medium or large.

  • Blocker

    A disabled user cannot finish a key task.

    Example: The Place order button cannot be reached by keyboard, or a CAPTCHA offers no other way through.

    Fix: before anything else

  • High

    A key task is possible but very hard, or a whole kind of content is unusable.

    Example: Form errors are never announced, or product photos have no text alternative.

    Fix: in the next release

  • Medium

    Confusion or extra effort, with a workaround.

    Example: Heading levels are skipped, or links only say "click here".

    Fix: within the quarter

  • Low

    Minor, or a good practice beyond WCAG A and AA.

    Example: A title attribute repeats the visible link text.

    Fix: when convenient

Evidence for every issue

  • The date and time, and the page we tested, on staging or in production.
  • The browser and the assistive technology, with their versions.
  • Steps to reproduce, with the expected and the actual result.
  • A screenshot with the element marked and any personal data blurred.
  • For screen reader issues, a transcript or a recording of what it said.
  • The selector when we know it, and a recommended fix with code.

We record the screen and the screen reader's speech with OBS Studio. A 20-second recording of NVDA saying "button, button, button" on a product page convinces a developer faster than any paragraph.

Before the report goes out

  • Every Blocker and High issue is checked a second time.
  • Every issue has its evidence attached.
  • No personal data from your site is visible.
  • Spelling and links are checked.
  • The report itself is checked with PAC and read with NVDA.

Your data and access

  • Test logins reach us through Bitwarden Send, never by email.
  • Where we can, we test on a staging site with test accounts. We never place real orders or use real customer accounts.
  • When the project ends, we ask you to switch off the test accounts and change any shared passwords. We delete our local copies and the stored logins.

See the method on your own site

Send us one page. We check it by hand and reply within three working days with its five most serious issues.

Get a free check