Automated accessibility testing reliably detects roughly 30 to 40% of WCAG success criteria. Everything else needs a person, because the remaining questions are about meaning, and a machine cannot tell whether alt text is accurate, whether a page reads in a sensible order, or whether captions match what was actually said.
This article argues against the sufficiency of our own product, which is deliberate. We sell automated scanning and dated remediation records. If we let you believe a scan is the whole job, we would be making the same claim that cost the largest overlay vendor a $1,000,000 FTC penalty in April 2025.
Where the 30 to 40% figure comes from, and what it means
The number moves depending on what you count, which is why you see different figures quoted confidently. Three separate things get measured, and they are not interchangeable.
| What is being counted | Typical figure | Why it differs |
|---|---|---|
| WCAG success criteria a tool can test at all | roughly 30 to 40% | Most criteria need a judgement about meaning, so no rule can evaluate them. |
| Individual issue instances found on a real site | higher, sometimes quoted above 50% | Contrast and missing alt attributes repeat many times per page, so they dominate the count. |
| Barriers that actually stopped a user completing a task | lower again | Blocking problems are usually structural or interactive, the category automation cannot see. |
The high number is not dishonest, but it flatters automation. A page with one image missing alt text and one keyboard trap has two problems. The tool finds one of them, which is 50% by count. The one it missed is the one that stopped somebody buying anything.
What a machine does check well
Automation is genuinely valuable for one class of failure: things that are objectively true or false about the markup, and that repeat at scale.
- Colour contrast ratios between text and its background, calculated exactly.
- Images with no alt attribute at all.
- Form inputs with no associated label of any kind.
- Empty links and buttons with no accessible name.
- Missing page language declarations.
- Duplicate ID values, which break the relationships ARIA depends on.
- ARIA attributes that are invalid, or used on roles that do not permit them.
- Skipped heading levels and pages with no headings at all.
- Tables that are used for layout but marked up as data tables.
These are worth finding. They are also cheap to fix, they regress easily when a site changes, and they are exactly what a scheduled scan is good at catching repeatedly over time. That is the honest case for automation: it is a smoke alarm, not a fire inspection.
What a machine cannot check, with real examples
Whether the alt text is meaningful
A scanner can confirm an alt attribute exists. It cannot tell you the attribute is useless. All of the following pass every automated check ever written:
- alt="DSC_04817.jpg" on a product photograph.
- alt="image" on the chart that contains the entire finding of a report.
- alt="Click here" on a logo that links to the home page.
- alt="Blue jumper" on a photo where the point is the cable knit pattern and the fact it has a zip.
- A decorative border image with a long, dutifully written description that a screen reader user must now sit through on every page.
The judgement required is: what is this image doing here, and what would someone miss if they could not see it? That is a question about purpose, and there is no rule to evaluate it against.
Whether the reading order makes sense
A screen reader reads the document in source order. CSS grid and flexbox can move content anywhere on screen regardless of where it sits in the markup. A page can look perfect and read as nonsense.
A common example: a three-column pricing table where the highlighted middle plan appears third in the source. A sighted user sees the recommended plan immediately. A screen reader user hears it last, after two plans they were not being offered. No automated tool flags this, because nothing is technically wrong. The markup is valid, the contrast passes. It just does not make sense when heard.
Whether the captions are accurate
A tool can detect that a video has a caption track attached. It cannot check whether the captions say what the speaker said. Auto-generated captions routinely mangle names, technical terms, prices and numbers, which are usually the words carrying the information. A video whose captions render "fifteen" as "fifty" has passed the automated check and misinformed the viewer. The same applies to caption timing, speaker identification, and whether relevant non-speech sound is described. All of it needs someone to watch the video with the captions on.
Whether link text means anything on its own
Screen reader users often pull up a list of every link on a page and navigate from it. In that list there is no surrounding sentence, so twelve links reading "read more" and "click here" are twelve identical entries. Every one passes an automated check, because each link has an accessible name.
Whether the headings describe the content
Automation checks that heading levels are not skipped. It cannot tell you that a section headed "Our approach" is actually the pricing, or that a large paragraph doing the job of a heading was never marked up as one. Headings are a table of contents for people who cannot scan the page visually, and one that lies is worse than none.
Whether an error message helps
WCAG asks that errors are identified and, where possible, that a correction is suggested. A tool can verify an error message is programmatically linked to its field. It cannot judge that "Invalid input" tells the user nothing, while "Enter the date as DD/MM/YYYY" tells them what to do. That distinction is the entire value of the criterion.
Whether keyboard focus goes somewhere sensible
This is the big one, and it needs a human every time. Open a dialog: does focus move into it? While it is open, can you tab out into the page behind, which is now invisible but still focusable? Does Escape close it? When it closes, does focus return to the button that opened it, or jump to the top of the document?
A tool can find fragments of this, for example a focus indicator removed with CSS. It cannot walk your checkout with a keyboard and tell you the flow is broken. That takes five minutes and a person who never touches the mouse.
Whether colour is the only thing carrying meaning
Contrast ratios are computed exactly. Whether information is conveyed by colour alone is not. A form where invalid fields simply turn red, a status column of coloured dots with no label, a chart with a colour-coded legend: all pass contrast checks and all fail a colour-blind user.
Automation also produces false positives
A scanner may flag contrast on text that is deliberately disabled, or on white text over a background image it cannot evaluate. It may flag a heading order issue in a component that is fine in context. Deciding which findings are real takes judgement, which is another reason a scan output is the start of a task rather than a task list.
A testing programme a small team can actually run
- Automated scan, scheduled, across the whole site. Catches mechanical failures, and catches them again when they return after a redesign.
- Keyboard-only walkthrough of your three most important journeys, once a quarter. Unplug the mouse. Buy something. Submit the contact form.
- Screen reader pass on those same journeys, once or twice a year. NVDA is free on Windows, VoiceOver is built into macOS.
- Alt text review whenever content is added, by the person adding it, who is the only one who knows what the image is for.
- Caption check on every video before publishing, watched once with the sound off.
- Zoom to 200% and to 400% and check nothing is cut off or overlapping.
- A published, monitored route for users to report barriers. Real users find things no methodology does.
Steps two, three and four find most of the genuine barriers. They are also free, which matters if budget is the reason nothing has happened yet.
Automated testing detects roughly 30 to 40% of WCAG success criteria. It cannot determine whether a site is legally compliant, and it is not a substitute for manual review and assistive-technology testing. AccessLedger sells a dated record of what automated checks found on a given day. Nothing more, and nothing less.
The reason we say this on every report is not modesty. If our scan told you that you were compliant, you would stop looking, and the barriers automation cannot see would stay exactly where they are. The scan is evidence of what a machine could check on a given date. Deciding what that is worth is a job for you and your lawyer, not for us.