Sefism is free for learners. We are here to help you grow, one step at a time.Join for free
Sefism.
Web

Web Accessibility Regression Testing in CI

A CI tool for web teams that runs accessibility checks on every pull request and fails the build only for newly introduced problems, so a site with existing issues can stop getting worse. An intermediate final year project in accessibility, testing and DevOps, with scripted keyboard checks that rule engines miss.

Topics
Difficulty

Intermediate

A small team, or one strong student willing to learn something new.
Effort

1 semester · 2 students

Deliverables

7 to ship

3 optional extras

Suggested stack

TypeScriptPlaywrightaxe-coreNode.jsGitHub ActionsSQLite or PostgreSQL (baselines)
A suggestion, not a requirement. Swap anything for what you already know.

What you should ship

  • Checker that visits configured routes, including pages behind a login, and runs axe-core rules in a real browser
  • Stable issue fingerprinting so the same problem is recognised across builds even when markup shifts slightly
  • Baseline file recording known issues, with commands to accept, expire or shrink the baseline deliberately
  • Differential report that fails the build only on issues absent from the baseline and posts a readable pull request comment
  • Scripted keyboard checks for focus order, visible focus, modal focus traps and controls unreachable without a mouse
  • Grouping of repeated issues by component so one root cause is reported once, not forty times
  • Evaluation on at least three open source web applications, reporting what was found and how much was actionable

If you have time left

  • Screen reader output snapshots (accessible name and role tree) compared across builds
  • A dashboard tracking the baseline shrinking over time per team or per component
  • Storybook integration so components are checked in isolation before they reach a page

The problem teams have with accessibility tools

Accessibility checkers exist and teams switch them off, for a specific and rational reason: pointed at an existing site they report four hundred issues, the build fails, and the check is disabled that afternoon. Meanwhile every new release adds a button with no label, a modal that traps focus, or text with poor contrast. People who use screen readers or keyboards pay for it, and so do organisations that have legal accessibility obligations.

The fix is not a better rule engine. It is a workflow a real team will keep switched on.

What already exists and the gap

  • axe-core is the widely used open source rule engine; @axe-core/playwright runs it inside a real browser in tests.
  • Pa11y CI checks lists of URLs or a sitemap and is good for scheduled site scans.
  • Lighthouse CI tracks an accessibility score on every build alongside performance.
  • WAVE is a browser tool for manual, structured audits by a person.

Practitioner guides already recommend blocking pull requests on new violations while tolerating a triaged legacy baseline. What most teams lack is tooling that makes that practical: recognising the same issue across builds when a class name or DOM path changes, grouping duplicates by component, and checking the keyboard behaviour that rule engines cannot see. That is your project.

What you build

A checker with a baseline. Record existing issues once, then fail the build only for new ones. The site stops getting worse immediately, and the baseline can be reduced deliberately over time.

Users are front end developers (who read the pull request comment), an accessibility lead or tech lead (who owns the baseline), and QA engineers (who add routes and user flows). Main flows: configure routes and login steps, create the first baseline, open a pull request and get a comment listing only new issues, then accept or fix them.

The technically interesting part

Two problems deserve your supervisor's attention.

Fingerprinting. A baseline is useless if every small markup change makes old issues look new. Design a fingerprint from rule id, the nearest stable landmark or component name, the element role and accessible name, and a normalised selector. Measure how many issues keep their identity across ten real commits of an open source project.

Keyboard checks. Automated rules check markup and contrast; they cannot tell you that focus jumps around, that a modal traps focus, or that a dropdown is unreachable without a mouse. Script these with Playwright: tab through a page, record the focus sequence, detect cycles and invisible focus, and compare the sequence with visual reading order.

How to evaluate it

  • Real sites: run it on at least three open source web applications. Report issues found, how many were genuinely actionable, and false positives. Honesty here strengthens the report.
  • Baseline stability: replay ten to twenty real commits and report how many baseline issues were wrongly reported as new.
  • Keyboard checks: seed ten known keyboard bugs into a demo app and report how many your checks catch.
  • Speed: CI time for 20 routes; target under two minutes.

Suggested plan

  1. Weeks 1 to 2: study WCAG basics, run axe on three sites by hand, collect examples.
  2. Weeks 3 to 5: route runner with login support and raw reports.
  3. Weeks 6 to 8: fingerprints, baseline file, differential report and pull request comment.
  4. Weeks 9 to 11: keyboard checks and component grouping.
  5. Weeks 12 to 14: evaluation on open source apps, write up.

Path to a product

Start as an open source GitHub Action with clear docs; agencies that build sites for clients with accessibility requirements are the first real users, because they inherit legacy issues constantly. A hosted dashboard with baseline history and per client reports could charge per project. Consulting on accessibility audits is a realistic side income for a team that has built this.

Scope warning

Automated tools catch a minority of accessibility problems. Say so explicitly and position the tool as a regression guard, not a compliance certificate. Do not write your own rule engine; use axe-core. Do not build a browser extension or a visual editor.

Learn what you need on Sefism

Get help with this idea

The baseline and fingerprinting design decide whether teams keep this switched on. Tauseef can help you scope routes and keyboard checks, choose open source apps to evaluate on, and design results examiners will trust. If you are new here, create a free account first.

Ideas and guidance, not finished projects

These are project ideas and scoping guidance, published free for students to use as a starting point. I do not build, write, or sell final-year projects, and I do not complete coursework for anyone. Take an idea, make it yours, and build it.

Not sure this one fits you?

I help students pick an idea that matches their skills and their deadline, then scope it down until it is finishable. Guidance only, never done for you.

Discuss this idea
Work with me

Stuck on something specific?

Writing only gets you so far. If you want an answer to your situation rather than the general case, book a session and we will work through it together. Sessions are free for approved Sefism members, and a few slots open each week.

Book a free sessionSee all services

Follow along

New writing, resources and project ideas land here first.