QA Test Suite for an Open Source Web App

The problem with student QA portfolios

QA and SDET hiring asks the same question in different words: can you find real bugs, explain them clearly, and build automation that keeps working? Most student test projects run against a practice site built to be tested. They show that you can write a Selenium script, not that you can decide what to test in an unfamiliar product, communicate with developers, or keep a suite green for months.

Open source maintainers, meanwhile, often have thin end-to-end coverage and more issues than time. A careful tester who files reproducible bugs and contributes tests is useful to them. This project serves both sides: the maintainers of a real app get bug reports and tests, and you get public evidence a hiring manager can check.

What already exists and the gap

• Playwright ships with retries, a trace viewer, parallel workers and screenshot comparison built in.
• Allure and ReportPortal are open source reporting tools that track pass and fail history across runs and help spot flaky tests; Currents is a hosted dashboard for Playwright in CI.
• Self-hosted apps such as Vikunja, Memos, Mealie and Plane run locally with Docker and have real features: accounts, roles, forms and an API.

The tooling is solved. The gap is judgement: choosing what to test in a real product, designing tests rather than recording clicks, and proving the suite is reliable. That is what this project demonstrates.

What you build

Choose an app with an active issue tracker, a contributing guide, a Docker setup and at least two user roles. Run it locally so you never test anyone's production system.

• Risk analysis and test plan: list features, rate likelihood and impact, decide what gets deep testing and what is out of scope.
• Designed test cases for the riskiest features using partitions and boundary values.
• Exploratory testing with charters, time boxes and notes, and bug reports on the public tracker after checking for duplicates.
• UI automation of five key journeys with page objects, role-based locators and test data created through the API before each test.
• API tests with positive, negative, schema and access control checks (can user B read user A's data?).
• CI that starts the app with Docker Compose, runs both suites in parallel, uploads reports and traces, and runs axe scans on key pages.

The technically interesting part

The hard problem is flakiness and test isolation. A suite that fails one run in ten is worse than no suite, because people learn to ignore it. Show your evaluators how you removed every source of randomness: no fixed sleeps, only web-first assertions; each test creates and cleans up its own data through the API; no test depends on another's order; the database is reset per run from a known seed.

Then prove it. Run the full suite 20 times in CI and report every failure with its cause. Treat retries as a symptom to investigate, not a fix. The second interesting part is the access control API tests: they often find the most serious real bugs.

How to evaluate it

• Bug quality: how many filed reports were confirmed, labelled or fixed by maintainers.
• Reliability: 20 consecutive CI runs with zero unexplained failures.
• Speed: total suite runtime with parallel workers, and how it scales when you add tests.
• Coverage of risk: map each high-risk feature from the plan to the tests or sessions that cover it.
• Judgement: a written list of what you deliberately left manual, and why.

Suggested plan

1. Weeks 1 to 2: choose the app, run it locally, read the contributing guide, write the risk analysis.
2. Weeks 3 to 4: test design and exploratory sessions; file the first bugs.
3. Weeks 5 to 8: UI suite with page objects and API-seeded data.
4. Weeks 9 to 10: API and access control tests.
5. Weeks 11 to 12: CI, reports, accessibility scans.
6. Weeks 13 to 15: 20-run reliability study, upstream contribution, final report.

Path to a product

The first outcome is a portfolio that gets QA and SDET interviews, with public issues and pull requests as proof. From there, testers with this skill do paid test automation for small SaaS teams that have no QA, or offer a fixed-price "suite setup" package. A hosted flaky-test dashboard is a possible product, but there are established tools, so treat it as a stretch rather than a business plan.

Scope warning

Do not automate everything. Do not test or load test the project's hosted services. Do not file duplicate or low-effort issues to inflate your count. Do not build your own test framework; use Playwright or Selenium as they are.

Learn what you need on Sefism

• Open source roadmap : how to work with maintainers, trackers and contributing guides.
• DevOps and cloud cheat sheet : Docker Compose and CI basics for running the app in GitHub Actions.
• Test Automation University : free courses on Playwright, Selenium and API testing.
• roadmap.sh: QA : the full QA skill map, to place this project in context.
• axe-core : the accessibility engine behind the automated scans.
• How to Contribute to Open Source : etiquette for filing issues and pull requests.

Get help with this idea

Picking the right app and writing the risk analysis decide how good this project is. Tauseef can help you choose an app with enough depth, decide what to automate and what to leave manual, and structure the reliability study for your evaluators. If you are new here, create a free account first.

• Discuss this idea with a mentor
• Send a query to Tauseef
• Book a 1:1 session