Sefism is free for learners. We are here to help you grow, one step at a time.Join for free
Sefism.
Other

Dependency Upgrade Impact Checker That Finds Which of Your Calls a Breaking Change Hits

Build a tool for TypeScript projects that compares the public API of two versions of an npm package, finds the breaking changes, then searches your own code for the exact call sites they affect. Dependabot and Renovate open upgrade pull requests but cannot say whether a major version bump touches anything you use. This tool answers that question with a list of files and lines, so small teams stop putting off upgrades.

Topics
Difficulty

Advanced

Needs a team, real planning, and a supervisor who knows the area.
Effort

2 semesters · 2 students

Deliverables

7 to ship

4 optional extras

Suggested stack

TypeScriptTypeScript compiler APINode.jsGitHub ActionsSQLite cachenpm registry API
A suggestion, not a requirement. Swap anything for what you already know.

What you should ship

  • An API extractor that reads the type declarations of a package version and records exported functions, classes, types and their signatures
  • An API differ that classifies changes as removed, renamed, signature changed, type narrowed or widened, and default changed where detectable
  • A usage scanner that resolves imports in a client project and maps every used symbol to the package API
  • An impact report listing each breaking change that affects the project with file, line and the call
  • A GitHub Action that comments the impact report on Dependabot or Renovate upgrade pull requests
  • A study on at least 30 real open source projects and 50 major version upgrades, checked against their actual build and test results
  • A cache of API snapshots so popular package versions are only analysed once

If you have time left

  • Suggested code changes for simple renames, clearly marked for review
  • Support for Python packages using type stubs
  • A public website showing the most disruptive breaking changes in popular packages
  • Detection of behaviour changes from changelog text as a low-confidence hint

The problem: upgrades nobody dares to merge

Every JavaScript and TypeScript project depends on dozens of packages. When one of them releases a new major version, Dependabot or Renovate opens a pull request. The changelog says "breaking changes" and lists twenty of them. The developer has no quick way to know whether any of those twenty affect their code, so the pull request sits open for months. Meanwhile security fixes land only in the new version, and the gap grows until an upgrade becomes a project of its own.

Small teams and university projects suffer most: there is no platform team to own upgrades, and every day spent on one is a day not spent on features.

What already exists and the gap

  • Dependabot and Renovate open upgrade pull requests and include release notes. They rely on your build and tests to reveal problems.
  • API report tools such as Microsoft's API Extractor help library authors track their own public API.
  • Java tools such as japicmp and Revapi compare two versions of a library and list binary and source incompatibilities.
  • The TypeScript compiler will find type errors after you upgrade, but only after you install and build, and only for what is typed.

The gap is joining the two halves: a diff of what changed in the library, filtered down to what your project actually uses, before you start the upgrade. Tests catch some of it; this tells you where to look.

Inspiration

Breaking bad? Semantic versioning and impact of breaking changes in Maven Central (Ochoa, Degueule, Falleri and Vinju, Empirical Software Engineering, 2022) studied how Java libraries signal breaking changes and how often those changes actually affect client projects. A key lesson is that a breaking change in a library does not mean a broken client: impact depends on what the client uses. This project applies that client-side view to the npm ecosystem and puts it in the upgrade pull request.

What you build

  • Developers see a comment on each upgrade pull request: "3 of 21 breaking changes affect this project", followed by each change, the files and lines, and the old and new signatures.
  • Teams see when an upgrade has no impact at all and can merge with more confidence once tests pass.
  • Maintainers of libraries can use the public study results to see which of their changes caused the most pain.

The technically interesting part

  • Extracting an API from declarations. Use the TypeScript compiler API to load a package's .d.ts files, follow re-exports and build a normalised model of every exported symbol and its type.
  • Diffing types properly. A parameter that changes from string | number to string is breaking for callers who pass numbers; a return type that widens can break callers too. Classify changes by direction and explain each.
  • Mapping usage. Resolve every import in the client, including namespace imports, re-exports through barrel files and destructuring, and record which symbols and members are used and how.
  • Accuracy you can measure. For each upgrade in your study, actually install the new version, build and run tests, and compare reality with your prediction.

How to evaluate it

  • Prediction quality: for 50 or more real upgrades, precision and recall of predicted affected call sites against compiler errors and test failures after the upgrade.
  • Missed impact: categorise breaking changes your tool missed, such as behaviour changes with identical types, and report them honestly.
  • Speed: time per pull request with and without the API snapshot cache.
  • Usefulness: show the reports to three or four developers and ask whether they would merge sooner.

Suggested plan

  • Weeks 1 to 4: study the TypeScript compiler API; extract and print the API of a few packages.
  • Weeks 5 to 10: the API differ with change classification and tests.
  • Weeks 11 to 15: the usage scanner and a first impact report.
  • Semester 2, weeks 1 to 5: GitHub Action, caching and pull request comments.
  • Semester 2, weeks 6 to 12: the study on real projects and upgrades.
  • Semester 2, weeks 13 to 15: report and demo.

Path to a product

Dependency maintenance is a constant cost for every software team, and security tools keep raising the pressure to upgrade. A free GitHub Action for open source with a paid tier for private repositories and organisation-wide dashboards is a well-known model. The study results are publishable and the project shows real compiler and tooling depth.

Scope warning

Support TypeScript and npm only; JavaScript packages without types can be a stretch goal. Do not try to detect behaviour changes from code; report type-level changes and clearly say what the tool cannot see. Never present the report as a guarantee that an upgrade is safe.

Learn what you need on Sefism

Get help with this idea

The risk is a tool that is right about types but wrong about what matters. Tauseef can help you scope the type diff, design the upgrade study and present the results in a way teams trust. If you are new here, create a free account first.

Ideas and guidance, not finished projects

These are project ideas and scoping guidance, published free for students to use as a starting point. I do not build, write, or sell final-year projects, and I do not complete coursework for anyone. Take an idea, make it yours, and build it.

Not sure this one fits you?

I help students pick an idea that matches their skills and their deadline, then scope it down until it is finishable. Guidance only, never done for you.

Discuss this idea
Work with me

Stuck on something specific?

Writing only gets you so far. If you want an answer to your situation rather than the general case, book a session and we will work through it together. Sessions are free for approved Sefism members, and a few slots open each week.

Book a free sessionSee all services

Follow along

New writing, resources and project ideas land here first.