Sefism is free for learners. We are here to help you grow, one step at a time.Join for free
Sefism.
Security

Model File Supply Chain Scanner for Open Model Downloads with an AI Bill of Materials

Build a scanner that teams run before loading an open model or dataset they downloaded. It flags unsafe serialisation formats such as pickle, reads the imports a pickle would perform without executing it, checks provenance, licence and model card completeness, pins exact revisions and produces an AI bill of materials for the project. It brings the discipline of dependency scanning to the AI supply chain.

Topics
Difficulty

Intermediate

A small team, or one strong student willing to learn something new.
Effort

1 semester · 2 students

Deliverables

7 to ship

4 optional extras

Suggested stack

Pythonpickletools and the safetensors libraryHugging Face Hub APISQLite or PostgreSQLTyper for the command lineGitHub Actions
A suggestion, not a requirement. Swap anything for what you already know.

What you should ship

  • A command-line scanner that takes a local folder or a model repository reference and lists every model and data file with its format
  • Static inspection of pickle-based files that lists the imports and calls they would perform, without ever loading them, and flags anything outside an allow list
  • Checks for safer alternatives: whether a safetensors version of the same weights exists, and a recommendation to use it
  • Provenance and metadata checks: publisher, exact revision pinned, licence present and known, model card sections present (intended use, limitations, training data summary)
  • An AI bill of materials for a project, listing models, datasets and adapters with versions, sources, licences and scan results, in a documented JSON format
  • A GitHub Actions step that fails a build when a project adds an unpinned or unsafe model file
  • A test corpus of harmless sample files you create yourself, covering safe and unsafe formats, with the scanner's results on each

If you have time left

  • Export of the AI bill of materials in a CycloneDX-compatible form
  • Licence compatibility warnings for commercial use
  • A small web dashboard showing every project's models and their scan status
  • Drift alerts when a pinned model's upstream repository changes

The problem: downloading a model is running someone else's code

Teams now pull open models, adapters and datasets from public hubs as casually as they install packages. But some model formats are not just data. Python's pickle format, long the default for PyTorch weights, can execute code when a file is loaded. Hugging Face's own documentation warns about this, describes its scanning of pickle imports on the Hub, and points to safetensors as a simple format designed to store weights without that risk.

Supply Chain is entry LLM03 in the OWASP Top 10 for LLM Applications (2025). Yet many student and startup projects have no record of which models they use, from whom, at which revision or under which licence.

What already exists and the gap

  • Hugging Face Hub scans uploaded files, including listing the imports in pickle files, and promotes the safetensors format.
  • ModelScan, an open source tool from Protect AI (acquired by Palo Alto Networks in 2025), scans model files for serialisation attacks.
  • picklescan and fickling (from Trail of Bits) are open source tools for analysing pickle files.
  • Software composition analysis tools such as Trivy and Dependabot cover code dependencies but not model files and model cards.

The gap for an FYP is a single, local-first tool that combines file safety, provenance, licence and documentation checks for everything AI in a project, pins revisions, and outputs an AI bill of materials a team can review and put in CI.

Inspiration

Software bills of materials (SBOMs) became standard practice so teams could answer "are we affected?" within minutes of a new vulnerability. This project applies the same idea to models and datasets, using the Hugging Face pickle scanning documentation and the OWASP supply chain guidance as the starting point.

What you build

  • A scanner for model and data files that never loads them.
  • Metadata checks against the hub's API for publisher, revision, licence and model card.
  • An AI bill of materials for each project.
  • A CI step so unsafe or unpinned models cannot slip in unnoticed.

Build your test corpus yourself from harmless files that demonstrate each format and flag, and never download or share real malicious samples.

The technically interesting part

  • Reading without running. Walk pickle opcodes with pickletools to list imports and calls, and design an allow list that keeps false positives low for normal PyTorch files.
  • Format detection. Recognise formats from their contents, not only file extensions, including archives that contain pickles.
  • Provenance. Resolve a model reference to an exact commit, and record it, so later scans compare like with like.
  • Useful output. A finding should say what the risk is and what to do: switch to safetensors, pin a revision, or review a licence.

How you will evaluate it

  • Detection results on your test corpus, with false positives and false negatives explained.
  • A scan of 50 popular public models' metadata (without loading any files) summarising formats, licences and model card completeness.
  • Time to scan a typical project and how that fits into CI.

Path to a product

As regulation and customer questionnaires ask companies what AI components they use, an AI bill of materials becomes a compliance need as well as a security one. A free command-line tool with a paid dashboard for many projects, or integration into existing SBOM platforms, are both realistic routes. The project is also strong evidence for AppSec, ML platform and AI security roles.

Scope warning

Never execute model files during scanning, and never collect or distribute malicious samples. Do not try to detect backdoors inside weights; that is a research problem. Keep dashboards as a stretch goal.

Learn what you need on Sefism

  • Cybersecurity roadmap: choose the AI security path for how LLM apps fail, the OWASP LLM risks with their defences, and testing and guardrails.
  • Cybersecurity cheat sheet: the AI security edition covers prompt injection, excessive agency, output handling, the AI supply chain and security evals in a line or two each.
  • OWASP Top 10 for LLM Applications: the 2025 list your project maps its checks to.
  • MITRE ATLAS: techniques, mitigations and case studies to cite in your threat model.
  • garak: once a model passes supply chain checks, see how its behaviour can be scanned.
  • Hugging Face Hub: model cards, licences and file formats in practice.
  • NIST AI Risk Management Framework: connect your bill of materials to the Map function.

Get help with this idea

The subtle part of this project is keeping false alarms low while never loading a file. Tauseef can help you scope the checks, build a safe test corpus and design output developers will actually act on. If you are new here, create a free account first.

Ideas and guidance, not finished projects

These are project ideas and scoping guidance, published free for students to use as a starting point. I do not build, write, or sell final-year projects, and I do not complete coursework for anyone. Take an idea, make it yours, and build it.

Not sure this one fits you?

I help students pick an idea that matches their skills and their deadline, then scope it down until it is finishable. Guidance only, never done for you.

Discuss this idea
Work with me

Stuck on something specific?

Writing only gets you so far. If you want an answer to your situation rather than the general case, book a session and we will work through it together. Sessions are free for approved Sefism members, and a few slots open each week.

Book a free sessionSee all services

Follow along

New writing, resources and project ideas land here first.