AI Agent Tool Permission Gateway with Human Approval and Audit Log
Build a policy gateway that sits between an AI agent and the tools it can call. It enforces least privilege per tool, checks arguments against rules, acts with the signed-in user's own permissions, asks a person to approve risky actions with a clear preview, caps steps and spend, and writes a tamper-evident log of every call. A practical answer to excessive agency, the OWASP LLM risk that turns a fooled model into real damage.
Advanced
Needs a team, real planning, and a supervisor who knows the area.1 semester · 2 to 3 students
7 to ship
4 optional extrasSuggested stack
What you should ship
- A gateway that exposes tools to an agent (over MCP or function calling) and forwards each call to the real tool only if policy allows it
- A policy file per agent: which tools, which argument rules (allowed recipients, amount limits, read-only paths), which calls need approval, and step and spend caps per task
- Delegated permissions: every tool call runs with the signed-in user's own scoped credential, never a shared admin key
- An approval inbox (web and email or chat notification) that shows exactly what will happen, with approve, edit and reject, and an expiry
- An append-only audit log with hash chaining so tampering is detectable, plus a search view per user, agent and tool
- A demo agent for a realistic task, such as an inbox and calendar assistant, running entirely against test accounts you own
- An evaluation: a set of scenarios where the model is steered towards unsafe actions by planted content in its own test inbox, showing which ones the gateway blocks or routes to approval
If you have time left
- Anomaly alerts when an agent's pattern of tool calls departs from its baseline
- Policy testing: a dry-run mode that replays logged calls against a new policy before it goes live
- Approval fatigue measurement: track how long people take to approve and how often they edit, and tune which actions need approval
- Export of the audit log to a SIEM such as Wazuh
The problem: agents act with too much power
AI agents now read inboxes, update calendars, file tickets and move money. The OWASP Top 10 for LLM Applications (2025) names Excessive Agency as a risk and describes three root causes: too much functionality, too many permissions and too much autonomy. Combined with prompt injection, which has no complete fix, excessive agency is how a manipulated model becomes a sent email, a deleted record or a refund nobody asked for.
Most teams building agents wire tools directly to the model with one broad API key, because it is quick. What they lack is a place in the architecture where rules are enforced in ordinary code, outside the model, whatever the model has been talked into.
What already exists and the gap
- The Model Context Protocol (MCP) standardises how agents connect to tools, but deciding what each tool may do is left to the developer.
- Agent frameworks such as the OpenAI Agents SDK and Google's Agent Development Kit offer guardrail hooks, but only inside that one framework.
- Commercial AI security platforms, such as Palo Alto Networks' Prisma AIRS (which absorbed Protect AI) and Check Point's Lakera, add runtime protection aimed at enterprises.
- The OWASP AI Agent Security Cheat Sheet lists least privilege, human-in-the-loop controls and monitoring as core practices.
The gap for an FYP is a small, framework-neutral gateway that turns those practices into enforceable policy: narrow tools, delegated credentials, argument rules, meaningful approvals and an audit trail you can trust.
Inspiration
The research paper "Design Patterns for Securing LLM Agents against Prompt Injections" (2025) argues for designing agents so that a successful injection cannot cause much harm, rather than hoping to detect every injection. This project builds one concrete piece of that idea: the model can ask for anything, but only calls that policy allows ever reach a tool.
What you build
- The gateway, which receives each tool call, evaluates it against the agent's policy and either forwards it, sends it for approval or refuses it.
- Policies that a developer can read and review like code.
- An approval inbox where the user sees the exact action, such as the recipient, subject and body of an email, before it happens.
- An audit log that records every request, decision and result.
- A demo agent that runs only against test accounts you create.
The technically interesting part
- Delegated credentials. Use scoped OAuth grants so each call acts as the user, and show that the agent cannot exceed what the user could do alone.
- Argument rules. Allow lists for recipients and domains, limits on amounts, and path rules for files, checked in code.
- Meaningful approval. If everything needs approval, people click yes without reading. Decide which actions are risky, show a precise preview and measure how people respond.
- Tamper evidence. Chain log entries with hashes so a deleted or edited entry is detectable.
- Loop and spend caps. Stop runaway tasks, which also addresses the Unbounded Consumption risk.
How you will evaluate it
- A scenario suite where planted content in the agent's own test inbox tries to steer it towards unsafe actions, reporting how many reach a tool with and without the gateway.
- Added latency per tool call.
- A small user study of the approval inbox: time to decide, and how often people catch a wrong action.
Path to a product
Every company deploying agents will need somewhere to enforce and audit what they are allowed to do. A self-hostable gateway with a policy language, approvals and audit export fits small teams that cannot buy enterprise platforms, and it can grow into a hosted product or an open source project with paid support.
Scope warning
Run everything against test accounts and data you own. Do not build a general agent framework; keep one demo agent and a handful of tools. Keep anomaly detection as a stretch goal.
Learn what you need on Sefism
- Cybersecurity roadmap: choose the AI security path for how LLM apps fail, the OWASP LLM risks with their defences, and testing and guardrails.
- Cybersecurity cheat sheet: the AI security edition covers prompt injection, excessive agency, output handling, the AI supply chain and security evals in a line or two each.
- OWASP Top 10 for LLM Applications: the 2025 list your project maps its checks to.
- MITRE ATLAS: techniques, mitigations and case studies to cite in your threat model.
- Model Context Protocol Documentation: how agents discover and call tools, which your gateway sits in front of.
- garak: an example of automated probing you can adapt ideas from for your scenario suite.
- NIST AI Risk Management Framework: place your controls under the Manage function in your report.
Get help with this idea
The risk with this project is building an approval screen people ignore, or a policy language nobody can read. Tauseef can help you choose the tools, design the policy model and plan an evaluation that shows the gateway really limits harm. If you are new here, create a free account first.
Ideas and guidance, not finished projects
These are project ideas and scoping guidance, published free for students to use as a starting point. I do not build, write, or sell final-year projects, and I do not complete coursework for anyone. Take an idea, make it yours, and build it.
Not sure this one fits you?
I help students pick an idea that matches their skills and their deadline, then scope it down until it is finishable. Guidance only, never done for you.
Stuck on something specific?
Writing only gets you so far. If you want an answer to your situation rather than the general case, book a session and we will work through it together. Sessions are free for approved Sefism members, and a few slots open each week.
Follow along
New writing, resources and project ideas land here first.