promptfoo
An open source command line tool for testing LLM applications: define test cases in a config file, compare prompts and models side by side, and run automated red teaming for prompt injection and data leaks. It fits into CI, which turns prompt changes into something you can review like any other code change.
Tool
AI & Machine Learning
Promptfoo
Intermediate
Free
An open source command line tool for testing LLM applications: define test cases in a config file, compare prompts and models side by side, and run automated red teaming for prompt injection and data leaks. It fits into CI, which turns prompt changes into something you can review like any other code change.
Ready to start?
Opens on Promptfoo in a new tab.
Stuck on something specific?
Writing only gets you so far. If you want an answer to your situation rather than the general case, book a session and we will work through it together. Sessions are free for approved Sefism members, and a few slots open each week.
Follow along
New writing, resources and project ideas land here first.