CLI and library for testing prompts, agents and RAG apps, including red teaming.
CLI and library for testing prompts, agents and RAG apps, including red teaming.
Promptfoo: LLM evals & red teaming
promptfoo is a CLI and library for evaluating and red-teaming LLM-based apps. Stop using trial-and-error... start shipping secure, reliable agents
Website ·
Getting Started ·
Red Teaming ·
Documentation ·
Discord
Promptfoo is now part of OpenAI. Promptfoo remains open source and MIT licensed. Read the company update.
npm install -g promptfoo
promptfoo init --example getting-started
Also available via brew install promptfoo and pip install promptfoo. Or use npx promptfoo@latest to run any command without installing.
Most LLM providers require an API key. Set yours as an environment variable:
export OPENAI_API_KEY=sk-abc123
Run an eval and view results:
cd getting-started
promptfoo eval
promptfoo view
See Getting Started (evals) or Red Teaming (vulnerability scanning) for more!
What can you do with Promptfoo?
• Test your prompts and models with automated evaluations
• Secure your LLM apps with red teaming and vulnerability scanning
• Compare models side-by-side (OpenAI, Anthropic, Azure, Bedrock, Ollama, and more)
• Automate checks in CI/CD
• Review pull requests for LLM-related security and compliance issues with code scanning
• Share results with your team
Here's what it looks like in action:
It works on the command line too:
It also can generate security vulnerability reports:
Why Promptfoo?
• Developer-first: Fast, with features like live reload and caching
• Private: LLM evals run 100% locally - your prompts never leave your machine
• Flexible: Works with any LLM API or programming language
• Battle-tested: Powers LLM apps serving 10M+ users in production
• Data-driven: Make decisions based on metrics, not gut feel
• Open source: MIT licensed, with an active community
• Getting Started
• Full Documentation
• Red Teaming Guide
• CLI Usage
• Node.js Package
• Supported Models
• Code Scanning Guide
We welcome contributions! Check out our contributing guide to get started.
Join our Discord community for help and discussion.
Builder tool · Builder tools / Observability & evals · Open Source · MIT · TypeScript
Website https://www.promptfoo.dev/ · Repo https://github.com/promptfoo/promptfoo · Docs https://www.promptfoo.dev/docs/intro/
27,075 stars · 2,713 forks · 964 open issues · pushed 2026-11-11 · verified 2026-11-12
Tags: ai-agents, tool, observability-evals
What is promptfoo? promptfoo is a coding website listed on Scriptiii Webnest. CLI and library for testing prompts, agents and RAG apps, including red teaming. Pricing: Free. Free to use. Best for: ai-agents, tool, observability-evals.
Everything you need to know about promptfoo
Similar websites in Coding
Open-source vector database that stores objects and vectors with hybrid search and filtering.
Coding·Free