Agent Eval Kit - 120-case failure-mode library + CLI + CI gate

Find exactly where your AI agent breaks - before your users do.

The same evaluation library behind our 48-hour audit, packaged to run yourself:

• The full 120-case YAML library across 15 failure categories: tool use, refusals, safety, prompt-injection resistance, extraction, code, long context, multilingual, data analysis, multi-turn, grounding, and more
• The agentproof CLI: point it at your agent endpoint, get an HTML report plus JSON/JUnit/CSV output
• A ready-made GitHub Actions CI gate that fails your build when the pass rate drops
• Every case ships with a repro you can replay in one command
• Docs: quickstart, case schema, writing your own cases, judge setup

Requirements: Python 3.11+, an HTTP endpoint for your agent. No subscription, no account, no tracking. One payment, the library is yours.

Free sample first: the 15-case starter pack runs against the included mock agent with zero API keys.