#agent-testing
Red-teaming an agent, without writing a single attack: a walkthrough
We added promptfoo's red-team thinking to Eval Studio by fusing it into the eval spine instead of bolting it on — our leak-veto was already an attack grade. This post is the concrete walkthrough: pick attack plugins from a library, run them against your agent, and read a matrix that says 'held against this probe' — never 'safe'. Built and merged, not yet deployed.
An access-control eval that actually arrives as the user
An access-control test is worthless unless the request arrives as the role being tested. So in Eval Studio each role carries a real credential — a pasted revocable token, or one minted against your own endpoint — and a cross-role leak-veto runs as an engine default, not an opt-in. This post walks through setting one up. Built and merged, not yet deployed.
One run, many axes — and no winner we can't defend
A Sweep varies model, prompt, language, and tenant over one suite and lays them out in a single matrix. The hard part isn't the cartesian — it's refusing the green 'winner' that's really just noise. This post walks through running a Sweep: pick your axes, read the cost gate, and read a matrix that calls a tie a tie. Built and merged, not yet deployed.
Evals for the QA who owns quality but doesn't write YAML
More teams are shipping agentic features than know how to test them — and the one tool that can, promptfoo, makes you write evals as code. Eval Studio brings that capability to the person whose job is quality, not the person who happens to write Python. This is the first post in a series on how we built it.