The new version feels better. Nobody can say how much.
Same cases, same judges, both versions. You get the gap — and whether it is real.
Prove which prompt version is better, then ship that one.
Evalwright versions your prompts, benchmarks them against each other with real statistics, and puts the winner into production behind an approval gate.
Evalwright never calls your model. Resolving returns text, so it stays your provider, your keys, your tokens.
support-summarizer5 versions
- v5In production2 hours ago
Deployed by Dilara after review. Tightened the refund wording.
- v4Approved6 days ago
Served production for six days. Kept in history with its benchmark.
- v3Rejected8 days ago
Lost to v2 on refund cases. The reason is recorded on the request.
v2 and v1 kept in history
v4 → v5Every version keeps its author
From an edit to production, in four steps
Each step leaves a record, so you can always answer why a prompt changed and what the change was worth.
- 01
Version
Every edit forks a new version instead of overwriting the last one. Nothing is lost, and nothing changes under a caller who did not ask for it.
- 02
Benchmark
Run versions against the same test cases and the same judges. You get a paired confidence interval, not a single number — so you can tell a real difference from sampling noise.
- 03
Approve
A production deploy can require a second pair of eyes. The requester cannot approve their own request, and a rejection has to say why.
- 04
Resolve
Your backend asks for a prompt by its id, with a key for one of its project's environments, and gets the text the benchmark ran. Deploying a new version changes what that environment runs, with no release on your side.
What it replaces
Four things that go wrong when a prompt ships on a hunch, and what stands in their place.
The winning version quietly costs four times as much to run.
Score and cost sit side by side, so you can see what the quality is worth.
One judge has a bad day and decides your release.
Several judges vote, and how much they agreed comes back with the score.
Someone edits the live prompt on a Friday afternoon.
Production deploys wait behind an approval gate, and every earlier version stays one deploy away.
One call in your backend
Ask for a prompt by its id and get finished text back, with its variables filled in on your side. Deploying a new version reaches every caller within about 30 seconds.
Read the quickstart// The version deployed to this key's environment.
const SUMMARIZER_ID = "6a1f0c3e9b2d4a7f8e5c1b90"; // the prompt's id
const { prompt, version } = await client.prompts.resolve(SUMMARIZER_ID, {
vars: { ticket: ticket.body },
});