Would you ship a model with 94% accuracy?
Choose a release candidate and state what this test still cannot prove.
“There is a clear gap between benchmark scores and real-world performance.” (opens the source)
The question
KestrelOps drafts maintenance answers for field technicians. Two candidate pipelines ran on the same 200 held-out tasks. Each task has one human-adjudicated primary label. A critical error gives an unsafe instruction or exposes another customer’s data.
| Correct | Noncritical error | Critical error | |
|---|---|---|---|
| Fast pipeline | 188 | 8 | 4 |
| Evidence pipeline | 184 | 16 | 0 |
200 tasks per pipeline. Critical errors are disjoint from noncritical errors.
At least 90% correct, zero observed critical errors, and a named fallback for missing evidence. This sample is a release check, not a guarantee of zero population risk.
“Choose a release candidate and state what this test still cannot prove.”
Not saved by this site. Your browser may restore text when you navigate back.
A strong answer, step by step
Open each step in turn. This is one strong answer, not the only one.
Step 1: Start with the contract
What would you do here? Open the move you’d make.
Choose the 94% pipeline.
Tempting, but it misses something. Accuracy alone ignores the stated severity limit.
Apply the critical-error gate before ranking accuracy.
The strong move. An average does not cancel a veto condition.
See how a strong candidate says it
“The fast pipeline is 94% correct, but its four critical failures violate the gate. The evidence pipeline is 92% correct and meets these observed quality limits.”
Why it works: An average does not cancel a veto condition.
Step 2: Inspect the failures
What would you do here? Open the move you’d make.
Call the four failures random noise.
Tempting, but it misses something. Discarding failures because the sample is small biases the decision.
Review the four critical failures and their task slices.
The strong move. A failure taxonomy suggests fixes and exposes missing coverage.
See how a strong candidate says it
“I would inspect the evidence, permissions, and output for each critical miss, then add related cases without editing this held-out result.”
Why it works: A failure taxonomy suggests fixes and exposes missing coverage.
Step 3: Separate tuning from testing
What would you do here? Open the move you’d make.
Keep editing until these 200 tasks pass.
Tempting, but it misses something. A perfect score on reused tasks may measure memorization of the test.
Tune on a development set and reserve a fresh check.
The strong move. Repeated tuning against the gate leaks test information.
See how a strong candidate says it
“I would preserve this run, tune on separate cases, and test the candidate on untouched cases plus regression tests.”
Why it works: Repeated tuning against the gate leaks test information.
Step 4: Bound the release
What would you do here? Open the move you’d make.
Declare the evidence pipeline safe.
Tempting, but it misses something. A finite sample cannot establish universal safety.
Start with a monitored, reversible release and an evidence fallback.
The strong move. Zero observed critical failures is bounded evidence, not proof of zero risk.
See how a strong candidate says it
“I would release the evidence pipeline only to a limited scope, route unsupported questions to a technician, and monitor critical failures with a stop rule.”
Why it works: Zero observed critical failures is bounded evidence, not proof of zero risk.
The answer, in short
- Decision
- Choose the evidence pipeline for a limited release.
- Because
- 184/200 is 92%; the fast pipeline’s 188/200 is 94% but fails the critical-error gate.
- What would change it
- A repaired fast pipeline must pass fresh held-out cases and the critical-failure regressions.
What new evidence would justify removing technician review?
Common traps
- Treating all wrong answers as equally costly.
- Calling zero observed failures a zero-risk system.
Check your answer
- Did I compute both rates?
- Did I apply the critical gate first?
- Did I preserve an untouched test?
- Did I name a fallback and stop rule?
Last reviewed Nov 10, 2026