Case study: evals at BenchSci

Scientists used AI to search ~800k scientific publications. The question: "How do you know if the AI is right?"

Scientists needed to trust AI outputs for research decisions

Every answer needed a citation back to its source publication, so scientists could verify and dispute it

Without systematic evaluation, no one could tell whether a change made answers better or worse

I owned evaluation end to end. The key insight: "I look at a rubric as the end-to-end test, and LLM-as-judge as a unit test. Did this unit do well on this input-output within this distribution sample?"

LLM-as-judge and a council of judges. LLM-as-judge evaluators, plus a council of judges whose agreement sets the confidence. 500 test cases across live use patterns.

Cascade evals. One eval's output feeds the next. Arize Phoenix early adopter; OpenTelemetry tracing in production.

Failure cases as signal. 100% accuracy is a red flag. We want the failure cases.

RAG with provenance. RAG over ~800k scientific publications. Neo4j provenance back to the source, so scientists could verify and dispute.

More agents, more failure modes. With multiple agents in a system, the failure modes multiply. Systematic evaluation is the only way to manage that.

Evals drive product direction. Evaluation isn't quality control. It's how you give direction to the product.

Domain literacy matters. You can't evaluate what you don't understand.

Cut LLM latency from 21 s to 4 s by parallelizing Bedrock calls, the day before a demo.

Your AI demo impressed leadership and now has to ship. Or a wrong answer reached a customer, and no one can say how often it happens.