One phone call, stage by stage

This page follows one synthetic example call through a voice agent, stage by stage, and shows how each stage can be measured before a change ships.

  1. +0.00sCaller. Hi, I'd like to move my delivery to Friday.
  2. +1.90sAgent. Sure. What's the order number?
  3. +3.30sCaller. It's 5 5 0 2.
  4. +4.87sAgent. Thanks. It's moved to Friday, between nine and noon.

Turn 2 from end of speech to first audio. Example turn, illustrative timings.

  • Speech-to-text180 ms
  • Retrieval90 ms
  • LLM340 ms
  • Text-to-speech150 ms

760 ms to first audio.

A delivery call. The caller asks to move a delivery, the agent asks for the order number, the caller answers, and the agent confirms the new day. The second turn takes 760 ms from end of speech to first audio in this example: speech-to-text 180 ms, retrieval 90 ms, LLM 340 ms, text-to-speech 150 ms. Illustrative timings.

Every stage is timed

A voice agent that gets slower doesn't throw an error. The caller waits, talks over it, or hangs up. By the time someone hears the problem, nobody knows which change caused it.

This synthetic example shows how a stage-level regression check can identify a slower voice turn.

OpenTelemetry tracing in production, and an early adopter of Arize Phoenix.

  • Speech-to-text180 ms
  • Retrieval90 ms
  • LLM340 ms
  • Text-to-speech750 ms

Gate blocked the release: text-to-speech is 600 ms over its baseline. The other three stages passed.

Example turn with a regression injected into text-to-speech: +600 ms. The per-stage gate blocks the release and names the stage.

The hard parts of phone audio

Callers interrupt. An agent that keeps talking sounds broken. One that stops but keeps its queue finishes the old sentence after the caller is done.

This synthetic example shows how barge-in and queue clearing can handle a caller interrupting the agent.

Agent
Caller
Queued speech
pickup tomorrowafter two.
Queue cleared
The caller talks over the agent. The agent stops, and the speech it had queued is dropped instead of finished.

A caller that never sleeps

Nobody can regression-test a phone agent by calling it after every change. So the calls are scripted.

Evals for voice: a scripted caller that phones the agent and grades the conversation. Every call is measured for turn latency and transcript correctness against a spec sheet.

100% accuracy is a red flag. I want the failure cases.

Example spec sheet12345678
Confirms the caller's identity before discussing an order
First audio within the turn budget
Answer matches the spec sheet
Hands out-of-scope questions to a person

31 passed, 1 failed, 0 pending. Caught: call 6 answered outside the turn budget.

Eight scripted calls graded against an example spec sheet. One check fails: the regression gets caught before release.

Before voice: evals for scientists

Before voice, I owned evaluation end to end at BenchSci. Read the case study.

On a separate engagement, I cut LLM latency from 21 s to 4 s by parallelizing Bedrock calls, the day before a demo.

Manual validation time40%

BenchSci. Down 60%.

LLM latency4 s

A separate engagement: Bedrock calls run in parallel, the day before a demo.

Two measured results. Manual validation time at BenchSci, down 60%. LLM latency on a separate engagement, 21 s to 4 s.

Work with me

The two ways to work together, and how an engagement starts, are on the work page.

Bring one AI feature you can't measure yet.

Book a 30-minute discovery call