Run regression simulations

Supported engine

Automated simulation suites currently support the Mistral pipeline. The editor directs Gemini Live and GPT Realtime agents to individual text or audio tests instead. An unavailable suite option does not mean those engines cannot run an editor test.

Prepare a suite

1. Open the agent's Simulation area.
2. Create a suite and give it a recognizable name, such as “Receptionist booking regression”.
3. Add scenarios with caller instructions, expected behavior and the appropriate language.
4. Review run configuration and any linked evaluation rubric.
5. Save, choose the intended agent version and launch the run.
6. Open the run to inspect each case, transcript, errors and available voice metrics.

Suites use provider APIs for the model and optional speech measurements, while tools, appointments, transfers and SMS use fixtures. They do not execute live external actions. Voice mode measures speech synthesis and transcription through phone-format audio without placing a carrier call. A suite cannot prove that your real booking credentials or phone route work.

Build a useful scenario set

Include a normal booking, no availability, a date correction, cancellation, rescheduling, urgent escalation, a caller who declines consent, an unsupported language and a tool timeout. For a long conversation, mention an early fact and ask for it again near the end.

Evaluate the exact outcome. A fluent “done” response is not a booking receipt. Check that the correct tool ran once, with the right clinic, contact, timezone and slot. When a simulated tool requires a second availability result, supply a valid fixture for that step.

A queued or running case is not a pass. Wait for completion, investigate failures and rerun only after correcting the configuration or fixture. Save the accepted version so later changes can be compared against it.