Interpreting reports

A Synthetic Users report is qualitative user research, not a test result. Treat it that way. It surfaces signal; you make the call.

The report sections

Persona

Reminds the reader who they’re hearing from. Skim it — if the report’s analysis doesn’t match this persona, you’re in a synthesis failure case (rare; file a bug).

Journey

A step-by-step narrative of what the synthetic user did and saw. Useful for:

Confirming the agent actually exercised the flow you care about.
Finding the moment a confusion point first appeared.

Confusion points

These are the highest-signal artifact. Each entry includes:

What the user expected vs found.
Where in the flow they hit it.
A short quote in the persona’s voice.

A confusion point that 2+ personas independently surface is almost certainly a real UX issue. Look for those in the aggregate report.

Quotes

Verbatim first-person reactions. These are deliberately persona-voiced — the synthesis pass is constrained to pull from react() tool outputs, not fabricate them (persona voice). Use these in slides and tickets; they tend to be the most quotable artifact.

Questions explored

Only appears when guided_exploration is standard or deep. These are not findings by themselves; they show which sidecar questions the browser-driving persona answered or dismissed, with the evidence text that closed each one.

What to act on

Signal	Action
One persona, one confusion point, single session	Note it. Don’t ship a fix on n=1.
Same confusion point across 2+ personas	Real issue. File and fix.
Persona stuck on the same step across runs	Critical-path UX defect.
”Would sign up” = false across all personas	Re-examine your value prop and landing.
Quotes contradicting each other across personas	Your audience is segmented. Pick one.

What to ignore

Persona over-reach. If a “non-technical small-business owner” objects to your developer terminology, that’s signal about your audience fit, not necessarily about your UX. Confirm with a buyer-persona run.
Single-quote drama. One sharp quote ≠ a finding. We optimise for voice density; you optimise for pattern detection.

When the report doesn’t look right

Empty confusion points + happy quotes on a target you know is broken → check termination_kind in the JSON report. If synthesis_failure, file a bug. If voluntary_finish the agent genuinely didn’t get stuck.
Same quote in multiple slots → known limitation; we deduplicate but the synthesis can echo. The transcript is the source of truth.
Open or dismissed guided questions → read them as investigation trail, not report conclusions. Findings still need browser evidence and persona reactions.