Speculative Fan-Out
Ask everything you might need in one call. Discarding answers is cheaper than a second round trip.
Because questions evaluate in parallel and output tokens are free, the usual instinct to ask only what you need is backwards. Ask everything you might need, then let your code decide what was relevant.
A ticket triage call can carry the category question alongside bug severity, reproducible steps, refund intent, and frustration. If the ticket turns out to be a feature request, you discard the severity answer. You have paid almost nothing for the option.
Interactive
Question builder
match statement · returns choice + probabilities + confidence
sort key · returns an expectation, which can land between levels
if statement · returns one 0–1 value, no confidence field
{
"state": "Hi, I placed an order (#98423) last Thursday and was charged twice. I also can't log in after the site update, and adding Apple Pay would be really helpful. This is getting frustrating.",
"model": "jev-1.13.0",
"questions": {
"category": {
"type": "choice",
"instructions": "Determine the broad category of this support ticket",
"criteria": {
"bug_report": "The user is reporting something that is broken or producing errors",
"billing": "Charges, invoices, refunds, subscriptions",
"feature_request": "The user is requesting new functionality",
"account": "Login, permissions, profile, security"
}
},
"frustration": {
"type": "score",
"instructions": "How frustrated the user appears",
"criteria": [
"Calm, matter-of-fact",
"Frustrated but civil",
"Very angry"
]
},
"refund_requested": {
"type": "noul",
"instructions": "The user is explicitly asking for a refund or credit"
}
}
}Every question here is evaluated in parallel against the same state, so adding one barely moves your response time. That is what makes speculative fan-out cheap.
Where it stops being free
Input tokens are still billed, and every question shares the 64k request budget with the state. Fan-out is cheap in latency, not infinitely cheap. The real limit is usually the 32k ceiling on state plus the longest single question.