'It Can't Hallucinate' — What That Claim Actually Means

The claim that caused the argument

When TypeSafe launched Jev on September 15, 2026, the Hacker News thread hit 1,941 points and 507 comments. A large share of those comments were about one phrase: that Jev cannot hallucinate.

The argument is worth understanding carefully, because both sides are partly right and the resolution is load-bearing for anyone putting this in production. There is a real, checkable guarantee here. It is also not the guarantee most readers will assume from the word "hallucinate".

What is genuinely guaranteed

Start with what actually holds, because it is not nothing.

Jev's output space is defined entirely by the criteria you send. A Choice returns a probability distribution over exactly the option keys you provided. A Score returns a distribution over exactly the levels in your array. A Noul returns one float in [0, 1]. There is no decoding path that produces anything else.

That rules out a specific and genuinely annoying family of failures:

• No parse errors. No truncated JSON, no markdown fences wrapped around the object, no trailing commentary. There is no parsing step, because there is no text.
• No invented enum values. If your options are billing, technical and sales, you cannot get back billing_and_technical, or Billing, or unknown. The distribution is over your keys.
• No schema drift. Every Choice answer has choice, probabilities and confidence. Every Score adds legend. The shape does not vary with input.
• No silent format regressions on model updates. An alias move can change the answers, but it cannot change the shape.

If you have written the retry-with-a-sterner-prompt loop, or a validator that repairs almost-JSON, or a normalizer that maps seven spellings of a category back onto three, you know how much code this deletes. TypeSafe built something real here.

There is also a second-order benefit that shows up in evaluation work. LangChain's LangSmith team ran Jev against 500 agent-evaluation decisions and found it matched human judgment on all 500, with 92-913x lower variance than the LLM-judge baseline, at $0.34 versus $28.17 for Claude. Variance that low is partly a property of the constrained output space. For an LLM-as-judge harness, run-to-run instability often hurts more than raw accuracy, because it drowns real regressions in noise.

What is not guaranteed

Here is the gap, in one sentence: a constrained output space guarantees the answer is well-formed, not that it is right.

Jev will always return one of your options. Nothing makes it the correct one. Route a billing ticket and get back technical with 0.91 probability and 0.87 confidence, and you have received a type-safe, schema-valid, confidently wrong answer. Your code cannot detect this, because from your code's perspective it is indistinguishable from a correct one.

The HN critics put it more sharply than I can. WhitneyLand:

Type safety is not factual correctness.

InsideOutSanta made the strongest version of the argument, about what the word means to the audience hearing it:

If you claim that "x can't hallucinate" in the context of LLMs, you're saying that x always gives accurate answers. It does not matter whether the answer is type safe.

That is the crux. "Hallucination" is not a term of art for schema violation. It entered the vocabulary describing a model stating something false with apparent confidence: the invented citation, the nonexistent API method, the plausible wrong date. Nothing about Jev's architecture addresses that. A wrong classification delivered with a peaked distribution is exactly the phenomenon the word was coined for, just wearing a type.

seizethecheese landed the analogy that got quoted most:

it's sort of like selling a boat and saying it doesn't need tire changes.

Technically true. Answers a question nobody asked. And the phrasing implies a reliability property the product does not have.

To TypeSafe's credit

The founders did not stonewall this. Diogo Almeida conceded the substantive point directly in the thread:

because these models are probabilistic, it's also possible to be confidently wrong (and all future models will be smarter still and still have that possibility)

That is the correct technical statement, and it concedes exactly what the critics were asking for. It also, notably, contradicts the plain reading of the marketing line.

Almeida also offered a reframe worth taking seriously:

Would you say a linear classifier hallucinates?

This is a better argument than it first appears. We do not say logistic regression hallucinates when it misclassifies. We say it was wrong, or poorly calibrated, or underfit. "Hallucination" carries the specific connotation of fabrication: producing an entity that does not exist. A classifier selecting the wrong label from a fixed set is not fabricating anything. It is erring.

Under that framing, "Jev can't hallucinate" is roughly "Jev is a classifier, and classifiers don't hallucinate, they just get things wrong." Defensible as taxonomy.

The problem is audience. Say "classifier" to an engineer and they correctly hear "this thing has an error rate I need to measure." Say "can't hallucinate" to the same engineer, in a market where that word has one meaning and it is about trusting outputs, and a meaningful number of people will hear "I don't have to verify this." The taxonomic argument is fine. The marketing choice that leaned on it was not, and the vehemence of the thread is the evidence.

It is also a bit of a definitional bait-and-switch. If the answer to "can it hallucinate?" is "that category doesn't apply to this kind of model," then the honest headline is "Jev is a calibrated classifier," not "Jev can't hallucinate." The first is accurate and still impressive. The second borrows credibility from a term whose meaning it then declines to accept.

What replaces the guarantee you thought you had

The useful move is to stop asking whether Jev hallucinates and start asking what your actual error budget is. Jev gives you a genuinely better tool for that than a text-generating model does: the full probability distribution, every time.

Confidence is your uncertainty signal, and it is honest. It is derived from probabilities, so it reflects the shape of the model's answer rather than a self-report. Independent testing supports that it means something. Emil Lindfors (lindfors.no) ran Jev over 24 Norwegian policy documents and found calibration held up: at confidence 0.9 and above, answers were correct about 94% of the time. Notable partly because Norwegian is not Jev's primary training language, and TypeSafe explicitly warns that non-English accuracy is lower.

Note what that finding is and is not. It is evidence that confidence tracks correctness in aggregate. It is not a promise about any single answer. At 0.9+ confidence, roughly 1 in 17 was still wrong.

Gate on confidence, scaled to what the mistake costs. The docs' own worked example does this well:

action = response.answers["action"]
confidence = action.confidence
 
if confidence < 0.5:
    # Model is genuinely unsure. Don't guess.
    route_to_human(user_message)
 
elif action.choice == "check_balance":
    # Low stakes. Showing the wrong screen is recoverable.
    show_balance(account_id)
 
elif action.choice == "approve_transfer":
    if confidence > 0.9:
        # High stakes, high confidence. Proceed with confirmation.
        confirm_then_execute(account_id)
    else:
        # High stakes, moderate confidence. Verify first.
        ask_user_to_confirm(account_id)

Showing the wrong balance screen is recoverable. Approving the wrong transfer is not. Same model, same call, different thresholds, because the threshold belongs to the consequence and not the model.

Design the option set so wrong answers fail safely. Always include an other or none of the above option when your list might not cover every input. Without an escape hatch the probability mass has to go somewhere, and it will go onto your real options. This is the closest thing to a structural defense against confidently wrong output: give the model somewhere honest to put its uncertainty.

Keep verifiable facts in code. Jev's own jaggedness notes say it is unreliable at counting, arithmetic, and date comparison. Anything a parser or a regex can establish should not be a question at all. Reserve the model for the part that is genuinely a judgment.

Be aware that state is not treated as hostile. TypeSafe documents that adversarial content, whether an injected instruction or text arguing for its own classification, can move the answer. If your state contains user-supplied text, that is a threat model you own. Type safety offers zero protection here; the injected text just moves probability mass between your legitimate options.

The fair summary

TypeSafe built something genuinely useful and then described it with a word that promised more than it delivers.

The engineering is real. A constrained output space eliminates an entire class of integration failures, and combined with calibrated probabilities it gives you a better uncertainty signal than parsing confidence out of prose ever did. The variance results and the independent calibration testing both hold up.

The claim oversold one specific thing. "It can't hallucinate" reads as a correctness guarantee to the audience it was aimed at, and the founder's own concession about being confidently wrong is the clearest evidence it is not one.

So use the precise version:

Jev guarantees you will always get a valid answer. It does not guarantee you will get a true one.

That is still worth paying for. It is just a different product than the headline implied, and knowing which one you bought determines whether the thing you build on top of it is sound.

Next: how confidence is actually computed and how to threshold on it.