Confidence scores from AI models: why a high percentage is no proof of correctness

You cannot treat a model's reported confidence score as evidence of correctness. The figure is shaped more by how you phrase your question than by whether the answer is right, and it often stays high even when the model is wrong. You must establish which uncertainty signal is reliable enough for your workflow, under what conditions, and with what human control around it.

An analysis of 4 September 2026 of confidence scores and their reliability as uncertainty signals argues that a model's self-reported certainty is heavily influenced by prompt wording and remains poorly calibrated to actual correctness. The research draws on studies examining how models verbalise confidence and where those estimates fail in practice. In our assessment, the visibility of a percentage in an interface creates an impression of rigour that the underlying signal does not always support; confidence is an interface element, not evidence.

What makes a single model's confidence score unreliable?

When you ask a model to attach a certainty percentage to its own answer, that figure is weakly connected to whether the answer is actually correct. Research shows that verbalised confidence scores are heavily quantised—they cluster around round numbers—and they often remain high even when the model produces a wrong answer. The score is also sensitive to how you phrase the question itself; rewording the prompt can shift the reported confidence without changing the underlying model or the factual basis of the answer. A single percentage from one model therefore cannot serve as your primary signal that an answer is trustworthy.

Which uncertainty signals are stronger than a model's self-report?

Disagreement between independent models captures uncertainty better than a single model queried repeatedly. When different models diverge on an answer, that divergence signals that the question is difficult or ambiguous—a more reliable indicator than any one model's confidence claim. Disagreement does not guarantee correctness; it indicates that the question warrants closer inspection. Task-specific calibration also matters: the same uncertainty signal may be well calibrated in one domain—say, legal research—and poorly calibrated in another, such as numerical analysis. You cannot validate confidence once and then trust it blindly across all your workflows.

What failure modes must you guard against?

• Overconfidence on wrong answers — models report high certainty even when their output is factually incorrect.
• Prompt sensitivity — the wording of your question shapes the confidence score more than the question's actual difficulty.
• Domain drift — uncertainty estimates calibrated for one specialism do not transfer reliably to another.
• Interface trust — a visible percentage creates an impression of reliability that the underlying signal may not support.
• Single-model reliance — querying one model repeatedly does not improve the signal; independent models are needed.

What controls must you demonstrate for high-risk workflows?

1. Establish which signal you will act on — define per task type which uncertainty measure (single-model confidence, multi-model disagreement, or domain-specific calibration) you treat as actionable.
2. Calibrate per domain and question type — test your chosen signal against known outcomes in each specialism you handle before deploying it.
3. Route uncertain answers to human review — document the threshold at which an answer moves from automated output to human inspection, and log which answers crossed that threshold.
4. Use independent models where disagreement matters — if you rely on multi-model signals, ensure the models are genuinely independent and not fine-tuned variants of the same base.
5. Separate confidence from correctness in your audit trail — record which confidence signal was reported, which signal you acted on, and whether the outcome was correct, so you can measure calibration over time.

What can tooling do, and what remains your responsibility?

A verification layer can route tasks through selected independent models, make disagreement and sources visible for inspection, and flag when an answer should move to human control. It cannot guarantee correctness and does not remove the need to check for hallucinations. The professional judgement—which signal to trust, when to override it, and what threshold of uncertainty warrants human review—remains yours. Tooling can make the uncertainty visible and structured; it cannot replace your assessment of what is reliable enough to act on in your context.