Numbers from AI are claims, not facts: why schema checks are not enough

You must now treat every numerical output from an AI system as an unverified claim. Schema validation alone will not catch the errors that matter. You need a documented chain of checks — semantic, relational, and human — before any figure enters a decision.

The prompt is an analysis of 10 August 2026 of structured outputs from large language models that pass schema validation yet contain semantic errors, which argues that formal correctness of data structure masks substantive errors in the numbers themselves. A benchmark study from July 2026 tested JSON, SQL and record formats and found that between 39 and 54 per cent of outputs contained at least one semantic hallucination, with schema validation catching most syntactic errors but missing around 8 per cent of semantic mistakes — particularly in type coercion and relational inconsistencies. In our assessment, this means you cannot rely on a passing schema check as evidence that a number is correct, and you must build a visible, auditable chain of validation before accepting any figure from an AI system into your workflow.

Why does a structurally valid output still contain wrong numbers?

Schema validation tests form: are the required fields present, are the types correct, is the structure valid? That work is necessary but incomplete. It says nothing about whether the number in a cell is the right number, whether an aggregation is correct, or whether rows and columns are consistent with each other. A JSON object can be perfectly formed and still contain a hallucinated figure. The StructHallu-Drift benchmark demonstrates that this is where errors concentrate — not in syntax, but in the relationship between fields and the semantic accuracy of the values themselves.

A June 2026 study on table reading showed that large language models systematically extract the wrong numbers from tables: they read the wrong cell, ignore relevant values or refer to values that do not exist. When researchers added a specialised critic model that checked answers against the underlying table, accuracy in data-referencing tasks rose by up to 12 percentage points. This indicates that a single model without a checking layer is structurally vulnerable to this class of error.

What does the evidence show about error frequency?

An analysis of 2,500 published AI papers found an average of 4.7 objective errors per paper, including incorrect calculations in tables and inconsistencies between text and table. Human experts confirmed 83.2 per cent of the errors reported by an automated Correctness Checker. Whilst this study examined published papers rather than production systems, it serves as an indication of how error-prone formulas and tables are across the field. The fact that automated checking found errors at this scale, and that human review confirmed most of them, shows both that checking is feasible and that no single layer catches everything.

Which concrete controls do you need to demonstrate?

1. Document which model generated each output — record the model name, version and date for every numerical result or table that enters your workflow.
2. Run schema validation as a first gate — confirm that the structure is valid before any semantic review begins.
3. Apply semantic and relational checks — verify that values are consistent with each other and that aggregations are correct, not just that fields are present.
4. Execute formulas and queries independently — run calculations against the source data to confirm the result, do not assume the AI's working is correct.
5. Route outputs through a critic model — have a separate system check data references against the original source before the output is used.
6. Retain a visible audit trail — record which checks were applied, which passed and which failed, so that later review can trace how a figure was validated.

What happens when users try to verify AI output themselves?

A study tracking twelve spreadsheet users found that all participants stated they never fully trust generative AI output. They combined five strategies: manual checks in the spreadsheet, follow-up questions to the same model, checking questions to other AI systems, verification by sighted people and testing against their own domain knowledge. However, about half of the observed errors went unnoticed, especially in visual elements such as charts, conditions and formatting. Even multiple verification strategies failed to catch everything. This is a sobering lesson: human vigilance alone is not assurance, and precisely for that reason the checks that have been carried out must be made explicit and visible rather than left in the user's head.

What role can tooling play in this chain?

Providers are now building checking into their products. A June 2026 update announced a feature that lets users have formula errors analysed and corrected in one click, with the AI able to analyse the error, explain the cause and propose a corrected formula. The announcement is explicit about the limit: it is a tool for error detection, not an assurance of correct outcomes. Responsibility for correct use and interpretation lies with the user. This means that organisations themselves must determine how an AI suggestion for a calculation is weighed before it counts in decision-making.

Tooling can make the checking steps visible and auditable. It can route a task via multiple independent models, show which model generated which output, record which validation steps were carried out and where something was corrected. It can support the chain of review and control. What tooling cannot do is remove the risk of hallucination or assure correctness. The professional final judgement on the figures always remains with you.