ResearchPractical AIProjectsOpen SourceAbout
Research/When the Answer Isn't in the Contract, a Small AI Makes One Up 6% of the Time
Information ExtractionContract ReviewHallucinationLocal AIPre-RegisteredOriginal Research

When the Answer Isn't in the Contract, a Small AI Makes One Up 6% of the Time

Part 3 of 6. Pulling dates and terms out of contracts is a common AI job, and most of the time the field is simply not in the excerpt. The baseline already instructed the model to use null when a field was absent; a JSON schema did not materially change the result, and a stricter rule that required a quoted source cost accuracy without reducing the invention.

Bart Porcincula · Published · Updated

Finding

On 315 contract-field instances where the answer was absent from the excerpt, Qwen3.5-9B produced a value 6.3% of the time (11% for governing law); the same prompt with a JSON schema gave 6.7%, requiring a quoted source or null gave 6.0% while cutting recall on present fields by 6.7 points, and a literal substring check on the quote caught 0 of 19 inventions.

Omit versus invent on absent contract fields (Oct 2026)

Short contract excerpts from the CUAD dataset with five fields to extract (governing law, agreement date, expiration date, renewal term, notice period). 315 field instances were absent from their excerpt. Four arms: a baseline prompt, the same prompt with a JSON schema requiring the value field, one that requires a null or a verbatim quote as evidence, and the same plus a code check that the quote is literally in the text. Context window 2048 tokens. Paired bootstrap 95% intervals against the baseline.

MeasurementValueNote
Invention rate on absent fields, baseline6.3% (20 of 315)the model returned a value where the true answer was nothing
Invention by field, baselinegoverning law 11% (10/93); agreement date 6%; expiration 4%; renewal 4%; notice period 0%single digits to low tens of cases; the ranking is suggestive
Invention with the JSON schema6.7% (21 of 315)+0.3 pt vs baseline [0.0, +1.0]
Invention when a quote or null is required6.0% (19 of 315)-0.3 pt [-1.9, +1.3], spans zero
Recall on fields that are present: baseline vs quote-required94.0% vs 87.3%-6.7 pt [-9.5, -3.8]
Exact-match accuracy: baseline vs quote-required91.7% vs 76.5%
Inventions caught by a substring check on the quote0 of 19it also removed 3 correct answers
Unparseable outputs0

The short version

Ask a model to pull the governing law, the renewal term and the notice period out of a contract, and some of those are not in the part you gave it. The right answer is "not present". I tested whether a small local model says that, and whether any cheap fix gets it to. All six use the same setup: the open Qwen3.5-9B model compressed to 4 bits, run through llama.cpp on one 16 GB RTX 5060 Ti, at temperature 0 (the setting that makes a model as repeatable as it gets), on contract excerpts (a public contract dataset). Each experiment's questions, thresholds and analysis were written down before it was run.

  • With the baseline instruction to use null when the field is absent, it returned a value for 6.3% of the missing fields. For governing law it was 11%.
  • Adding a JSON schema requiring the value field changed little: 6.7%.
  • Requiring a quoted source for every answer, or null, did not reduce invention (6.0%) and made the model worse at the fields that were there: recall fell from 94.0% to 87.3%.
  • A code check that the quote really appears in the contract caught none of the 19 inventions. The invented values came with a real quote, just a quote from nearby text that does not answer the question.

Takeaway: "invent" here is not making up text. It is reading a nearby sentence as if it answered the question. A permission slip in the prompt does not fix that, and a check that the cited quote exists will pass it. If missing fields matter, you need a check that the quote actually supports that field, or a person.

How I tested it

I used contract excerpts and picked fields so that roughly half were present and half were genuinely absent, then spot-read samples to check the gold answers. A field counts as "invented" only when the true answer is nothing and the model returned a value. There were four arms:

ArmWhat the model is told
Baselineextract the field; use null if it is absent
JSON schemathe same prompt with a JSON schema requiring the value field
Evidence requiredevery answer must carry a verbatim quote, otherwise null
Evidence + checkthe same, then code verifies the quote is literally in the text

Results

ArmInvention on absent fieldsRecall on present fieldsExact match
Baseline6.3% (20/315)94.0%91.7%
JSON schema6.7% (21/315)94.6%92.1%
Evidence required6.0% (19/315)87.3%76.5%
Evidence + check6.0% (19/315)86.3%75.6%

By field, in the baseline: governing law 11% (10 of 93), agreement date 6%, expiration date 4%, renewal term 4%, notice period 0%.

Two things to notice. Both baseline and the schema arm told the model to use null for absent fields; the comparison does not isolate an instruction to omit. And the evidence rule traded real accuracy for nothing: when the model has to quote, it hedges on fields that are there, and still invents on fields that are not.

Why the substring check caught nothing

I expected requiring a quote to expose the inventions, since a made-up value should have no source. It did not, because the model does not make up values from nowhere. It finds a real sentence that mentions a date or a place and treats it as the answer. The quote is genuine; the inference is wrong. A check that the quote exists is a check on the wrong thing.

What I would not claim

  • Twenty inventions is a small count. The per-field ranking rests on single digits to low tens.
  • One model, one dataset, short excerpts at a 2048-token context. Longer documents give more nearby text to misread.
  • Contract excerpts are cut at arbitrary boundaries, so some "absent" fields may sit just outside the excerpt. I removed the worst of these in a read-through but not all.

Part 3 of 6. Previous: one appended sentence. Next: the code check that stops matching.

Can I rely on an AI-filled document field without checking the source?

This experiment gives a reason to check: the model sometimes supplied a value when the answer was absent from the excerpt, and asking for a quote did not resolve the problem. The invention rate applies to absent fields in this contract test, not all document answers. Keep the relevant source available to the reviewer and measure missing answers separately from invented ones.

Cite this research

Bart Porcincula. "When the Answer Isn't in the Contract, a Small AI Makes One Up 6% of the Time." Published 2026-10-29; updated 2026-11-08. https://porsync.com/research/ai-invents-values-for-missing-contract-fields

Permanent article URL