# When “Valid JSON” Is Not the Same as Valid Data
## Introduction
For months, our automated intake system for patient discharge summaries ran flawlessly. Every document came back as clean structured data: medications listed, dosages confirmed, follow-up dates recorded. No errors, no exceptions, no malformed records. Our QA dashboard glowed green.
Then a clinical auditor flagged something unsettling. Three patients across two months had follow-up dates listed as belonging to dates that hadn’t happened yet at the time of their discharge. These weren’t crashes, parsing failures, or obvious data corruption. The JSON was perfectly valid. The types were correct. The fields were populated with confident, specific values. The values were just entirely fabricated.
That experience reshaped how I think about structured output from language models. And I think it should reshape how you think about it too.
—
## The Illusion of Completeness
Structured output formats from large language models solve a genuine and long-standing engineering headache. Before native schema enforcement, extracting reliable data from unstructured text meant building elaborate regex pipelines, writing defensive retry logic, and crafting prompts that begged the model not to include any markdown formatting or conversational filler around the JSON payload.
When you define a schema and feed it to a model with structured output support, the response arrives as a valid, type-checked object. Every key is present. Every value matches the declared type. You can pass it straight into your database, your downstream pipeline, or your business logic without writing a single line of defensive parsing code.
This is a real achievement. But it is not the whole achievement.
The moment I stopped treating “the data is structurally sound” as the finish line, I began to notice a class of errors that no schema validator will ever catch. They are not crashes. They are not malformed records. They are records that look completely reasonable, pass every type check, and contain values that simply do not correspond to anything in the source material.
—
## The Hallucination Problem Inside Your Schema
Consider a discharge summary that mentions a medication and dosage but never specifies when the patient should follow up. If your schema declares `follow_up_date` as a required field of type date, the model faces a dilemma. It cannot return nothing—that would violate the schema. So it reaches for a value anyway.
What value does it reach for? In my experience, it tends to be a plausible-looking date: the current date, a date a few days forward, a date that aligns with typical clinical timelines. The result is a record that is structurally perfect and clinically dangerous.
This is not a bug in the model. It is a fundamental mismatch between what a schema guarantees and what a schema promises. A schema guarantees shape. It says nothing about whether the content within that shape is real, derived, or invented.
When I first understood this, I realized I had been conflating two entirely different concepts: extraction and inference.
**Extraction** means pulling exactly what is present in the source text. **Inference** means deriving something that is implied but not explicitly stated. Both are legitimate tasks, but they require different tooling and different guardrails.
—
## Making Fields Optional When the Truth Is Absence
The simplest structural fix is also the most important: stop requiring fields that may legitimately be absent from the source material.
When a field is marked as required and the source text does not contain the relevant information, the model is forced to invent something. When a field is marked as optional, the model can return a null value, which honestly represents the state of the source material.
“`
class DischargeRecord:
patient_name: str | None
medication: str | None
dosage_mg: float | None
follow_up_date: date | None
referring_physician: str | None
“`
With this approach, a summary that mentions a medication but no follow-up date will correctly return `None` for `follow_up_date`. Your downstream code can then decide what to do with that gap: flag it for human review, trigger a request for clarification, or apply a default policy.
That decision belongs in your code, not in the model’s imagination.
This distinction matters because language models are extraordinarily good at filling silence. They are trained to be helpful, complete, and confident. An empty field feels like an incompleteness to them, and they will resolve that feeling by generating a value every single time, unless you give them explicit permission not to.
—
## Tracking Where Values Come From
Nullable fields solve the problem of invented values when no source exists. But they do not solve a deeper problem: the model gives you a value that happens to be present in the text, and you have no way to verify whether it actually read that value or hallucinated it.
When working with freeform chat responses, you can at least watch the model reason its way to an answer. Structured outputs skip straight to the final form, bypassing any trace of reasoning.
To address this, I began adding a parallel field alongside every extracted value: a field that stores the exact span of source text that supposedly supports the extracted value.
“`
class EvidencedValue:
value: str | float | date | None
source_text: str | None # the exact quote from the document that supports this value
“`
By placing the source field before the value field in the schema, the model is forced to document what it has found before it commits to an extraction. This small structural choice acts as a forcing function for transparency.
When you review the output, you can quickly check whether `source_text` is populated and whether it actually appears in the original document. If `value` is populated but `source_text` is empty, you have a hallucination that no schema validator will catch.
This approach does carry a cost. Adding evidence fields across a schema increased token usage by roughly thirty percent in my benchmarks, and latency became a meaningful factor at scale. For a five-digit postal code or a boolean flag, that cost is hard to justify. For a clinical date that triggers a treatment plan, the cost is negligible compared to the alternative.
—
## Separating Generation from Validation
Even with nullable fields and evidence tracking, there remains an entire category of errors that neither technique touches: values that are present, sourced, and plausible—but factually wrong in the real world.
A schema can enforce that `dosage_mg` is a positive number. It cannot enforce that the dosage is clinically appropriate. A schema can enforce that `follow_up_date` is a valid calendar date. It cannot enforce that the date is not three months in the future when the discharge summary was written yesterday.
Prompt instructions like “the dosage must be greater than zero” are an unreliable substitute for actual validation. Language models are not calculators. They are pattern recognizers that happen to output text.
The solution is to add a validation layer that operates entirely outside the language model, using the same structured data types you are already working with:
“`
def validate_record(record):
if record.dosage_mg is not None and record.dosage_mg <= 0:
raise ValueError(f"Dosage must be positive, got {record.dosage_mg}")
if record.follow_up_date is not None and record.follow_up_date > date.today():
raise ValueError(f”Follow-up date {record.follow_up_date} is in the future”)
if record.source_text and record.source_text not in original_document:
raise ValueError(“Source text not found in original document”)
return record
“`
This validation step runs every time, deterministically, with no model involvement. When it catches an error, you have a decision to make: route the record to a human reviewer, or feed the validation error back to the model and ask it to try again.
I chose the retry path but with a strict cap: two attempts maximum. My experience taught me that two failed attempts almost always indicate a genuinely problematic source document rather than a prompt issue. A third automated attempt just burns API credits on something that a human can resolve in seconds.
—
## The Deeper Lesson
When I first got structured outputs working reliably, my bar for success was embarrassingly simple: did the model produce parseable output without breaking my code?
Looking back, that bar rewards the wrong thing entirely. A model that eagerly fills every field regardless of what is actually present in the source is not reliable. It is merely confident. Confidence without accuracy is the most dangerous outcome of any data pipeline, because it produces no errors to alert you.
Structured output formats are genuinely excellent at what they were designed to do. They guarantee syntactic structure. They eliminate an entire category of parsing failures. They make integration with typed codebases straightforward and maintainable.
What they do not do is guarantee truth. And once you stop worrying about brackets, commas, and quote escaping, the real question remains: does every value in this object have an actual reason to exist?
That question was always the hard part. The schema just used to hide it from me.
—
## Frequently Asked Questions
**What is the difference between extraction and inference in the context of structured outputs?**
Extraction is the process of pulling exactly what is explicitly stated in the source material. Inference is the process of deriving something that is implied but not directly expressed. A schema that demands a specific field will force the model to fill that field one way or another, which means it may conflate extraction with inference and present an inferred value as though it were directly extracted. Making fields nullable gives you the ability to distinguish between “the source says this” and “the source does not say this.”
**Why would a model invent a value for a required field when the source does not contain it?**
Large language models are trained to be helpful and complete. When a schema declares a field as required and the source material lacks the relevant information, the model perceives an incomplete response as a failure. It will generate a plausible value to satisfy the schema constraint, even when no basis for that value exists in the input. This behavior is not a malfunction; it is the expected behavior of a model optimized for helpfulness. The solution is to make fields optional when the absence of information is itself a valid and meaningful result.
**Does adding evidence fields to every extracted value make sense for all projects?**
No. Evidence fields increase token consumption and latency, which matters at scale. For low-stakes extractions such as categorizing a support ticket by topic or extracting a zip code from an address, the overhead is rarely justified. Evidence fields become worthwhile when the extracted value feeds into a decision that has real consequences—financial calculations, clinical data, legal classifications, or compliance records. A good rule of thumb is to add evidence fields for any value that a human would need to verify if something went wrong.
**Is the validation layer approach specific to any particular language model provider?**
No. The principle of separating extraction from validation is provider-agnostic. Whether you are using structured outputs from one provider, tool-use patterns from another, or a self-hosted model with a schema generation library, the validation logic operates on the final structured object and has no dependency on the model itself. The schema definition and the Pydantic-style validation classes remain identical; only the API call wrapping them changes.
**What should I do when my validation layer catches an error in a structured output?**
There are two reasonable paths. The first is to route the record to a human reviewer with the validation error and the original source material attached. This is appropriate for high-stakes domains where incorrect data has serious consequences. The second is to feed the validation error message back to the model as a correction prompt and let it regenerate the structured output. If you choose this path, impose a hard cap on retries—two or three attempts at most. Persistent failures almost always indicate a problem with the source document rather than the prompt, and continued automated attempts waste resources on records that require human attention regardless.
**How can I tell if my extraction pipeline is producing hallucinated data?**
The honest answer is that it is difficult to tell without additional provenance mechanisms. A well-structured output with populated fields and no errors will look identical to a well-structured output with accurate data. This is precisely what makes silent hallucination so dangerous. Adding evidence fields, cross-referencing extracted values against the source text, and running independent factual validators are the most effective strategies for surfacing hallucinations after the fact. If you are not actively checking for them, you should assume they are present at some rate proportional to the ambiguity of your source material.
—
## Conclusion
Structured outputs from language models represent a genuine leap forward in how we integrate unstructured text into typed systems. They eliminate tedious parsing hacks and make extraction pipelines dramatically more maintainable. But they also create a new and subtler class of failure mode: outputs that are structurally perfect, type-safe, and completely fabricated.
The engineering discipline required is not just about defining schemas correctly. It is about understanding that a schema is a contract about shape, not a contract about truth. Treating structural validity as synonymous with data correctness is the trap that catches every extraction pipeline eventually, often quietly, and often long after the damage has already been done.
Build nullable schemas for honest absences. Add provenance tracking for critical values. Validate independently of the model. And never let a green QA dashboard replace human judgment when the stakes are real.
—
Thank you for reading
