What the old work looked like
Connecting a model to a system meant coaxing text into a shape a program could read. Prompts carried elaborate formatting instructions. Parsers stripped code fences, handled trailing commas, and coped with the model explaining its answer before giving it. A retry loop caught what the parser could not.
A meaningful share of the code in an early AI integration was not about the task at all. It was about the transport.
Schema-constrained output removed most of that
Supplying a schema and having the response conform to it structurally removes an entire category of defect. No fences to strip, no preamble to discard, no retry because the model decided to be conversational.
The integration code gets much smaller and much less interesting, which is the correct direction for integration code.
Structurally valid is not the same as correct
This is the part that catches people out. A schema guarantees the shape: that the amount field is a number and the date field is a string. It guarantees nothing about whether the amount is the right amount.
A model that cannot find a value in the source will still produce one, because the schema requires a value. The failure has moved from unparseable to parseable and wrong, which is harder to notice and considerably more dangerous downstream.
- Make fields nullable when "not present" is a real answer, and check for null
- Validate ranges and formats yourself; the schema only checks types
- Cross-check derived values against their inputs where you can
- Keep a found flag per field when the source may legitimately lack it
Design the schema to permit uncertainty
The most common mistake is a schema where every field is required. It reads as rigour and produces fabrication, because it forecloses the model's ability to say the information was not there.
Allowing null, or including an explicit found flag, lets uncertainty surface where it can be handled. Whether that routes to a person or to a fallback is then a decision you get to make. Without it, you never get to make it.
The hard part is now schema design
Deciding what the fields are, which are optional, what the enumerations contain, and what to do when the source disagrees with itself — that is where the difficulty went. It is a better place for it, because those are domain questions with real answers rather than a fight with a text format.
It also means the schema is worth reviewing with whoever owns the process, not only with whoever writes the code. They are the one who knows a record can legitimately have no reference number.
What to carry forward
Structured output is a transport improvement, and a large one. It is not a correctness guarantee, and treating it as one replaces a loud failure with a quiet one.
Keep the validation layer you wrote for the old world. It is doing a different job now — checking meaning rather than shape — and that job never went away.
Null is a real answer and should be allowed to be one
The single most useful schema habit is making a field nullable whenever "not present in the source" is a legitimate outcome, and then handling the null.
It sounds obvious and is routinely skipped, because a required field feels more rigorous. It is not: it removes the model's only honest option and forces a guess. A nullable field with an explicit downstream branch turns a silent fabrication into a visible decision.
Enumerations need an escape hatch
A closed set of values is good schema design right up to the input that belongs to none of them. Without an escape, the model picks the nearest wrong option and the record looks fine.
Include an "other" or "unknown" member, and route it somewhere a person sees. The count of items landing there is also a useful signal in itself: a rising number usually means the real world has developed a category your schema does not have yet.
Validate the meaning, not just the shape
The schema checks types. Everything about whether the values make sense is still yours to check: dates within a plausible range, amounts matching their currency, identifiers matching a known format, totals equal to the sum of their parts.
These checks are cheap and catch exactly the failures a schema cannot. They are also the natural place to decide what happens next — reject, queue for review, or accept with a flag — which is a business decision rather than a parsing one.
Keep the raw response
Store what the model returned alongside the parsed record, at least for a while.
When a value looks wrong later, the question is whether the model produced it or something downstream transformed it, and only the raw response answers that. It is also what you need to build an evaluation set from real traffic, which is much better than one assembled from imagination.