Structured generation is becoming the contract layer for AI systems

JSON mode, schema-constrained output and strict tool use solve different problems. JSON mode aims to produce parseable JSON. A schema-constrained response limits the allowed shape and values. Strict tool use additionally binds the model to declared tool names and argument structures. None of these mechanisms can determine whether a factual value is supported by the input or whether an action is appropriate.

A field named confidence can contain 0.99 and still be meaningless. A required invoice number can be invented when the document contains none. An enum can force the model to choose approved or rejected even when the evidence supports unknown. The structure may be perfect while the decision is wrong.

Production acceptance should therefore expose separate counters for parse success, schema conformance, evidence support, semantic validity, authorization and final business success. Combining them into one “structured output success” rate hides the failures that matter.

A schema is not merely a parser configuration. Property names, descriptions, enums and nesting tell the model what the application expects, while downstream code treats those fields as an interface. Poor names and vague descriptions create semantic errors even when the output is structurally valid.

Use bounded enums for genuine closed sets, not for questions that may require uncertainty. Distinguish absence from null, zero and an empty list. State units in field names or descriptions. Prefer stable identifiers over display labels. Forbid undeclared properties when silently accepting them would hide model drift.

Schema wording itself can influence model behavior. Recent research finds that key names and schema-level descriptions can act as an instruction channel. This means a harmless-looking rename may change task accuracy, not just serialization.

Autoregressive models normally score every token in the vocabulary. A structured-generation engine tracks the current grammar or schema state and removes tokens that would make completion impossible. The model then samples only from valid continuations.

Regular expressions and finite-state machines cover many flat formats. Context-free grammars handle nesting and recursive structures. JSON Schema engines translate a supported subset into one of these internal representations. Systems such as Outlines, SGLang, XGrammar, llama.cpp, vLLM and TensorRT-LLM implement variations of token masking, parser state, precomputed token maps and grammar caching.

The grammar should constrain the final data channel, not accidentally constrain private reasoning or tool-result text that follows a different protocol. Modern runtimes increasingly support tagged or phase-specific constraints so free-form reasoning and strict final output can coexist.

JSON Schema Draft 2020-12 includes references, conditional keywords, annotations, unevaluated properties, numeric constraints, formats and recursive constructs. Model APIs and serving engines usually support a subset. Some ignore unsupported keywords, some reject the request and some translate the schema into a different grammar with additional restrictions.

A contract that works on one provider may weaken or fail on another. Property ordering, recursion, references, regex flavor, format enforcement, optional fields and additional properties deserve explicit compatibility tests. A Pydantic or Zod model is also not the runtime contract until its generated schema is inspected.

Maintain a provider capability matrix and a canonical test corpus. When a feature cannot be represented, fail the deployment or introduce a documented adapter. Silent downgrade from a business constraint to a prompt suggestion is particularly dangerous.

A final structured response might populate a UI or database staging table. A tool call can send money, delete data or publish a message. Both can use JSON Schema, but their risk is different.

For tool use, the application should first validate the tool name and arguments, then check the initiating user, tenant, target resource, current state and operation-specific policy. Approval should show exact parameters and bind to an idempotency key or artifact digest. The executor should not accept a raw model object directly.

Tool results are also untrusted data. A structurally valid result can contain prompt injection, stale state or an error encoded as a normal string. The agent loop needs a typed result contract and explicit error channel rather than asking the model to infer success from prose.

Required fields are useful for integration, but they can pressure a model to invent values when the source is incomplete. A robust extraction contract separates found, absent, ambiguous, conflicting and refused outcomes.

Use tagged unions or a top-level status field with status-specific payloads. A successful extraction may require data and evidence. An insufficient-evidence response may require missing-field names. A refusal should remain distinguishable from a technical timeout or truncated stream.

Partial streaming adds another complication: a prefix can be syntactically valid so far without being a complete instance. Consumers must wait for an explicit completion signal before parsing or executing, unless the protocol defines independently valid incremental events.

After generation, parse with a standard JSON library and validate using an independent schema implementation. Then run domain checks: identifier existence, cross-field relationships, allowed date ranges, currency and unit consistency, evidence coverage and user authority.

Blind repair loops are risky. A second model can make an invalid object parseable by changing a value, deleting a conflicting field or inventing a missing property. Repair instructions should include the exact validation errors, preserve original evidence and prohibit unsupported factual changes. High-consequence failures should return to a person or deterministic workflow rather than retry indefinitely.

Record the original output, effective schema, validator errors, repaired output and acceptance decision. Without that trail, a high final parse rate can conceal systematic semantic corruption.

Adding a required property, narrowing an enum or changing nullability can break clients immediately. Renaming a field may also change model behavior because the schema carries instructions. Model upgrades, tokenizer changes and decoder backends can alter latency, supported constraints and semantic accuracy.

Use contract tests with real and adversarial examples. Compare old and new schemas for backward and forward compatibility, then canary the combined model-schema-decoder release. Monitor structural failure, semantic failure, refusal, repair and business postcondition metrics separately.

Keep schemas small enough to understand and operate. A giant universal schema increases prompt tokens, grammar compilation, model confusion and downstream coupling. Several narrow contracts are usually easier to evaluate and authorize than one object with dozens of optional branches.