Structured Outputs with JSON Schema: A Practical Guide

The moment you try to feed a model's answer into code, "return JSON" stops being good enough. Ask politely and you get JSON wrapped in an apology, JSON inside a Markdown fence, or JSON that is valid syntax but missing the field your parser needs. The reliable fix is to stop asking and start constraining: tell the provider the exact schema you expect and let it guarantee — or at least strongly enforce — that the response matches. This article covers the mechanisms that do this today — OpenAI's strict json_schema, Gemini's responseSchema, and Anthropic's native output_config.format (with strict tool use as its function-calling counterpart) — the schema rules each imposes, and why you still validate afterward.

Two failure modes, one root cause

There are two distinct ways free-text JSON breaks, and it helps to name them. Syntactic failure is output that will not parse at all: a trailing comma, a stray sentence, a code fence. Semantic failure is output that parses but is wrong for your program: an enum value you never defined, a missing required field, a string where you needed a number. Prompt engineering ("respond with only JSON") reduces the first kind and does almost nothing for the second. Schema-constrained generation attacks both, because the schema is enforced during decoding rather than checked after the fact.

OpenAI: strict: true is the whole feature

OpenAI's Structured Outputs is the evolution of the older JSON mode, and the difference is one flag. Plain JSON mode (response_format: { "type": "json_object" }) guarantees the output is valid JSON but nothing about its shape. Structured Outputs adds a schema and a strict switch that binds the model to it.

On Chat Completions you pass a response_format of type json_schema:

from openai import OpenAI
client = OpenAI(base_url="https://your-gateway.example.com/v1", api_key="AVE_GATEWAY_API_KEY")

resp = client.chat.completions.create(
    model="gpt-5.6",
    messages=[{"role": "user", "content": "Extract the invoice fields."}],
    response_format={
        "type": "json_schema",
        "json_schema": {
            "name": "invoice",
            "strict": True,
            "schema": {
                "type": "object",
                "properties": {
                    "vendor": {"type": "string"},
                    "total": {"type": "number"},
                    "currency": {"type": "string", "enum": ["USD", "EUR", "JPY"]},
                },
                "required": ["vendor", "total", "currency"],
                "additionalProperties": False,
            },
        },
    },
)

On the Responses API the same schema moves to a different field — text.format instead of response_format. There is no top-level output_parsed on a plain responses.create call; to get a parsed object, use the SDK's parse helper with a typed format, or read response.output_text and validate it yourself:

from pydantic import BaseModel

class Invoice(BaseModel):
    vendor: str
    total: float
    currency: str

resp = client.responses.parse(
    model="gpt-5.6",
    input="Extract the invoice fields.",
    text_format=Invoice,        # SDK builds the strict json_schema and parses the result
)
invoice = resp.output_parsed    # a validated Invoice instance

If you prefer the raw path, send text={"format": {"type": "json_schema", "name": "invoice", "strict": True, "schema": SCHEMA}} on responses.create, then parse and validate resp.output_text explicitly — do not expect a parsed field to appear on its own.

Sending response_format to Responses, or text.format to Chat Completions, is a frequent and confusing error — the field name is the thing that changed, not the concept. Coverage through a gateway is adaptor-specific, not uniform: this gateway maps an OpenAI-shaped response_format onto Gemini's responseSchema, but it does not currently translate it into Claude's native structured-output field, so for Claude you send the Anthropic-native shape (below) rather than assuming your response_format reaches it unchanged.

The strict-schema subset has rules

strict: true supports much of JSON Schema but not all of it, and the constraints are strict enough that a schema which validates fine in a normal validator can be rejected here:

• Every property must be listed in required. There are no truly optional fields. To model "may be absent," make the field a nullable union — "type": ["string", "null"] — rather than omitting it from required.
• additionalProperties: false is mandatory on every object. The model may not invent extra keys.
• The root must be an object. A top-level array is not allowed; wrap it in an object with an array-typed property.
• Recursion uses {"$ref": "#"} to reference the root, which is how you express nested tree structures.

When the model declines for a safety reason, that is surfaced programmatically as a refusal field on the message rather than as schema-shaped content, so you check for a refusal before trusting the parse. And a response can be cut off by length: inspect finish_reason (a value of "length" means truncated JSON), because a truncated strict response is still invalid JSON even though the schema was enforced.

Gemini: responseSchema plus a JSON mime type

Gemini constrains output through generationConfig. You set responseMimeType to "application/json" and supply a responseSchema (an OpenAPI-style schema object) — or, for standard JSON Schema, responseJsonSchema:

from google import genai
from google.genai import types

client = genai.Client()
resp = client.models.generate_content(
    model="gemini-3.5-flash",
    contents="Classify the ticket.",
    config=types.GenerateContentConfig(
        response_mime_type="application/json",
        response_schema={
            "type": "object",
            "properties": {
                "category": {"type": "string", "enum": ["billing", "bug", "feature"]},
                "priority": {"type": "integer"},
            },
            "required": ["category", "priority"],
        },
    ),
)

Gemini constrains generation to the schema, supports enum for classification, and lets you pin field order with propertyOrdering — useful because ordering can affect quality and consistency. Note two caveats the docs are explicit about: not every JSON Schema feature is supported, and very complex schemas may be rejected. Gemini also emphasizes that the output is valid, schema-shaped JSON but that you should still validate values — it can produce something schema-compliant yet semantically wrong (a plausible-but-incorrect category). The mechanism guarantees shape, not truth.

Anthropic: native output_config.format, with strict tool use

Anthropic now offers native structured outputs, so forced tool use is no longer the only path — it is the compatibility fallback. Two complementary features cover it. JSON outputs put an output_config object on the request with a format of {"type": "json_schema", "schema": ...}, and Claude's answer comes back as schema-matching JSON in the text content. Strict tool use adds strict: true to a tool's input_schema to guarantee schema-valid tool arguments. You can use either alone or both together.

import anthropic
client = anthropic.Anthropic(base_url="https://your-gateway.example.com", api_key="AVE_GATEWAY_API_KEY")

msg = client.messages.create(
    model="claude-opus-4-8",
    max_tokens=1024,
    output_config={
        "format": {
            "type": "json_schema",
            "schema": {
                "type": "object",
                "properties": {"vendor": {"type": "string"}, "total": {"type": "number"}},
                "required": ["vendor", "total"],
                "additionalProperties": False,
            },
        }
    },
    messages=[{"role": "user", "content": "Extract the invoice fields."}],
)
data = msg.content[0].text   # valid JSON matching your schema

A few current limits are worth knowing: additionalProperties must be false, and numeric/string constraints (minimum, maxLength, and similar) and recursive schemas are not yet supported — sending them returns a 400. The Python SDK also exposes client.messages.parse(..., output_format=PydanticModel) to build the schema and parse the reply in one step. When you specifically need the model to invoke something, strict tool use is the idiomatic route: define a tool whose input_schema is your target schema, add strict: true, and force it with tool_choice.

msg = client.messages.create(
    model="claude-opus-4-8",
    max_tokens=1024,
    tools=[{
        "name": "record_invoice",
        "strict": True,
        "input_schema": {
            "type": "object",
            "properties": {"vendor": {"type": "string"}, "total": {"type": "number"}},
            "required": ["vendor", "total"],
            "additionalProperties": False,
        },
    }],
    tool_choice={"type": "tool", "name": "record_invoice"},
    messages=[{"role": "user", "content": "Extract the invoice fields."}],
)
data = next(b.input for b in msg.content if b.type == "tool_use")

Forcing the tool makes it deterministic: the model must call record_invoice, and its input is your object. Reach for JSON outputs when you want a typed answer, and for strict tool use when the model should call something with guaranteed-valid arguments. Forced tool use without strict remains a legacy fallback for older models or the occasional tool-loop shape; on current models, prefer the native features. If your product already routes tools across providers, the definitions carry over — see portable tool calling across providers.

Structured output vs function calling: pick by intent

These two uses of JSON Schema look similar and are easy to conflate. The distinction is intent. Use function calling when the model should decide whether and how to invoke a tool that does something — query a database, call an API, run code. Use a structured response format (json_schema / text.format / responseSchema / output_config.format) when you simply want the model's answer shaped as data for your program to consume. All three providers now expose both: OpenAI supports strict schemas in response formats and tools, Anthropic pairs native JSON outputs with strict tool use, and Gemini offers response schemas and function declarations separately. Reaching for a tool call when you only wanted a typed answer adds a round trip you do not need.

You still validate — always

Schema enforcement is not a license to skip validation, for three concrete reasons. First, enforcement guarantees shape, never correctness: a model can return a well-typed value that is factually wrong, and only your business rules catch that. Second, truncation happens — a length-limited or timed-out response yields incomplete JSON no schema mode can save, so you must handle the finish_reason/status and the parse error. Third, schema support is a subset that differs by provider, so a schema that enforces strictly on one upstream may be treated more loosely on another. Run the parsed object through a validator (Pydantic, Zod, JSON Schema) at the boundary, and treat a validation failure as a normal, expected branch rather than an exception. This is the same defense-in-depth principle behind bounding any external input.

A gateway helps here by giving you one place to send response_format and one place to see which requests used structured output and how often they were retried, without teaching every client three providers' field names. Behind an OpenAI-compatible API gateway you write the OpenAI-shaped response_format once, and the gateway routes it to whichever channel serves the model. Be aware that translation is adaptor-specific: it reaches OpenAI unchanged and is mapped onto Gemini's responseSchema, but Claude's native output_config.format is not currently produced from it, so target Claude with the Anthropic-native shape.

FAQ

What is the difference between JSON mode and Structured Outputs?

JSON mode (response_format: {"type": "json_object"}) guarantees the output parses as JSON but says nothing about its shape — fields, types, and enums are unconstrained. Structured Outputs adds a json_schema with strict: true, which binds the model to your schema. Prefer Structured Outputs whenever the provider supports it; reserve bare JSON mode for cases where you genuinely do not have a fixed shape.

Why is my strict schema being rejected when it validates elsewhere?

The strict subset is narrower than full JSON Schema. The usual culprits are a missing additionalProperties: false, a property left out of required, or a non-object root. Make every field required (using nullable unions for optional ones), set additionalProperties: false on each object, and wrap top-level arrays in an object. On Claude specifically, numeric/string constraints and recursive schemas are rejected outright, so strip those too.

Does structured output guarantee the values are correct?

No. It guarantees the response matches the schema's shape — types, required fields, allowed enum values — not that the content is accurate. A model can return a perfectly-typed but wrong answer. Always validate values against your business rules after parsing, and treat schema conformance as a syntax guarantee, not a correctness one.

How do I get schema-constrained output from Claude?

Send an output_config with a format of {"type": "json_schema", "schema": ...} for a typed answer, or add strict: true to a forced tool when the model should invoke something. Both are native features on current models; the older force-a-tool-without-strict trick still works but is now a fallback rather than the intended path.

Delete one regex

Take one place in your code where you currently parse a model's free-text answer and regex out the JSON. Replace the prompt with a json_schema (or Claude's output_config.format), set strict: true, and wrap the parsed result in a schema validator. Then send a deliberately hard input and confirm two things: the request succeeds without a fenced-JSON cleanup step, and a refusal or truncation surfaces as a handled branch rather than a crash. Removing that regex is the concrete signal the constraint is doing the work.

References

• OpenAI: Structured Outputs
• Google Gemini: Structured output
• Anthropic: Structured outputs
• JSON Schema specification