Structured Outputs and Function Calling: Getting Reliable JSON from LLMs

Part 9 solves the problem that breaks most first LLM features: the model returns prose when you needed JSON, or JSON with a missing field. If your code does json.loads on free text, it will crash in production. This part shows two reliable approaches, structured outputs that constrain the response to a schema, and function calling where the model fills a typed tool input, and how a clear prompt makes both work better.

1. The fragile way, and why it fails

Asking for JSON in the prompt and parsing the reply works until the model adds a sentence before the brace, wraps the JSON in a code fence, or drops a field. You cannot ship that. The fixes below remove the guesswork.

2. Structured outputs

Structured outputs constrain the response to a JSON schema you provide, so the result is valid by construction. You then validate it with the same Pydantic model from Part 2 and you have a typed object, not a hope.

from anthropic import Anthropic

client = Anthropic()

schema = {
    "type": "object",
    "properties": {
        "sentiment": {"type": "string", "enum": ["positive", "neutral", "negative"]},
        "summary": {"type": "string"},
    },
    "required": ["sentiment", "summary"],
    "additionalProperties": False,
}

resp = client.messages.create(
    model="claude-opus-4-8",
    max_tokens=512,
    messages=[{"role": "user", "content": "Review: the API is fast but the docs are thin."}],
    output_config={"format": {"type": "json_schema", "schema": schema}},
)
print(resp.content[0].text)   # guaranteed to match the schema shape

3. Function calling

When the model should trigger an action rather than just answer, define a tool with a typed input schema. The model returns a tool_use block whose input already matches the schema, which you validate and execute. This is the foundation of the agent you build in Part 12.

resp = client.messages.create(
    model="claude-opus-4-8",
    max_tokens=512,
    tools=[{
        "name": "create_ticket",
        "description": "Open a support ticket. Call this when the user reports a bug.",
        "input_schema": {
            "type": "object",
            "properties": {
                "title": {"type": "string"},
                "severity": {"type": "string", "enum": ["low", "medium", "high"]},
            },
            "required": ["title", "severity"],
        },
    }],
    messages=[{"role": "user", "content": "The export button throws a 500 error."}],
)

for block in resp.content:
    if block.type == "tool_use":
        print(block.name, block.input)   # e.g. create_ticket {'title': ..., 'severity': 'high'}

4. The prompt still matters

Schemas guarantee shape, not judgment. A clear instruction about role, task, and the meaning of each field makes the model fill the schema correctly. Use the builder below to assemble a structured extraction prompt, then copy it into your own tests.

5. Validate the result into a Pydantic model

Structured outputs guarantee the JSON matches your schema, but your code still wants a typed Python object, not a raw string. So parse the result into the same Pydantic model you would use anywhere else. Now the rest of your code works with attributes, gets editor autocomplete, and shares one definition of the shape with the rest of the app. This is the moment the modeling work from Part 2 and the model call from Part 8 click together.

import json
from pydantic import BaseModel
from anthropic import Anthropic

class Review(BaseModel):
    sentiment: str
    summary: str

client = Anthropic()
resp = client.messages.create(
    model="claude-opus-4-8",
    max_tokens=512,
    messages=[{"role": "user", "content": "Review: fast API, thin docs."}],
    output_config={"format": {"type": "json_schema", "schema": Review.model_json_schema()}},
)
review = Review.model_validate_json(resp.content[0].text)   # typed object
print(review.sentiment, "|", review.summary)

Notice that Review.model_json_schema produces the schema from the model, so the model and the schema can never drift apart. One class defines the shape, the API enforces it, and Pydantic validates it back into that same class.

6. Classification with an enum

A very common task is classification: pick exactly one label from a fixed set. Free text answers are a nightmare to handle here, because the model might say positive, Positive, or the sentiment is positive. An enum in the schema removes that ambiguity entirely, since the only legal outputs are the values you listed. Combine it with the Pydantic model and you get a guaranteed, typed label.

from enum import Enum
from pydantic import BaseModel

class Priority(str, Enum):
    low = "low"
    medium = "medium"
    high = "high"

class Triage(BaseModel):
    priority: Priority
    reason: str

# The schema from Triage.model_json_schema() constrains priority to exactly
# low, medium, or high, so the parsed value is always a valid enum member.

7. Strict tools and parallel actions

Function calling gets stronger with two refinements. Marking a tool as strict makes the model fill its input to match the schema exactly, the same guarantee structured outputs give for responses. And the model can request several tool calls in one turn when the actions are independent, so you execute them together rather than in slow sequence. You still validate every tool input before running it, because a schema describes shape, not safety.

# A tool can be marked strict so its input is guaranteed to match the schema.
tools = [{
    "name": "create_ticket",
    "description": "Open a support ticket when the user reports a bug.",
    "strict": True,
    "input_schema": {
        "type": "object",
        "properties": {
            "title": {"type": "string"},
            "severity": {"type": "string", "enum": ["low", "medium", "high"]},
        },
        "required": ["title", "severity"],
        "additionalProperties": False,
    },
}]
# The model may return multiple tool_use blocks in one response; iterate them all.

8. A complete extraction endpoint

Put the pieces into a real FastAPI route and the reliability becomes concrete. The endpoint takes some text, asks the model to extract fields constrained to a schema, validates the result into a Pydantic model, and returns a typed response. There is no fragile parsing and no defensive string checking, because the schema and the model together guarantee the shape before your code ever sees it.

from anthropic import Anthropic
from fastapi import APIRouter, Depends
from pydantic import BaseModel
from llm_app.deps import get_model_client

router = APIRouter(prefix="/extract", tags=["extract"])

class Extracted(BaseModel):
    sentiment: str
    summary: str

class ExtractIn(BaseModel):
    text: str

@router.post("", response_model=Extracted)
async def extract(body: ExtractIn, client: Anthropic = Depends(get_model_client)) -> Extracted:
    resp = client.messages.create(
        model="claude-opus-4-8",
        max_tokens=400,
        messages=[{"role": "user", "content": body.text}],
        output_config={"format": {"type": "json_schema", "schema": Extracted.model_json_schema()}},
    )
    return Extracted.model_validate_json(resp.content[0].text)

Because the route declares Extracted as its response model, the docs show the exact output shape, and the test client can assert on it directly. This is the same endpoint pattern from Part 4, now producing reliable model output instead of a hand built dictionary.

9. Choosing your approach

With both tools in hand, the decision is usually clear. Reach for structured outputs when you want a shaped answer to use in code: an extraction, a classification, a parsed form. Reach for function calling when the model should decide whether to act and with what arguments, which is the agent pattern. And when neither shape is needed, when you just want prose for a human to read, do not force a schema at all, because the constraint only adds value when the next step is code that depends on the structure.

The bottom line

Reliable features need reliable data. Use structured outputs to force a schema, use function calling when the model should drive an action, and validate the result with Pydantic so the rest of your code works with typed objects. With dependable structure in hand, the next part tackles a bigger feature: answering questions over your own documents with retrieval.

Up next: Part 10, building a RAG service.