# Structured Output with Local LLMs: A Practical Guide
Local large language models (LLMs) offer compelling benefits for modern applications. By running models on your own hardware, you can keep sensitive data private and reduce dependency on external cloud APIs. However, simply running a model locally is only the first step. In real-world applications, the local LLM is typically part of a larger workflow, and its responses often need to be consumed by downstream components. Free-form text is notoriously difficult to parse reliably, which makes structured output essential.
Structured output allows you to define an expected schema in advance, have the local serving runtime constrain the model’s generation to that shape, and receive a regular Python object that is easy to parse and use. In this article, we illustrate this pattern through a concrete smart‑home case study using Gemma 4 as the local LLM, Ollama as the serving runtime, and Pydantic to define and validate the output schema.
## 1. How Do We Implement Structured Output with a Local LLM?
### 1.1 A Smart-Home Case Study
Imagine a smart‑home application where a user asks:
> “Should the dishwasher run now or later?”
Before answering, the application needs to extract device information, timing constraints, and electricity tariffs from household notes. Since these notes contain private information, a local LLM is a natural first step. It can transform the raw notes into a structured object that retains only the facts needed for scheduling, stripping away unnecessary personal details.
We then pass this sanitized object to a more capable cloud LLM for reasoning and scheduling. Here, we focus on the local transformation step that produces structured output.
### 1.2 Define the Expected Structure
The downstream component needs the current time, the device mentioned in the question, controller capacity, electricity prices, and a list of devices that still require scheduling with their runtime and timing constraints. We represent this using Pydantic models:
“`python
from typing import Annotated
from pydantic import BaseModel, Field
ClockTime = Annotated[
str,
Field(
min_length=5,
max_length=5,
description=”Clock time in HH:MM format.”,
),
]
class DeviceToSchedule(BaseModel):
device_name: str
duration_minutes: int
energy_kwh: float
earliest_start: ClockTime
finish_by: ClockTime | None
class SchedulingContext(BaseModel):
current_time: ClockTime
focus_device: str
max_concurrent_devices: int
current_price_per_kwh: float
off_peak_start: ClockTime
off_peak_end: ClockTime
off_peak_price_per_kwh: float
devices_to_schedule: list[DeviceToSchedule] = Field(
description=(
“Devices that have not completed their work ”
“and still need to be scheduled.”
)
)
“`
This schema is nested but straightforward. `SchedulingContext` contains shared household facts and a list of `DeviceToSchedule` objects. This is the shape we want the local LLM to output.
### 1.3 Setting Ollama and Local LLM
Before proceeding, ensure Ollama is installed and running locally. On Windows, you can install it with:
“`powershell
winget install Ollama.Ollama
“`
On macOS or Linux, run:
“`bash
curl -fsSL https://ollama.com/install.sh | sh
“`
Once installed, pull the Gemma 4 model:
“`bash
ollama pull gemma4:e4b
“`
Install the required Python packages:
“`bash
pip install ollama pydantic
“`
For this case study, we use the compact 4B variant of Gemma 4.
### 1.4 Connect Pydantic to Ollama
We connect our schema to the local model using the `format` parameter and `model_json_schema()`:
“`python
import ollama
def call_local_llm(schema, instructions, prompt):
response = ollama.chat(
model=”gemma4:e4b”,
messages=[
{“role”: “system”, “content”: instructions},
{“role”: “user”, “content”: prompt},
],
think=”medium”,
format=schema.model_json_schema(),
)
return schema.model_validate_json(response.message.content)
“`
Two key points:
– `model_json_schema()` converts our Pydantic model into a schema that Ollama can use to constrain generation.
– `model_validate_json()` parses the response back into a validated Pydantic object, making it easy to consume downstream.
### 1.5 Make the Structured-Output Call
Define instructions and build the prompt:
“`python
STRUCTURING_INSTRUCTIONS = “””
Convert the supplied source material into the structured scheduling context.
Do not decide or propose a schedule.
“””.strip()
def build_structuring_prompt(source_material):
return f”””
User question:
{USER_QUESTION}
Source material:
{source_material}
“””.strip()
“`
Run the call:
“`python
one_step_context = call_local_llm(
SchedulingContext,
STRUCTURING_INSTRUCTIONS,
build_structuring_prompt(SMART_HOME_CONTEXT),
)
“`
That’s it. The pattern is simple: define the schema, pass it to the model, and parse the validated response.
## 2. Valid Structure, Wrong Content
At first glance, the output appears correct:
“`python
print(type(one_step_context).__name__)
print([device.device_name for device in one_step_context.devices_to_schedule])
“`
Result:
“`
SchedulingContext
[‘Dishwasher’, ‘EV Charger’, ‘Washing Machine’, ‘Robot Vacuum (Kitchen Pass)’]
“`
The JSON matches our schema and Pydantic parses it successfully. However, the robot vacuum should not be included because the notes state it already completed its kitchen pass and no more vacuuming is needed today.
This reveals an important distinction:
> **Structured output only enforces the shape of the response. It does not, by itself, guarantee that the model puts the right information inside that shape.**
Why did the model get it wrong? The one‑shot call forced Gemma 4 to perform multiple tasks simultaneously:
– Determine which devices still need scheduling
– Extract relevant facts
– Map facts to schema fields
– Assemble a nested object
This complexity strains a small local model and increases the chance of errors.
## 3. Decompose the Task
A practical solution is to decompose the task into simpler steps.
### Step 1: Determine the Scheduling Scope
First, identify the focus device and which devices need scheduling:
“`python
class SchedulingScope(BaseModel):
focus_device: str
device_names_to_schedule: list[str]
“`
With instructions:
“`python
SCOPE_INSTRUCTIONS = “””
Identify the focus device and the household devices that still need scheduling.
Do not decide or propose a schedule.
“””.strip()
“`
Running this yields:
“`python
scope = call_local_llm(
SchedulingScope,
SCOPE_INSTRUCTIONS,
build_structuring_prompt(SMART_HOME_CONTEXT),
)
print(scope.model_dump_json(indent=2))
“`
Result:
“`json
{
“focus_device”: “Dishwasher”,
“device_names_to_schedule”: [“Dishwasher”, “EV charger”, “Washing machine”]
}
“`
The robot vacuum is correctly excluded.
### Step 2: Fill the Final Schema
Next, extract the detailed scheduling facts only for the selected devices:
“`python
DETAILS_INSTRUCTIONS = “””
Convert the supplied source material into the structured scheduling context
for the supplied devices. Do not decide or propose a schedule.
“””.strip()
details_prompt = f”””
Selected devices:
{json.dumps(scope.device_names_to_schedule)}
User question:
{USER_QUESTION}
Source material:
{SMART_HOME_CONTEXT}
“””.strip()
decomposed_context = call_local_llm(
SchedulingContext,
DETAILS_INSTRUCTIONS,
details_prompt,
)
“`
The final output is now fully correct: the robot vacuum is excluded, device records match the original notes, and personal information is removed.
## 4. Final Thoughts
Local LLMs are an attractive option when working with sensitive data. With structured output, we can integrate them into larger workflows where downstream components can easily consume the results.
The implementation is straightforward:
1. Define the expected schema with Pydantic.
2. Pass it to the local model via the serving runtime.
3. Parse and validate the response.
In practice, remember that **valid structure does not guarantee correct content**. As we saw, a direct call can follow the schema but still produce incorrect results.
By decomposing the task—separating scope determination from fact extraction—we achieved both valid structure and correct content. When a small local LLM struggles with a complex schema, decomposition is a practical strategy worth trying.
