Structured output — explanation

What is this about?

Some answers are meant for a program rather than a person: a list of countries for a script, a bug triage for a coordinator agent. Free text is a poor carrier for that — the program has to guess where the data starts, and a model drifts from the format it was asked for. Servers that speak the OpenAI API can instead constrain the generation: given a JSON Schema in response_format, llama.cpp compiles it into a grammar, and the model cannot produce a token outside it. structuredOutput (ticket 80) is how an mm agent opts into that.

Why Go declarations

A JSON Schema is the wire format, but it is verbose and easy to get subtly wrong by hand. The section is written as Go type declarations instead: compact, readable, familiar to the people who write mm's configurations — and to models trained on code.

No Go type is created at runtime: Go cannot compile code while it runs. mm reads the text as source. go/parser, from the standard library, builds its syntax tree; internal/structured walks it and translates each declaration — a struct into a closed object, a slice into an array, a declared type into a $ref — then hands the schema to Genkit with ai.WithOutputSchema. Unsupported constructs are refused when the configuration loads, with their line in the YAML block, rather than as a failed answer later.

Rejected alternatives:

reflect.StructOf builds struct types at runtime, but cannot express named or recursive types, ignores comments, and would only re-check what the schema already says.
An embedded Go interpreter (yaegi): a large dependency to run code of which only the declarations are ever used.
A YAML shape notation (countries: [{name: string}]): shorter, but a mini-language to invent and document, with no place for descriptions.
A raw JSON Schema in the config: possible, but it is what the Go form exists to spare people from writing.

Why the declarations are also in the system prompt

The schema constrains the syntax, not the meaning. When the constraint is native, Genkit puts nothing about the schema in the prompt, and llama.cpp keeps only the grammar: the model sees key names, never the descriptions. Measured on Qwen3.5-4B: asked for capitals, it wrote the country's name in capital every time. mm therefore appends the declarations themselves, comments included, to the system prompt. That is why the comments matter: they are the instructions the model reads.

Why keys keep their declaration order — and why it may not help

properties are sent in declaration order, because a model fills a field from what it has already written: declaring Name first is a way to say "decide this first". A plain Go map would have sorted them. But llama.cpp (build b11429, measured through llmman) reorders the keys alphabetically when it builds its grammar, so on that server capital is still written before name. Servers that respect the order benefit; on llama.cpp, the remedy is a self-sufficient comment on every field.

Why an agent with tools cannot have one

A server that constrains every answer to the schema leaves no room for a tool call, and llama.cpp refuses grammar constraints together with tools. Genkit's own rule says the same (ConstrainedSupportNoTools). Rather than a hidden two-phase turn (work with tools, then reformat), mm keeps the roles apart: an agent either acts, or answers in a fixed shape. The two meet through ask_agent — a coordinator with tools asks a JSON agent and receives the JSON as the tool's result. The configuration error lists the tools to turn off.

What guarantees the answer

Three layers: the server's grammar (when the server supports it), Genkit's validation of the answer against the schema (a non-conforming answer fails the turn), and — in -p — an exit status a script can test, with nothing but the JSON on stdout. The usual failure is not bad syntax but a cut answer: a small model repeating list items until max_tokens. mm names that case explicitly, because the fix is a setting or a bounded list, not the model's syntax.

How it relates to the rest

Asking another agent — how the JSON reaches a coordinator.
Thinking models — a lookup agent rarely needs reasoning; a coordinator often does (on Qwen3.5-4B, without it, the coordinator answered from memory and claimed it had asked).
How to get answers as JSON and the structuredOutput reference.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40


# Structured output — explanation

## What is this about?

Some answers are meant for a program rather than a person: a list of countries for a script, a bug triage for a coordinator agent. Free text is a poor carrier for that — the program has to guess where the data starts, and a model drifts from the format it was asked for. Servers that speak the OpenAI API can instead **constrain** the generation: given a JSON Schema in `response_format`, llama.cpp compiles it into a grammar, and the model cannot produce a token outside it. `structuredOutput` (ticket 80) is how an mm agent opts into that.

## Why Go declarations

A JSON Schema is the wire format, but it is verbose and easy to get subtly wrong by hand. The section is written as Go type declarations instead: compact, readable, familiar to the people who write mm's configurations — and to models trained on code.

No Go type is created at runtime: Go cannot compile code while it runs. mm reads the text as **source**. `go/parser`, from the standard library, builds its syntax tree; `internal/structured` walks it and translates each declaration — a struct into a closed object, a slice into an array, a declared type into a `$ref` — then hands the schema to Genkit with `ai.WithOutputSchema`. Unsupported constructs are refused when the configuration loads, with their line in the YAML block, rather than as a failed answer later.

Rejected alternatives:

- **`reflect.StructOf`** builds struct types at runtime, but cannot express named or recursive types, ignores comments, and would only re-check what the schema already says.
- **An embedded Go interpreter** (yaegi): a large dependency to run code of which only the declarations are ever used.
- **A YAML shape notation** (`countries: [{name: string}]`): shorter, but a mini-language to invent and document, with no place for descriptions.
- **A raw JSON Schema** in the config: possible, but it is what the Go form exists to spare people from writing.

## Why the declarations are also in the system prompt

The schema constrains the **syntax**, not the meaning. When the constraint is native, Genkit puts nothing about the schema in the prompt, and llama.cpp keeps only the grammar: the model sees key names, never the descriptions. Measured on Qwen3.5-4B: asked for capitals, it wrote the country's name in `capital` every time. mm therefore appends the declarations themselves, comments included, to the system prompt. That is why the comments matter: they are the instructions the model reads.

## Why keys keep their declaration order — and why it may not help

`properties` are sent in declaration order, because a model fills a field from what it has already written: declaring `Name` first is a way to say "decide this first". A plain Go map would have sorted them. But llama.cpp (build b11429, measured through llmman) reorders the keys alphabetically when it builds its grammar, so on that server `capital` is still written before `name`. Servers that respect the order benefit; on llama.cpp, the remedy is a self-sufficient comment on every field.

## Why an agent with tools cannot have one

A server that constrains every answer to the schema leaves no room for a tool call, and llama.cpp refuses grammar constraints together with tools. Genkit's own rule says the same (`ConstrainedSupportNoTools`). Rather than a hidden two-phase turn (work with tools, then reformat), mm keeps the roles apart: an agent either acts, or answers in a fixed shape. The two meet through `ask_agent` — a coordinator with tools asks a JSON agent and receives the JSON as the tool's result. The configuration error lists the tools to turn off.

## What guarantees the answer

Three layers: the server's grammar (when the server supports it), Genkit's validation of the answer against the schema (a non-conforming answer fails the turn), and — in `-p` — an exit status a script can test, with nothing but the JSON on stdout. The usual failure is not bad syntax but a cut answer: a small model repeating list items until `max_tokens`. mm names that case explicitly, because the fix is a setting or a bounded list, not the model's syntax.

## How it relates to the rest

- [Asking another agent](asking-another-agent.md) — how the JSON reaches a coordinator.
- [Thinking models](thinking-models.md) — a lookup agent rarely needs reasoning; a coordinator often does (on Qwen3.5-4B, without it, the coordinator answered from memory and claimed it had asked).
- [How to get answers as JSON](../how-to/get-json-answers.md) and the [`structuredOutput` reference](../reference/configuration.md#structuredoutput).