Structured Output You Can Trust
The endpoint did one small thing. It took a line of text a human typed, like “grabbed lunch with the design team, about twelve bucks”, and returned an expense object the accounting system could file. It worked in the demo. It worked in the demo maybe forty times in a row.
The first morning in front of real traffic, the pager went off. SyntaxError: Unexpected token 'S', "Sure! Here"... is not valid JSON. The model had answered like this:
Sure! Here's the structured expense you asked for:
{ "amount": 12, "currency": "USD", "category": "meals" }
Let me know if you'd like me to adjust anything!JSON.parse hit the S in “Sure” and gave up before it reached a single brace. On other days it did something worse and wrapped the object in a markdown code fence, so the raw string started with three backticks and the word json. The model did nothing wrong by its own lights. It produced friendly, helpful prose around the data, which is exactly the behaviour it was trained to reward. Prose is a fine answer for a person. It is a terrible value for JSON.parse.
The moment you want to store a model’s answer, or branch on it, or add tax to it, you stop wanting text and start wanting a typed object. That gap is what this article is about: how to get JSON out of a text model reliably, and how to stay safe when reliably still is not always.
Text is a great answer and a useless value
Here is the same fact in two shapes. On one side, a sentence a human reads and understands instantly. On the other, an object your code can index into, multiply, compare, and write to a database.
The whole job of this article is dragging the model from the left shape to the right one, and being honest about how strong each technique’s guarantee actually is.
The naive way: ask for JSON and hope
The obvious first attempt is to just ask. Put “respond with JSON” in the prompt, describe the fields, and parse whatever comes back.
const prompt = `Extract the expense from this note as JSON with keys
amount (number), currency, category. Note: ${note}`;
const raw = await callModel(prompt);
const expense = JSON.parse(raw); // 🤞This works often enough to demo and rarely enough to page you. The failure is not that the model is dumb. It is that “produce some text” and “produce exactly this data structure, every single time, with no decoration” are different tasks, and plain prompting only asks for the first one. You get three recurring failure modes, and you will meet all three in your first week of real traffic.
The prose wrapper. The model is chatty by training. It says “Sure! Here you go:” or fences the JSON in triple backticks or signs off with “Hope that helps!”. Now JSON.parse chokes on the first non-JSON character. This is the one that paged me.
Wrong types. The shape looks right and the types are wrong. "amount": "twelve" as a string, or "amount": "$12" with the currency glued on, or "quantity": "2" where your code does math and gets "22" from a stray concatenation. It parses cleanly. It breaks three functions downstream, far from the cause.
Shape drift. Run the same prompt a hundred times and you get slightly different objects. Sometimes category, sometimes cat, sometimes a nested { "expense": { ... } } wrapper that was not there yesterday. Optional fields appear and vanish. Your code reads expense.category and gets undefined on the one call that decided to call it type.
You can beat these back with prompt engineering. “Respond with only JSON and no other text.” “Do not use markdown.” Few-shot examples of the exact shape. It genuinely helps, and it never fully stops, because you are asking politely for a guarantee the mechanism cannot give. Politeness is not a contract. There are two better rungs to climb.
Rung one: JSON mode
The first real feature most providers give you is a JSON mode. You set a flag on the request, and the provider guarantees the output is syntactically valid JSON. No prose wrapper, no trailing commas, no unquoted keys. JSON.parse will succeed.
// one common wire shape; the flag's name varies by provider
const body = {
model: "your-model-name",
messages: [
{ role: "system", content: "Extract the expense. Reply as a JSON object with keys amount, currency, category." },
{ role: "user", content: note },
],
response_format: { type: "json_object" },
};Read the guarantee carefully, because it is narrower than it feels. JSON mode promises valid JSON. It does not promise your JSON. The model can still hand you { "result": "twelve dollars" } or { "expense_amount": 12 } or an array when you wanted an object. It fixed the parse errors and left the shape and the types entirely up to the model’s mood.
JSON mode is a real improvement over hoping. It kills the prose-wrapper class of bug outright. It leaves wrong types and shape drift completely alive. For anything you will branch on, you want the next rung.
Rung two: constrain the decoder to your schema
Here is the rung that actually changes the guarantee. You hand the provider a JSON Schema describing exactly the object you want, and the provider restricts the model’s decoding so that it cannot produce anything off-schema. Not “is discouraged from”. Cannot. As of 2026 this goes by names like structured outputs, strict mode, or response schema, and it is available on the major providers. Expect the exact field names to keep shifting.
To see why the guarantee is hard and not hopeful, you need to know what “decoding” means. A language model produces text one token at a time. At each step it computes a score for every token in its vocabulary and samples one. Left unconstrained, a curly brace or the letter S or a comma is a legal pick at any moment, which is precisely why the model can wander off into prose or a trailing comma.
Constrained decoding slips a step in between the scores and the pick. Your schema gets compiled into a grammar, essentially a state machine that knows, given everything produced so far, which tokens could come next and keep the output valid. Every token that would break the schema has its score forced to negative infinity before sampling. The model literally samples only from tokens that keep the JSON on a valid path.
The effect is a categorical change, not a nudge. There is no trailing comma to strip, because the grammar never lets one be sampled. There is no prose wrapper, because after the closing brace the only legal move is to stop. Fields your schema marks as an enum can only emit one of the allowed strings. A provider’s own evaluations of this approach report effectively perfect schema conformance, in the neighbourhood of 100 percent, which no amount of prompt wording ever reached.
In practice you rarely hand-write the schema on the wire. You describe it once with a schema library and let the SDK translate. The portable shape looks like this:
import { z } from "zod";
import { generateObject } from "ai"; // provider-neutral toolkit
const Expense = z.object({
amount: z.number(),
currency: z.enum(["USD", "EUR", "GBP"]),
category: z.enum(["meals", "travel", "software", "other"]),
});
const { object } = await generateObject({
model: "your-provider/your-model",
schema: Expense,
prompt: note,
});
// object is typed as { amount: number; currency: ...; category: ... }
object.amount + 1; // the compiler already knows this is a numberOne schema definition does three jobs: it becomes the constraint sent to the model, it gives you a TypeScript type for free (see TypeScript for JS developers), and it is the thing you validate against when the guarantee does not hold. That last clause matters more than it looks, and the next section is entirely about it.
It is the same machine as tool calling
If constrained decoding feels familiar, it should. It is the exact mechanism behind tool calling. When a model “calls a function”, it is not running your code. It is emitting a small structured object, the arguments, constrained to the tool’s input schema. Strict tool use and structured output are the same trick pointed at two different targets: one fills in a function’s parameters, the other fills in your response object. Learn one and you have learned both. The wire fields differ, the guarantee is identical.
A valid shape is not a correct value
Constrained decoding guarantees the shape. It does not guarantee the truth. This is the trap that catches people who just discovered structured outputs and switched off their skepticism.
The grammar makes sure amount is a number. It has no opinion on whether that number is negative, or a thousand times too large because the model misread “twelve bucks” as “twelve hundred”. It makes sure category is one of your four enum values. It cannot know that the model picked travel for a lunch receipt. It makes sure date is a string. It will happily let the model invent February 30th or a date three years in the future. A schema-valid object can be completely, confidently wrong.
So you validate what comes back. With the same schema, in your own process, treating the model like any other untrusted source of input. This is exactly the discipline from validating at the boundary, and the same principle applies: parse, do not validate. Do not check the object and then keep using the loose value. Run it through the schema and get back a typed value, or an error. A model is just another client sending you data you did not write.
const Expense = z.object({
amount: z.number().positive(), // grammar won't catch a negative
currency: z.enum(["USD", "EUR", "GBP"]),
category: z.enum(["meals", "travel", "software", "other"]),
date: z.string().regex(/^\d{4}-\d{2}-\d{2}$/), // and won't catch a fake date's format
});
const parsed = Expense.safeParse(candidate);
if (!parsed.success) {
// the object was shaped right but failed a real rule
handleMiss(parsed.error);
}Notice which rules earn their place here. The plain z.number() and the two enums are already enforced by the grammar, so re-checking them is cheap insurance for the models where the grammar was not available. The .positive() and the date .regex() are doing work the grammar never promised. That is the layer that catches the model being wrong rather than the model being malformed.
When it misses: retry, repair, or fail
A validation miss is not a crash. It is a branch you decide in advance. There are three sane responses and one bad one.
Retry with the error fed back. The strongest option for a semantic miss. Append the model’s bad answer and the validation error to the conversation and ask again. Models are good at fixing a specific complaint. “amount must be positive, you returned -12” is an instruction the next attempt usually honours.
async function extractExpense(note: string) {
const messages = [
{ role: "system", content: "Extract the expense as JSON matching the schema." },
{ role: "user", content: note },
];
for (let attempt = 0; attempt < 3; attempt++) {
const candidate = await callModel(messages); // constrained to the schema
const parsed = Expense.safeParse(candidate);
if (parsed.success) return parsed.data; // typed, trusted, done
messages.push({ role: "assistant", content: JSON.stringify(candidate) });
messages.push({
role: "user",
content: `That failed validation: ${parsed.error.message}. Return corrected JSON only.`,
});
}
throw new Error("no valid expense after 3 attempts");
}Repair in code. For narrow, predictable defects, fix it yourself rather than paying for another call. If the only recurring problem is a stray code fence around otherwise-valid JSON, a five-line extractor that grabs the substring between the first { and the last } is faster and cheaper than a round trip. Keep repairs boring and specific. A clever general-purpose JSON fixer is a bug farm.
Fail loudly. Sometimes the right answer is to give up, log the raw output, and return an error to the caller. A retry budget of three, then a clean failure, beats an infinite loop burning tokens on a request the model cannot satisfy.
The bad option is the one people reach for first: silently coerce. Number(candidate.amount) || 0 looks defensive and is a landmine. It turns a loud, fixable failure into a 0 that flows into someone’s expense report. When in doubt, fail where you can see it.
Design schemas the model can actually fill
The model reads your schema. Field names, descriptions, and enum values are part of the prompt whether you meant them to be or not. A schema is not just a validator, it is instructions, and a badly designed one drags accuracy down even with the decoder constrained. A few habits pay off.
Name fields the way you would explain them. amount beats val. isRefundable beats flag. The model uses the name to decide what goes there, so a clear name is free accuracy.
Prefer an enum to a free string. Any time the answer is one of a known set, make it an enum. category: enum(...) gives the model four rails instead of an open field, and the constraint enforces it. Free-text category is how you end up with “Food & Drink”, “food”, “meals”, and “lunch” all meaning the same thing across four rows.
Write descriptions. Most schema languages let you attach a description to a field, and the model reads it. One sentence, “ISO 8601 date, the day the expense occurred, not today”, removes a whole class of ambiguity. This is the cheapest accuracy you will ever buy.
Keep it flat. A model fills { amount, currency, category } more reliably than { data: { payload: { value } } }. Deep nesting gives it more brackets to track and more chances to misplace a field. Flatten aggressively. Nest only when the structure is genuinely nested.
Ask for a wrapper object, not a bare array. When you want a list, wrap it: { "expenses": [ ... ] }, not a top-level [ ... ]. Several providers require the schema root to be an object anyway, a bare array is a common source of a rejected schema, and the wrapper gives you a natural home for metadata like a count or a confidence field later.
Required fields are a loaded gun
Here is the least obvious lesson, and the one that separates a schema that works from one that quietly lies. Making a field required forces the model to produce a value even when the honest answer is “I do not know”.
A model constrained to your schema cannot leave a required field blank. The grammar will not let it stop until every required key has a value. So if you mark phoneNumber required and the note has no phone number, the model does not error and does not omit it. It invents a plausible one, because a fabricated phone number is the only legal move left. You told it to always answer, so it always answers, truth be damned.
The fix is to make truly optional things optional. Give the model a way to say nothing. An optional field it can omit, or a nullable field it can set to null, is an escape hatch from hallucination. Reserve required for the fields that genuinely must be present in every valid answer.
const Expense = z.object({
amount: z.number().positive(), // must be there
category: z.enum(["meals", "travel", "software", "other"]),
merchant: z.string().nullable(), // often absent: let it say null
note: z.string().optional(), // nice to have, never required
});There is a second, sharper edge. Forcing the model to emit rigid JSON while it is still working out the answer measurably hurts its reasoning. Controlled studies have found double-digit drops in accuracy on hard reasoning tasks when a model must produce JSON directly, compared to letting it think in plain text first and formatting afterward. The JSON syntax competes for the model’s attention with the actual problem.
The practical remedy is not to abandon structure. It is to order it: think first, format later. If a task needs real reasoning, either run it in two steps (reason in free text, then a cheap second call formats the conclusion into your schema) or give the schema a reasoning field before the answer fields, so the model reasons on the way in.
const Decision = z.object({
reasoning: z.string(), // model works through it here, first
approved: z.boolean(), // ...then commits to the answer
amount: z.number().positive(),
});Field order matters because the model generates top to bottom and can read what it already wrote. Put reasoning first and approved second, and the boolean is chosen with the reasoning already on the page. Reverse them and the model commits to a verdict before it has argued for one. If you already reach for reasoning models, they do this thinking natively and the effect is smaller, but the ordering habit still helps every other model.
See it run with no model
Below is the extract-then-validate pipeline with no model and no network anywhere in it. Pick one of the canned “model outputs”, including the three failure modes from earlier, then run it through an extractor that strips any prose wrapper, a parser, and a validator that checks each field against a small schema. A clean output passes. A broken one reports the exact field that failed, which is precisely what you would feed back on a retry. Edit the text and run it again to probe the edges.
Play with the “wrong type” sample and read the validator’s message. amount must be a number, got "twelve" is not a stack trace, it is a sentence you can hand straight back to the model. That is the whole loop: the schema constrains the model going out, and the same schema catches it coming back in, and a miss produces an instruction, not a crash.
Summary
- Text is the right output for a human and the wrong one for code. The instant you store or branch on a model’s answer, you want a typed object, not prose.
- Prompt-and-hope fails three ways: a prose wrapper breaks the parse, wrong types slip through it, and the shape drifts call to call. Prompt wording reduces this but never guarantees it.
- JSON mode guarantees valid JSON syntax, which kills the prose-wrapper bug. It does not guarantee your shape or your types.
- Schema-constrained decoding is the real fix. You hand over a JSON Schema and the decoder is restricted so off-schema tokens are impossible. As of 2026 this is broadly available; the field names keep changing.
- Under the hood it is the same machine as tool calling: the model emits a structure constrained to a schema, whether that structure is a response object or a function’s arguments.
- A valid shape is not a correct value. Constrained decoding guarantees types and enums, not that a number is positive or a date is real. Validate what comes back with your own schema, and parse, do not validate.
- On a miss, decide up front: retry with the error fed back, repair narrow known defects in code, or fail loudly. Never silently coerce a bad value into a default.
- Design schemas the model can fill: clear names, enums over free text, descriptions, flat over nested, and a wrapper object instead of a bare array.
- Required fields force the model to answer, so it fabricates when it should abstain. Make genuinely optional things optional or nullable, and let the model reason before it formats.