← Graph

Structured LLM Output

concept 4 connections

Technique for getting parseable data out of an LLM despite it only emitting text. Old approach (still visible in LangChain's prompts, one in JSON and one in YAML retry form): ask 'please generate data matching this JSON schema', validate the output, and on failure re-prompt with the error message and schema — the same loop developers run against junior engineers. Modern LLM servers now enforce structured output internally, returning clean JSON with messages listed as arrays of objects and tool calls as structured fields. Hasiński highlights that server-side enforcement is superior because fault-tolerant per-token validation has to happen locally (latency of a remote round-trip would be prohibitive) — llama.cpp already does something similar.

category
pattern
about
Structured LLM Output concept
Walks through prompt-based JSON schema enforcement and its modern server-side replacement.
about
Structured LLM Output concept
The question is about enforcing structured output one token at a time.
concept Structured LLM Output
related_to
LangChain's JSON and YAML retry prompts exemplify the old client-side approach.
concept Structured LLM Output
related_to
llama.cpp tool
llama.cpp implements server-side per-token enforcement of structured output.

Provenance

Read by
9 extractions