Guidance

Guidance is a programming approach for steering language models. It is described as controlling output structure while reducing latency and cost compared with ordinary prompting or fine-tuning. It constrains generation with regular expressions and context-free grammars (CFG), and lets you naturally mix control such as conditionals, loops, and tool use with generation. It is installed via PyPI and supports several backends, including Transformers, llama.cpp, and OpenAI. In Jupyter notebooks, results are shown with a widget.

• Working with language models using Python syntax
• Restrict output format with regular expressions and CFG (constrained generation)
• Make the model pick just one item from a fixed list with select()
• Control that mixes generation with conditionals, loops, and tool use
• Various backends such as Transformers, llama.cpp, and OpenAI
• Provides a Jupyter notebook widget
• Easily capture generated text by name

Open source + paid cloud-managed plans

Open-source edition is free, SaaS managed edition is a paid subscription based on usage/seats

Estimate based on AI knowledge - please verify on the official site

• One of 27 AI tools in the Observability & Evals field.
• Connected to 3 other tools — #20 in its field.