Structured outputs

2024-12-05 · Source: Ollama Blog · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering · Depth: Intermediate, short

Summary

Ollama has introduced support for structured outputs, enabling users to constrain a language model's responses to a specific format defined by a JSON schema. This feature, released on December 6, 2024, is accessible via updated Ollama Python and JavaScript libraries, as well as cURL requests. Structured outputs offer enhanced reliability and consistency compared to traditional JSON mode, facilitating use cases such as parsing data from documents, extracting information from images, and standardizing all language model responses. The implementation allows for schema definition using Pydantic in Python or Zod in JavaScript, with examples provided for data extraction and image description using models like llama3.1 and llama3.2-vision. OpenAI compatibility is also demonstrated.

Key takeaway

For AI Engineers building applications requiring precise data extraction or consistent LLM responses, integrating Ollama's new structured output feature is crucial. You should define your output schemas using Pydantic or Zod and set the model's temperature to 0 to ensure deterministic and reliable JSON formatting. This approach will significantly improve the robustness and predictability of your LLM-powered workflows, reducing post-processing complexity.

Key insights

Ollama now supports constraining LLM outputs to a JSON schema for reliable, consistent structured data.

Principles

• JSON schema defines output format.
• Pydantic/Zod simplify schema definition.

Method

Define a JSON schema (e.g., with Pydantic/Zod), pass it to the Ollama API's `format` parameter, and set temperature to 0 for deterministic output.

In practice

• Extract data from documents.
• Describe images with vision models.
• Standardize all LLM responses.

Topics

• Structured Outputs
• JSON Schema
• Ollama
• Data Extraction
• Vision Models

Code references

• ollama/ollama-python
• ollama/ollama-js

Best for: AI Engineer, Machine Learning Engineer, Software Engineer

Related on AIssential

• Structured Outputs with LLMs: JSON Mode, Function Calling, and When to Use Each — LLM structured output methods vary in schema enforcement, suiting different application reliability and consistency needs.
• Issue #128 - Structured LLM Outputs with Pydantic — Combine Pydantic schemas with LangChain's output parsers and LCEL for robust, structured LLM outputs.
• Ollama: Unleash the Power of Local LLMs on Your Machine — Ollama enables private, cost-effective, and low-latency local LLM inference and customization via a Python library.
• Featuring Every Eval Ever Results on Hugging Face Model Pages — Standardized evaluation reporting via EEE and Hugging Face integration improves AI model transparency and comparability.
• How to Build & Run Reusable Extractors | Enterprise h2oGPTe — Extractors in h2oGPTe provide a reusable, scalable method for converting unstructured data into structured JSON using LLMs and JSON schemas.
• Production-Ready AI Agents: 5 Lessons from Refactoring a Monolith — Refactoring monolithic AI agents into orchestrated sub-agents with structured outputs and dynamic RAG improves reliability and scalability.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Ollama Blog.