Get Vision LLMs to Follow Your Rules: Prompt-Guided JSON Formatting

2026-02-09 · Source: Andrej Baranovskij · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering · Depth: Intermediate, short

Summary

This content demonstrates how to control Large Language Model (LLM) output for structured data generation using prompt-guided JSON formatting. Specifically, it illustrates influencing the formatting of numerical values extracted from documents. The demonstration uses a Mistral 3.2 model running locally via Olama to process bond valuation data. By embedding specific formatting rules within the JSON schema query sent to the LLM, the model successfully formats numbers according to either European standards (period as thousand separator, comma as decimal) or US standards (comma as thousand separator, period as decimal). This technique allows LLMs to perform data post-processing directly during generation, reducing the need for external coding or manual manipulation.

Key takeaway

For AI Engineers building data extraction pipelines, you should integrate prompt-guided JSON formatting to enforce specific output standards directly within your LLM calls. This approach eliminates the need for external post-processing scripts, streamlining your workflow and ensuring data consistency for diverse regional or document-specific requirements.

Key insights

Prompt-guided JSON formatting enables LLMs to generate structured data with specific, user-defined output formats.

Principles

• Embed formatting rules directly into LLM prompts.
• LLMs can perform in-generation data post-processing.

Method

Construct a JSON schema query that includes textual rule descriptions for specific fields, guiding the LLM to format extracted data according to the specified rules (e.g., number separators).

In practice

• Use JSON hints for locale-specific number formatting.
• Automate data post-processing within LLM generation.

Topics

• LLM Output Control
• Prompt Engineering
• JSON Schema
• Structured Data
• Data Formatting

Best for: AI Engineer, Machine Learning Engineer, Prompt Engineer

Related on AIssential

• Structured Outputs with LLMs: JSON Mode, Function Calling, and When to Use Each — LLM structured output methods vary in schema enforcement, suiting different application reliability and consistency needs.
• Automate Writing Your LLM Prompts — DSPy automates and optimizes LLM prompt creation and evaluation, ensuring robust, production-ready applications.
• Structured Data Retrieval with Sparrow using OCR and Vision LLM [Improved Accuracy] — Combining OCR with Vision LLMs via structured prompts improves large table data extraction accuracy and reduces hallucinations.
• LLM-Powered Deep Parsing for Industrial Inventory Search — LLM-powered deep parsing extracts structured data from messy industrial text for improved search and deduplication.
• Build and Train an LLM from Scratch — Training an LLM from scratch involves data preparation, custom Transformer architecture, and tokenizer implementation.
• Soft-Prompt Tuning for Fair and Efficient LLM Benchmark Evaluation — Soft-prompt tuning efficiently reveals true LLM knowledge by overcoming format-following limitations in benchmarks.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Andrej Baranovskij.