DeepSeek OCR Markdown Processing in Sparrow for Large Tables

2026-01-22 · Source: Andrej Baranovskij · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics, Software Development & Engineering · Depth: Intermediate, short

Summary

DeepSeek OCR has been integrated into Sparrow to enhance structured data extraction from large tables in financial statements and other documents. This two-stage process first uses DeepSeek OCR to convert PDF or image documents into a well-structured markdown format, complete with HTML-like tags for columns and rows, which provides additional context for text-based Large Language Models (LLMs). The second stage employs Sparrow's instructor pipeline, which processes this markdown text using a local LLM, such as Mistral 3.2, to extract specific financial values and descriptions based on a user-defined query. This method, demonstrated on a Mac Mini M4 Pro, aims to reduce hallucinations often associated with vision-based LLMs processing large tables directly from images, by providing a text-only input to the LLM.

Key takeaway

For AI Engineers building document processing pipelines, integrating DeepSeek OCR with a text-based LLM via markdown conversion offers a robust approach to extracting structured data from large tables. This method significantly reduces the risk of hallucinations compared to direct image processing by vision-based models, ensuring more reliable data extraction. Consider adopting this two-stage pipeline to improve accuracy and efficiency in your document analysis tasks.

Key insights

Combining DeepSeek OCR with text-based LLMs via markdown improves structured data extraction from large tables.

Principles

• Markdown provides LLMs with critical table structure.
• Text-based LLMs reduce hallucination risk with structured text.
• Two-stage processing enhances accuracy for complex documents.

Method

Convert documents to markdown using DeepSeek OCR, then feed the structured markdown to a text-based LLM via Sparrow's instructor pipeline to extract queried data.

In practice

• Use `markdown` flag in Sparrow for DeepSeek OCR conversion.
• Run DeepSeek OCR on Apple platforms with MLX backend.
• Process multi-page documents with post-processing steps.

Topics

• DeepSeek OCR
• Sparrow Platform
• Markdown Conversion
• Structured Data Extraction
• Large Language Models

Best for: AI Engineer, Machine Learning Engineer, Software Engineer

Related on AIssential

• Large Table Extraction to JSON with dots.ocr — No Vision LLM Hallucinations — Sparrow's new table processing mode uses OCR and custom logic for efficient, accurate large table data extraction.
• Structured Data Retrieval with Sparrow using OCR and Vision LLM [Improved Accuracy] — Combining OCR with Vision LLMs via structured prompts improves large table data extraction accuracy and reduces hallucinations.
• Convert Any Document To LLM Knowledge with Docling & Ollama (100% Local) | PDF to Markdown Pipeline — A local pipeline converts complex PDFs to markdown with tables and image descriptions for RAG applications.
• DeepSeek open sources DSpark, a new framework to speed up LLM inference by up to 85% — Speculative decoding, enhanced by confidence-scheduled verification, significantly accelerates LLM inference while preserving output.
• Multilingual OCR-Aware Fine-Tuning and Prompt-Guided Chain-of-Thought Reasoning for Multimodal Large Language Models — OCR-aware data curation and fine-tuning significantly improve MLLM robustness for multilingual text in degraded images.
• SafeLLM: Extraction as a Hallucination-Resistant Alternative to Rewriting in Safety-Critical Settings — Extraction, especially line-number-based, reduces LLM hallucinations in RAG for safety-critical documents better than rewriting or safety-focused filtering.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Andrej Baranovskij.