Parse PDFs for RAG Locally with Docling: Rich Tables, No Cloud Upload

2026-06-13 · Source: Towards Data Science · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering · Depth: Advanced, long

Summary

Docling, an open-source document parser from IBM Research, is presented as a local alternative to cloud-based solutions like Azure DI or simpler parsers such as PyMuPDF (fitz) for enterprise RAG systems. Running entirely on a local machine, Docling addresses critical enterprise concerns including confidentiality, data residency, air-gapped environments, and cost at scale. It offers advanced parsing capabilities, encompassing layout detection, OCR, reading-order, and TableFormer, a deep-learning model that accurately detects table structure, recovers table cells, OCR text within figures, and captions often missed by fitz. Docling's output maintains the same relational table structure as fitz and Azure, ensuring compatibility with downstream RAG pipelines. While fitz is instant and free, and Azure offers a managed service, Docling provides a local, free-to-run option with a latency of 1-5 seconds per page on CPU, requiring a one-time model download (hundreds of MB) and PyTorch installation. The article details how Docling enriches "line_df", "image_df", "toc_df", and "object_registry" with richer data.

Key takeaway

For AI Architects designing RAG systems with confidential documents or in air-gapped environments, Docling provides a critical local parsing solution. It eliminates cloud upload risks and per-page costs, offering rich table and layout extraction capabilities comparable to managed services. You should integrate Docling as a primary or fallback parser, leveraging its "parsing_method" column for adaptive routing. This ensures compliance and cost efficiency while maintaining high RAG quality.

Key insights

Docling offers robust, local PDF parsing for RAG, overcoming cloud compliance and cost issues with rich table and layout extraction.

Principles

• Data residency dictates parser choice.
• Local compute trades cost for control.
• Richer parsing improves RAG quality.

Method

Docling's pipeline runs layout detection, TableFormer for table structure, and optional OCR on scanned pages, then converts to a standardized dict of relational tables for RAG.

In practice

• Use "pip install docling" for local setup.
• Implement "parsing_method" for adaptive routing.
• Flatten table cells into markdown rows.

Topics

• Docling
• PDF Parsing
• RAG Systems
• Enterprise Document Intelligence
• Data Residency
• TableFormer

Code references

• docling-project/docling

Best for: AI Engineer, MLOps Engineer, AI Architect

Related on AIssential

• When PyMuPDF Can’t See the Table: Parse PDFs for RAG with Azure Layout — Azure Layout enriches PDF parsing for RAG by providing structured tables, OCR, and semantic roles where PyMuPDF fails.
• If You’re Paying a Managed API to Parse Documents at Scale, Someone Is About to Open a Very… — IBM's open-source Docling offers superior document parsing accuracy and cost efficiency compared to managed APIs.
• Build 100% Local Advanced RAG System for Financial PDFs with Qwen 3.5 | Docling, LangGraph & Ollama — A local RAG system can analyze financial PDFs using open-source tools for privacy and offline capability.
• Stop Returning Flat Text from a PDF: The Relational Tables RAG Needs — PDF parsing for RAG should produce a relational set of linked tables, not flat text, to preserve document structure.
• Why Naive Chunking Breaks RAG, and What to Build Instead — Structure-aware parsing and multimodal retrieval techniques are crucial for effective RAG on complex documents.
• The PDF Paradox: Why Document Parsing Is Still Hard — And Why the Hybrid Stack Is Winning — PDF's fixed-layout design, while excellent for rendering, creates a significant parsing challenge for AI.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Towards Data Science.