When Your PDF Parser Returns 26 Pages of Nothing

2026-07-06 · Source: Data Engineering on Medium · Field: Technology & Digital — Software Development & Engineering, Artificial Intelligence & Machine Learning · Depth: Intermediate, medium

Summary

A common issue in document ingestion pipelines is encountering image-only PDFs, which cause standard parsers like PyMuPDF to return empty content, as demonstrated by a 26-page document yielding zero characters. To address this silent failure, a pluggable OCR fallback layer was developed, mirroring existing LLM abstraction patterns. This layer offers two OCR engines: RapidOCR, an offline solution installed via "pip install rapidocr-onnxruntime" that infers structure heuristically from bounding boxes, and Claude Vision, an API-based option using "claude-3-5-sonnet-20241022" for higher quality, especially with diagrams, by prompting for clean markdown. The updated PDF parser now includes a heuristic "_has_text_layer" check and, if no text is found, rasterizes pages at 150 DPI for OCR processing. Configuration allows "auto" selection based on "ANTHROPIC_API_KEY" presence, defaulting to RapidOCR.

Key takeaway

For AI Engineers building document ingestion systems, you must proactively integrate an OCR fallback for image-only PDFs. Relying solely on text-layer parsers risks silently discarding critical content from common formats like scanned documents or web exports. Implement a pluggable OCR layer, using an offline engine like RapidOCR as a default and a high-quality API like Claude Vision for complex cases, to ensure comprehensive data extraction and prevent dangerous, unlogged data loss.

Key insights

Image-only PDFs require a pluggable OCR fallback in document ingestion pipelines to prevent silent data loss.

Principles

• Robust document ingestion requires handling image-only PDFs.
• Abstracting external services enables swappable backends.
• Silent failures in data pipelines are critical to prevent.

Method

The proposed method involves checking for a PDF text layer; if absent, rasterize pages to PNGs (e.g., 150 DPI) and process with a pluggable OCR engine (e.g., RapidOCR or Claude Vision) to reconstruct markdown.

In practice

• Implement a "_has_text_layer" check early.
• Use RapidOCR for offline, structured text extraction.
• Employ Claude Vision for complex layouts and diagrams.

Topics

• PDF Parsing
• OCR Fallback
• Document Ingestion
• RapidOCR
• Claude Vision
• Image-only Documents

Best for: AI Engineer, Machine Learning Engineer, Software Engineer

Related on AIssential

• Parse Scanned PDFs for RAG with EasyOCR: Free OCR Gives You Words, Not a Document — Traditional OCR recovers text; layout models make that text usable for RAG by adding structural context.
• Unlimited OCR in 6 mins! — Baidu's Unlimited OCR leverages reference sliding window attention for high-throughput, memory-efficient document parsing.
• PaddleOCR 3.5: Running OCR and Document Parsing Tasks with a Transformers Backend — PaddleOCR 3.5 enables using Hugging Face Transformers as an inference backend for its OCR and document parsing models.
• How I Replaced a 20-Person Data Entry Team with an OCR Pipeline Processing 1,200 Documents a Day — Production document extraction requires handling diverse PDF types, normalizing data, and explicit LLM prompting.
• I Spent May Evaluating Different Engines for OCR — OCR is a routing problem; no single engine excels across all document types.
• Best Way to OCR a PDF in Python — Spacey layout unifies PDF OCR, layout detection, and NLP into a single, efficient Python pipeline.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Data Engineering on Medium.