NuExtract3 released: open-weight 4B VLM for Markdown, OCR and structured extraction (self-hostable) [P]

2026-05-22 · Source: Machine Learning · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Advanced, short

Summary

Numind has released NuExtract3, an open-weight 4B Visual Language Model (VLM) based on Qwen3.5-4B, available under an Apache-2.0 license. This model is designed to enhance information extraction from complex documents such as PDFs, screenshots, forms, tables, receipts, invoices, and multi-page documents. NuExtract3, the successor to NuMarkdown, converts document images to Markdown, extracts structured data using JSON templates, and handles layout-heavy pages with both text and visual inputs. Trained for three days on an 8xH100 node, it supports self-hosting with Safetensors, GGUF, and MLX weights, offering various quantizations like GPTQ, W8A8, FP8, Q4, and Q6, requiring as little as 4GB of VRAM. A free Hugging Face space is available for testing, and Numind is developing benchmarks with hand-annotated documents and a confidence scoring system, with the model designed to return "null" for uncertain extractions.

Key takeaway

For MLOps Engineers evaluating document information extraction solutions, NuExtract3 provides a compelling open-weight, self-hostable option. Its 4B VLM, designed for complex documents and structured data extraction, runs on as little as 4GB VRAM with various quantizations. You should test its performance on your specific document types via the free Hugging Face space to assess its "null" response to uncertainty and its ability to handle layout variance before integrating.

Key insights

NuExtract3 is an open-weight 4B VLM for robust, self-hostable document information extraction, designed to avoid hallucination.

Principles

• Open-weight models can make complex extraction practical.
• Prioritize real-world, difficult document benchmarks.
• Train models to return "null" instead of hallucinating.

Method

The model converts document images to Markdown or extracts structured data using a target JSON template, processing both text and visual inputs from complex documents.

In practice

• Test NuExtract3 on its free Hugging Face space.
• Self-host with 4GB VRAM using provided weights.
• Parallelize Markdown conversion page by page.

Topics

• NuExtract3
• Visual Language Models
• Document Information Extraction
• Self-hosting AI
• Open-weight Models
• Structured Data Extraction

Best for: AI Architect, NLP Engineer, Computer Vision Engineer, AI Engineer, Machine Learning Engineer, MLOps Engineer

Related on AIssential

• ExStrucTiny: A Benchmark for Schema-Variable Structured Information Extraction from Document Images — ExStrucTiny is a new benchmark for structured information extraction from diverse, schema-variable document images.
• Mistral Document AI (with OCR 4) and Mistral Medium 3.5 arrive in Microsoft Foundry — Mistral's new models in Microsoft Foundry enable integrated, structured document processing and advanced reasoning for enterprise AI.
• Building a Production RAG Ingestion Pipeline on AWS: Unstructured.io, S3 Vectors, and a Private VPC — Self-hosting Unstructured.io within a private AWS VPC enables robust, secure, and cost-effective RAG ingestion.
• [R] IDP Leaderboard: Open benchmark for document AI across 16 VLMs, 9,000+ documents, 3 benchmark suites — An open benchmark evaluates 16 VLMs on document understanding, revealing tight performance margins among top models.
• DocQAC: Adaptive Trie-Guided Decoding for Effective In-Document Query Auto-Completion — Adaptive trie-guided decoding improves in-document query auto-completion by leveraging document context and balancing model confidence with trie guidance.
• LlamaParse vs  LLMs: Live OCR Battleground — Complex document parsing requires a hybrid approach combining traditional OCR with VLMs and structured harnesses to overcome LLM limitations.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Machine Learning.