Digitizing Old Ukrainian Texts: A Prompt-Based OCR Pipeline and Evaluation Dataset

2026-06-28 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics, Natural Language Processing · Depth: Expert, quick

Summary

A new methodology and an open dataset have been developed for Optical Character Recognition (OCR) of challenging Old Ukrainian texts. This initiative focuses on 430 handwritten index cards containing a scholarly transcription of "Perestoroha" by Iov Boretskyi (Lviv, 1605–1606), a 17th-century polemical text. The cards, created by 20th-century researchers, feature complex orthography including archaic diacritics, titlos, superscript letters, and ligatures, making automated recognition difficult. The researchers implemented a prompt-based OCR pipeline, utilizing a custom instruction set iteratively designed from the source material's conventions. Evaluation against human-proofread ground truth showed a proprietary configuration (Claude Opus 4.7, xhigh) achieved a 2.5% Character Error Rate. An Opus 4.6 baseline recorded 4.2% CER, while open-source Qwen3.6 variants (dense 27B and 35B-A3B MoE) reached 14.6% and 14.8% respectively. The fully digitized text, aligned to 300 DPI scanned images, is released as a scholarly resource and training data.

Key takeaway

For research scientists or NLP engineers tasked with digitizing historical manuscripts featuring complex orthography, you should consider a prompt-based OCR pipeline with iteratively refined instructions. While open-source models offer local processing, proprietary LLMs like Claude Opus 4.7 demonstrate significantly lower Character Error Rates (2.5% vs. 14.6%), justifying their use for critical accuracy requirements. Utilize the released dataset to benchmark your own systems or train new models for Old Slavic manuscripts.

Key insights

A prompt-based OCR pipeline, guided by custom instructions, effectively digitizes complex Old Ukrainian texts, with proprietary models achieving superior accuracy.

Principles

• Iterative instruction design improves OCR for complex orthography.
• Proprietary LLMs offer superior accuracy for specialized OCR tasks.
• Ground truth evaluation is essential for validating OCR pipelines.

Method

Develop a prompt-based OCR pipeline using an iteratively designed custom instruction set tailored to specific orthographic conventions, then evaluate against human-proofread ground truth.

In practice

• Digitize historical manuscripts featuring complex scripts.
• Generate training datasets for specialized OCR systems.
• Benchmark LLM performance on challenging text recognition.

Topics

• Optical Character Recognition
• Old Ukrainian Texts
• Prompt Engineering
• Large Language Models
• Historical Manuscripts
• Evaluation Datasets

Best for: AI Scientist, NLP Engineer, Research Scientist

Related on AIssential

• Structured Data Retrieval with Sparrow using OCR and Vision LLM [Improved Accuracy] — Combining OCR with Vision LLMs via structured prompts improves large table data extraction accuracy and reduces hallucinations.
• Tokenizer Fertility and Zero-Shot Performance of Foundation Models on Ukrainian Legal Text: A Comparative Study — Tokenizer efficiency and zero-shot performance vary significantly across foundation models on Ukrainian legal text.
• A Text Recognition Dataset from Sahidic Coptic Ancient Manuscripts — A new dataset, SCAM, highlights significant challenges for Handwritten Text Recognition in low-resource historical scripts like Sahidic Coptic.
• Automatic Metrical Scansion of Galician Poetry: First Results — A new system provides automatic metrical scansion for Galician poetry with 88% accuracy.
• Mistral OCR 3 Technical Review: SOTA Document Parsing at Commodity Pricing — Mistral OCR 3 offers superior document structure preservation and accuracy for RAG pipelines at a significantly lower cost.
• CzechDocs: A Multiway Parallel Dataset of Formatted Documents for Minority Languages in Czechia — CzechDocs is a multiway parallel dataset for evaluating format-preserving machine translation across Czech and minority languages.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.