Persian Pixel: A large-scale synthetic OCR dataset for Persian language

· Source: Computer Vision and Pattern Recognition · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Advanced, quick

Summary

Persian Pixel is a new, large-scale synthetic OCR dataset designed to overcome challenges in Persian language optical character recognition, a field less developed than Latin-script OCR. Persian script's complexity, including cursive connectivity, context-dependent glyphs, and extensive ligatures, combined with a scarcity of annotated data, has hindered progress. This dataset comprises over 343,000 high-fidelity image-text pairs, covering sentence, paragraph, and full-page layouts. It was generated from a seven-million-word Persian corpus using the SynthOCR-Gen framework, accurately modeling typographic features like contextual character joining and diacritic placement. To enhance realism, over twenty-five stochastic degradation models simulate real-world document artifacts such as ink bleed, blur, and scanner imperfections. Persian Pixel provides an openly available resource for training and fine-tuning modern OCR models like TrOCR and Donut, supporting research in Persian document analysis and digitization.

Key takeaway

For Machine Learning Engineers developing OCR solutions for low-resource or complex scripts, Persian Pixel demonstrates a viable path forward. You should consider synthetic data generation as a primary strategy to overcome data scarcity and annotation costs. This approach allows you to train robust models like TrOCR and Donut without extensive manual labeling, accelerating progress in areas like document digitization for languages with intricate writing systems.

Key insights

Synthetic data generation, exemplified by Persian Pixel, offers a scalable solution for OCR in typographically complex, low-resource languages.

Principles

Method

Generate image-text pairs from a large corpus using a rendering framework, modeling script typography. Apply stochastic degradation models to simulate real-world document acquisition artifacts.

In practice

Topics

Best for: Research Scientist, AI Engineer, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computer Vision and Pattern Recognition.