Persian Pixel: A large-scale synthetic OCR dataset for Persian language
Summary
Persian Pixel is a new, large-scale synthetic OCR dataset designed to overcome challenges in Persian language optical character recognition, a field less developed than Latin-script OCR. Persian script's complexity, including cursive connectivity, context-dependent glyphs, and extensive ligatures, combined with a scarcity of annotated data, has hindered progress. This dataset comprises over 343,000 high-fidelity image-text pairs, covering sentence, paragraph, and full-page layouts. It was generated from a seven-million-word Persian corpus using the SynthOCR-Gen framework, accurately modeling typographic features like contextual character joining and diacritic placement. To enhance realism, over twenty-five stochastic degradation models simulate real-world document artifacts such as ink bleed, blur, and scanner imperfections. Persian Pixel provides an openly available resource for training and fine-tuning modern OCR models like TrOCR and Donut, supporting research in Persian document analysis and digitization.
Key takeaway
For Machine Learning Engineers developing OCR solutions for low-resource or complex scripts, Persian Pixel demonstrates a viable path forward. You should consider synthetic data generation as a primary strategy to overcome data scarcity and annotation costs. This approach allows you to train robust models like TrOCR and Donut without extensive manual labeling, accelerating progress in areas like document digitization for languages with intricate writing systems.
Key insights
Synthetic data generation, exemplified by Persian Pixel, offers a scalable solution for OCR in typographically complex, low-resource languages.
Principles
- Complex scripts require specialized typographic modeling.
- Synthetic data can bridge real-world domain gaps.
- Costly manual annotation can be replaced.
Method
Generate image-text pairs from a large corpus using a rendering framework, modeling script typography. Apply stochastic degradation models to simulate real-world document acquisition artifacts.
In practice
- Train/fine-tune TrOCR and Donut models.
- Advance Persian document analysis.
- Digitize historical Persian manuscripts.
Topics
- Optical Character Recognition
- Persian Language
- Synthetic Data Generation
- Low-Resource Languages
- Document Digitization
- OCR Datasets
Best for: Research Scientist, AI Engineer, AI Scientist, Machine Learning Engineer, NLP Engineer
Related on AIssential
See Counsel's argued verdicts on the open AI decisions leaders are weighing →
Editorial summary, takeaway, and curation by AIssential. Original article published by Computer Vision and Pattern Recognition.