Scaling Sentence Similarity for Classical Tibetan with Automatic Annotations

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Natural Language Processing · Depth: Expert, quick

Summary

A new scalable automatic annotation pipeline has been developed to train semantic embedding models for Classical Tibetan, a low-resource language. This pipeline addresses the challenge of scarce large annotated datasets by combining unsupervised contrastive bootstrapping with iterative pair mining. It generates silver-standard similarity labels through two complementary strategies: an ensemble of embedding models and rerankers, and an LLM-as-a-judge committee utilizing best–worst scaling. When sequentially fine-tuned with a domain-specific, gold-standard annotated dataset, the resulting text-similarity model achieves a Spearman correlation of 0.864 on the Semantic Textual Similarity (STS) task. This innovation facilitates effective semantic search in Classical Tibetan and provides a robust framework for automatic supervision in other low-resource languages within digital humanities.

Key takeaway

For NLP Engineers developing semantic models for low-resource languages, this pipeline offers a robust solution to data scarcity. You should consider adopting its automatic annotation strategies, particularly the LLM-as-a-judge committee, to generate high-quality silver-standard datasets. This approach can significantly reduce manual annotation effort and achieve strong performance, as demonstrated by the 0.864 Spearman correlation for Classical Tibetan. Implement this framework to accelerate digital humanities projects.

Key insights

A scalable pipeline uses automatic annotation and LLM-as-a-judge to train high-performing semantic embedding models for low-resource languages.

Principles

• Low-resource languages benefit from automatic annotation.
• Combining unsupervised and LLM-based methods improves data quality.
• Sequential fine-tuning enhances domain-specific model performance.

Method

The pipeline combines unsupervised contrastive bootstrapping with iterative pair mining. It generates silver-standard labels using an ensemble of embedding models/rerankers and an LLM-as-a-judge committee with best–worst scaling.

In practice

• Apply pipeline for semantic search in low-resource texts.
• Use LLM-as-a-judge for silver-standard data generation.
• Fine-tune models with small gold-standard datasets.

Topics

• Classical Tibetan
• Semantic Embeddings
• Low-Resource NLP
• Automatic Annotation
• LLM-as-a-Judge
• Digital Humanities

Best for: AI Scientist, NLP Engineer, Research Scientist

Related on AIssential

• LLMs for automatic annotation of Mandarin narrative transcripts — LLMs can automate complex discourse-level annotation in non-English speech, significantly reducing time.
• DySem: Uncovering Dynamic Semantic Components via Multilingual Consensus for Calculating Semantic Textual Similarity — DySem uses multilingual consensus to find dynamic, sample-specific semantic dimensions for improved STS in LLMs.
• I spent 4 days training an AI model to understand my country. It barely works — Low-resource NLP projects face significant challenges in data collection, translation, and accurate labeling, often yielding modest baselines.
• Data Augmentation for Historical NER: A Systematic Comparison of Lexical and LLM-based Approaches — This paper systematically compares lexical and LLM-based data augmentation for Historical Named Entity Recognition.
• Ancient Greek to Modern Greek Machine Translation: A Novel Benchmark and Fine-Tuning Experiments on LLMs and NMT Models — A new AG-MG parallel corpus and benchmark significantly advance Ancient Greek to Modern Greek machine translation.
• IndicDB -- Benchmarking Multilingual Text-to-SQL Capabilities in Indian Languages — IndicDB reveals a significant "Indic Gap" in LLM Text-to-SQL performance for Indian languages compared to English.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.