Learn2Zinc: Fine-tuning Small Language Models for Text-to-Model Translation in MiniZinc

2026-07-25 · Source: cs.CL updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering · Depth: Expert, extended

Summary

Learn2Zinc introduces a fine-tuning methodology for small language models (SLMs) ranging from 0.6B to 20B parameters to generate MiniZinc constraint models from natural language. Initial tests revealed near-zero execution accuracy (0.0-6.0%) for models like Qwen3, LLaMa, Gemma, and GPT-OSS on this domain-specific language. The proposed cross-model error bootstrapping approach collects syntax errors from multiple LLM runs to create an error correction training dataset. This strategy, combined with self-reflection and ensembling, boosts execution accuracy to 98%, effectively solving the MiniZinc syntax problem. However, solution accuracy remains at 35%, highlighting that semantic constraint reasoning is still a significant challenge. The project open-sources its fine-tuning pipeline, datasets, and models.

Key takeaway

For Machine Learning Engineers developing solutions for domain-specific languages like MiniZinc, relying on out-of-the-box or even chain-of-thought LLMs is insufficient due to pervasive syntax errors. You should prioritize targeted fine-tuning with error correction datasets, potentially generated via cross-model bootstrapping, and implement self-reflection loops to achieve high execution accuracy. Recognize that while syntax is largely solvable, improving semantic reasoning for complex constraint modeling remains a critical area for further research and development.

Key insights

Targeted fine-tuning with error correction enables small language models to master domain-specific language syntax.

Principles

• Pre-trained LLMs exhibit near-zero accuracy on rare, domain-specific languages.
• Syntax errors are the primary bottleneck for SLMs generating specialized code.
• Chain-of-thought fine-tuning is ineffective for SLMs lacking core syntax knowledge.

Method

A cross-model error bootstrapping approach collects LLM syntax errors, uses GPT-5.2 for minimal fixes, and combines with grammar-based synthetic corruptions to create an error correction dataset for fine-tuning.

In practice

• Fine-tune SLMs (0.6B-20B parameters) for domain-specific code generation.
• Employ self-reflection loops with compiler feedback to correct syntax errors.
• Ensemble fine-tuned SLMs to achieve high execution accuracy for specialized tasks.

Topics

• MiniZinc
• Small Language Models
• Fine-tuning
• Text-to-Model Translation
• Error Correction
• Constraint Programming
• Optimization Modeling

Code references

• skadio/learn2zinc

Best for: NLP Engineer, Research Scientist, AI Scientist, Machine Learning Engineer, AI Engineer

Related on AIssential

• SRCB at SemEval-2026 Task 5 A Multi-Target Finetuning Framework for Large Language Models with Joint Regression and Text Generation — Unified regressive–generative modeling enhances fine-grained plausibility estimation in large language models.
• Multilingual Fine-Tuning via Localized Gradient Conflict Resolution — Localized gradient conflict resolution via parameter buckets significantly improves multilingual LLM fine-tuning efficiency and performance.
• Asking language models how to represent data for fine-tuning — Base model performance on data formats reliably predicts post-fine-tuning efficacy, enabling inference-based format selection.
• Estimating Uncertainty from Reasoning: A Large-Scale Study of Multi- and Crosslingual MCQA Performance in LLMs — Prompting LLMs to reason in English improves multilingual uncertainty estimation, especially for low-resource languages.
• Bridging Scientific Heritage: An Arabic--Russian Parallel Corpus and LLM Benchmark for Sustainable Knowledge Transfer — Domain-specific fine-tuning significantly improves scientific machine translation between Arabic and Russian.
• Beyond Bilingual Transfer: Multilingual Code-Switching in Instruction Tuning — Multilingual code-switching instruction tuning improves LLM performance beyond bilingual settings.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CL updates on arXiv.org.