How the community trained Gemma to "Think" with Tunix and TPUs

2026-05-28 · Source: Google Developers Blog - AI · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering · Depth: Advanced, short

Summary

The Google Tunix Hackathon, held on Kaggle, challenged over 11,000 entrants to train Gemma models for general reasoning using Tunix and Kaggle TPUs. Developers transformed non-reasoning base models (Gemma-2-2B and Gemma-3-1B) into general reasoning models with limited compute (Kaggle TPU v5e-8 for 9 hours). Winning techniques combined supervised learning, preference optimization, and reinforcement learning. The first-place G-RaR used Supervised Fine-Tuning (SFT) with GRPO and a rubric-based LLM-as-judge reward system, employing a Gemma-3-12B judge model. Second-place Pinocchio-1B evolved a 1B parameter model via SFT, SimPO, and GRPO, extending Tunix for custom loss and asynchronous evaluation. Third-place IDEA-E distilled an ethical reasoning framework into a 2B model using curriculum-guided GRPO and a TF-IDF reward. Other approaches included on-policy distillation and custom dataset curation for domain-specific reasoning in medical, chemistry, legal, and robotics.

Key takeaway

For AI Engineers developing reasoning capabilities on smaller LLMs, you should explore multi-stage post-training pipelines combining SFT, preference optimization, and GRPO. Utilize Tunix and Kaggle TPUs to achieve strong results even with limited compute budgets, as demonstrated by the hackathon winners. Consider implementing custom reward functions or LLM-as-judge systems to provide dense feedback and refine reasoning logic for your specific applications.

Key insights

Community hackathons can effectively train LLMs for general reasoning using limited compute.

Principles

• LLM-as-judge provides dense, smooth feedback.
• Sequential training stages refine reasoning.
• Custom reward functions enhance logic.

Method

Winning methods generally involve a multi-stage post-training pipeline: Supervised Fine-Tuning (SFT) for baseline, followed by GRPO or SimPO for refinement, often with custom reward functions or judge models.

In practice

• Utilize Tunix with Kaggle TPUs.
• Implement multi-stage training pipelines.
• Integrate custom reward functions.

Topics

• Gemma Models
• Chain-of-Thought Reasoning
• Tunix Framework
• Reinforcement Learning
• Supervised Fine-Tuning
• Kaggle TPUs

Code references

• google/tunix

Best for: AI Engineer, Machine Learning Engineer, AI Scientist

Related on AIssential

• Why We Think — Allocating more test-time compute via methods like Chain-of-Thought significantly boosts LLM reasoning and problem-solving capabilities.
• Karpathy's LLM Wiki with Local Gemma 4 & llama.cpp | Agentic Tool Calling Test — Karpathy's LLM Wiki concept offers a vectorless, agentic approach to knowledge management, with LLMs maintaining an evolving, indexed knowledge base.
• NEW GEMMA 4 beats GPT-5.4: The A4B Model — Gemma 4's 4B active MoE model excels in complex logical reasoning and self-correction, outperforming its 31B dense counterpart.
• Paper: DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning — DeepSeek R1 enhances LLM reasoning via pure reinforcement learning and rule-based rewards, fostering autonomous problem-solving without supervised data.
• From 46% to 90%: Fine-Tuning Tiny LLMs for On-Device Agents — Cormac Brick, Google — Fine-tuning tiny LLMs with synthetic data dramatically enhances on-device agent performance and reliability for specific tasks.
• New method could increase LLM training efficiency — TLT doubles LLM training speed by adaptively using idle compute to train a smaller, verifying drafter model.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Google Developers Blog - AI.