inclusionAI / AReaL

2025-02-24 · Source: Github Trending: All languages · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering, Cloud Computing & IT Infrastructure · Depth: Advanced, medium

Summary

AReaL is an open-source, fully asynchronous reinforcement learning (RL) training system designed for large reasoning and agentic models. Developed by Tsinghua IIIS and Ant Group, it builds upon ReaLHF and emphasizes open-source principles by providing training details, data, infrastructure, and models for reproducibility. AReaL offers flexibility for agentic and online RL training, boasts stable and industry-leading speed, and delivers cutting-edge performance across math, coding, search, and customer service agents. Recent developments include AReaL-SEA, a self-evolving data synthesis engine that, when combined with AReaL, enabled a 235B MoE model to surpass GPT 5 and match Gemini 3.0 Pro on Math-bench. The system also supports Ascend NPU devices and offers a lightweight version, AReaL-lite, for rapid prototyping.

Key takeaway

For AI Architects and MLOps Engineers seeking to develop or deploy large reasoning and agentic models, AReaL offers a robust, open-source asynchronous RL training platform. Its demonstrated scalability and performance, including surpassing GPT 5 with AReaL-SEA, suggest it can significantly accelerate model development and deployment. You should explore its integration capabilities with existing agentic runtimes and consider its support for various hardware, including Ascend NPUs, to optimize your training infrastructure.

Key insights

AReaL provides a flexible, scalable, and high-performance asynchronous RL system for large agentic and reasoning models.

Principles

• Asynchronous RL training enhances speed and stability.
• Open-source commitment fosters reproducibility and accessibility.
• Algorithm-first API design simplifies development.

Method

AReaL facilitates RL training by allowing users to replace `base_url` and `api_key` for agentic RL services, supporting various algorithms like GRPO and PPO, and integrating with backends like Megatron and PyTorch FSDP.

In practice

• Train custom OpenClaw agents by updating service URLs.
• Utilize AReaL-SEA for self-evolving data synthesis.
• Deploy on Ascend NPU devices for specialized hardware acceleration.

Topics

• Asynchronous Reinforcement Learning
• Large Language Models
• AI Agents
• Reinforcement Learning Systems
• Data Synthesis

Code references

• inclusionAI/AReaL
• openpsi-project/ReaLHF
• inclusionAI/ASearcher
• camel-ai/seta
• YOUR-USERNAME/AReaL

Best for: MLOps Engineer, AI Scientist, AI Architect, AI Engineer, Machine Learning Engineer, AI Researcher

Related on AIssential

• RL for Agents Workshop - Deep Dive on Training Agents with RL and Open Source — AI agents are evolving rapidly, enabling long-horizon tasks through advanced RL training, custom models, and robust evaluation.
• Keep the Tokens Flowing: Lessons from 16 Open-Source RL Libraries — Asynchronous RL training disaggregates inference and training to overcome generation bottlenecks and GPU idle time.
• Context-Aware RL for Agentic and Multimodal LLMs — ContextRL uses an indirect RL objective to reward fine-grained grounding by selecting supporting contexts for query-answer pairs.
• Lessons from Trillion Token Deployments at Fortune 500s — Alessandro Cappelli, Adaptive ML — Reinforcement Learning systematically integrates feedback to accelerate LLM lifecycle, enabling production-ready, cost-effective, and performant models.
• Agentic RL: Frameworks and Best Practices — Agentic RL requires specialized frameworks and techniques for stable, scalable multi-turn training with LLMs.
• Libra: Efficient Resource Management for Agentic RL Post-Training — Libra efficiently manages resources for agentic RL post-training by dynamically optimizing GPU allocation and scheduling complex workloads.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Github Trending: All languages.