NVIDIA and Hugging Face Bring New Models and Frameworks to LeRobot for the Open Robotics Community
Summary
NVIDIA and Hugging Face are integrating key NVIDIA physical AI capabilities into LeRobot, Hugging Face's open source robotics library. This collaboration introduces the NVIDIA Isaac GR00T 1.7 reasoning vision language action (VLA) model for humanoid robots and the NVIDIA Isaac Teleop framework, with the NVIDIA Cosmos 3 frontier physical AI model planned for future integration. These additions aim to standardize and simplify end-to-end robot development, fostering innovation within the open robotics community by connecting NVIDIA's 3 million robotics developers with Hugging Face's 16 million AI builders. The integrations facilitate common workflows for data collection, model training, fine-tuning, performance evaluation, and deployment. Isaac Teleop enables high-quality human demonstration capture and dataset sharing. Isaac GR00T 1.7 supports post-training and deployment across various robot embodiments. Cosmos 3 will enhance data generation and simulation. Existing NVIDIA resources, including a physical AI dataset with over 350,000 trajectories and 57 million grasps, Isaac Sim/Lab simulation frameworks, and Jetson Thor integration, further support the full robotics development loop within LeRobot.
Key takeaway
For Robotics Engineers developing and deploying physical AI, this collaboration offers a standardized, open source pathway within LeRobot. You can now use Isaac Teleop to efficiently collect high-quality human demonstration data and adapt Isaac GR00T 1.7 for diverse robot embodiments and tasks. Utilize Isaac Sim/Lab for robust policy testing and prepare for NVIDIA Cosmos 3 to augment data and simulate complex scenarios, streamlining your development cycle and reducing resource fragmentation.
Key insights
Open source collaboration between NVIDIA and Hugging Face standardizes robot development with new models and tools in LeRobot.
Principles
- Open source accelerates robotics innovation.
- Standardized tools reduce development fragmentation.
- Shared models and data drive community progress.
In practice
- Use Isaac Teleop for human demonstration data.
- Adapt Isaac GR00T 1.7 for new robot tasks.
- Utilize Isaac Sim/Lab for robot policy testing.
Topics
- Hugging Face LeRobot
- Robot Foundation Models
- NVIDIA Isaac GR00T
- Physical AI
- Robotics Simulation
- Open-Source Robotics
Best for: Computer Vision Engineer, Robotics Engineer, Machine Learning Engineer, AI Engineer
Related on AIssential
Editorial summary, takeaway, and curation by AIssential. Original article published by NVIDIA Blog.