NVIDIA and Hugging Face Bring New Models and Frameworks to LeRobot for the Open Robotics Community

· Source: NVIDIA Blog · Field: Technology & Digital — Robotics & Autonomous Systems, Artificial Intelligence & Machine Learning · Depth: Advanced, quick

Summary

NVIDIA and Hugging Face are integrating key NVIDIA physical AI capabilities into LeRobot, Hugging Face's open source robotics library. This collaboration introduces the NVIDIA Isaac GR00T 1.7 reasoning vision language action (VLA) model for humanoid robots and the NVIDIA Isaac Teleop framework, with the NVIDIA Cosmos 3 frontier physical AI model planned for future integration. These additions aim to standardize and simplify end-to-end robot development, fostering innovation within the open robotics community by connecting NVIDIA's 3 million robotics developers with Hugging Face's 16 million AI builders. The integrations facilitate common workflows for data collection, model training, fine-tuning, performance evaluation, and deployment. Isaac Teleop enables high-quality human demonstration capture and dataset sharing. Isaac GR00T 1.7 supports post-training and deployment across various robot embodiments. Cosmos 3 will enhance data generation and simulation. Existing NVIDIA resources, including a physical AI dataset with over 350,000 trajectories and 57 million grasps, Isaac Sim/Lab simulation frameworks, and Jetson Thor integration, further support the full robotics development loop within LeRobot.

Key takeaway

For Robotics Engineers developing and deploying physical AI, this collaboration offers a standardized, open source pathway within LeRobot. You can now use Isaac Teleop to efficiently collect high-quality human demonstration data and adapt Isaac GR00T 1.7 for diverse robot embodiments and tasks. Utilize Isaac Sim/Lab for robust policy testing and prepare for NVIDIA Cosmos 3 to augment data and simulate complex scenarios, streamlining your development cycle and reducing resource fragmentation.

Key insights

Open source collaboration between NVIDIA and Hugging Face standardizes robot development with new models and tools in LeRobot.

Principles

In practice

Topics

Best for: Computer Vision Engineer, Robotics Engineer, Machine Learning Engineer, AI Engineer

Related on AIssential

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by NVIDIA Blog.