Seedance Makes A Splash, Nvidia's AI-Guided Chip Designs, Helping Robots Not Forget

2026-05-08 · Source: The Batch | DeepLearning.AI · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems, Emerging Technologies & Innovation · Depth: Intermediate, long

Summary

ByteDance has integrated its multimodal video generation model, Seedance 2.0, into its widely used CapCut video-editing app, making it available to hundreds of millions of users across various regions. This model accepts text, images, audio, and video inputs to produce synchronized video and audio outputs ranging from 4 to 15 seconds, with features like lip-synced dialogue, ambient sound, and multiple camera shots. Seedance 2.0 ranks highly on independent video leaderboards, often placing first or second against competitors like Alibaba's HappyHorse-1.0. The release comes as OpenAI discontinues its Sora app, highlighting a shift in the video generation market where Chinese developers are accelerating their releases, and ByteDance benefits from owning both a powerful generator and a massive user base through CapCut's 736 million monthly active users.

Key takeaway

For AI researchers and robotics engineers developing adaptive systems, this research demonstrates a robust method to mitigate catastrophic forgetting. By combining large pretrained vision-language-action models with LoRA and on-policy reinforcement learning, you can enable robots to learn new tasks sequentially while retaining proficiency in previously acquired skills, crucial for dynamic operational environments.

Key insights

Large pretrained models, LoRA, and on-policy reinforcement learning reduce catastrophic forgetting in sequential robotics task learning.

Principles

• Small model updates preserve existing knowledge.
• LoRA limits model change during inference.
• On-policy RL rewards actions, limiting updates.

Method

Fine-tune a large pretrained VLA model (OpenVLA-OFT) on sequential robotics tasks using GRPO and LoRA, without reusing prior task data, to minimize catastrophic forgetting.

In practice

• Apply LoRA for efficient model adaptation.
• Use GRPO for stable reinforcement learning.
• Integrate VLA models for complex robot control.

Topics

• AI Workforce Impact
• Video Generation AI
• NVIDIA Chip Design
• Reinforcement Learning
• Catastrophic Forgetting

Code references

• Tencent-Hunyuan/HunyuanWorld-1.0

Best for: CTO, Research Scientist, AI Product Manager, AI Scientist, Director of AI/ML, AI Engineer

Related on AIssential

• Towards Continual Learning — Continual learning, crucial for human-like AI, involves updating models either through weight adjustments or external context/harness modifications.
• Fine-Tuning NVIDIA Cosmos Predict 2.5 with LoRA/DoRA for Robot Video Generation — LoRA/DoRA fine-tuning effectively adapts large world models like Cosmos Predict 2.5 for domain-specific robot video generation.
• Mask the Target: A Plug-and-Play Regularizer Against LoRA Forgetting — LoRA fine-tuning can cause significant forgetting, which a new output space regularizer effectively mitigates without replay data.
• ARROW: Augmented Replay for RObust World models — Augmented replay to a World Model significantly reduces catastrophic forgetting in continual reinforcement learning.
• PHASER: Phase-Aware and Semantic Experience Replay for Vision-Language-Action Models — PHASER prevents catastrophic forgetting in VLA models by intelligently prioritizing critical sub-skills and high-risk phases during experience replay.
• Control-Plane Placement Shapes Forgetting: An Architectural Study of Agent Memory Across Thirteen System Configurations — LLM placement within an agent's memory control plane critically determines recovery from various forgetting failure modes.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by The Batch | DeepLearning.AI.