OnDeFog: Online Decision Transformer under Frame Dropping

2026-06-18 · Source: Artificial Intelligence · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, quick

Summary

OnDeFog, an online decision transformer, is proposed to mitigate performance degradation caused by frame dropping in real-world reinforcement learning applications. Frame dropping, often due to communication delays or sensor failures, prevents agents from receiving states and rewards. While the offline Decision Transformer under Random Frame Dropping (DeFog) previously addressed this, it struggled with generalizing to novel states not in its training data. OnDeFog integrates DeFog's mechanisms with the online decision transformer (ODT), allowing it to learn policies through direct environmental interaction. Experimental evaluation, published on 2026-06-18, demonstrates OnDeFog's superior performance over ODT in environments with high dropping frame rates and its outperformance of DeFog on datasets containing significant low-reward data.

Key takeaway

For Machine Learning Engineers designing robust reinforcement learning systems for real-world deployment, you should consider OnDeFog. This online decision transformer effectively addresses performance degradation from frame dropping, outperforming ODT in high dropping rate scenarios and DeFog on low-reward datasets. Its ability to generalize to novel states through direct environmental interaction makes it a strong candidate for applications facing communication delays or sensor failures.

Key insights

OnDeFog combines offline frame-dropping mitigation with online learning for robust decision-making in real-world, state-missing environments.

Principles

• Online learning improves generalization to novel states.
• Frame dropping degrades RL agent performance.
• Integrating offline mechanisms into online methods can enhance robustness.

Method

OnDeFog integrates DeFog's frame-dropping mitigation mechanisms into the Online Decision Transformer (ODT) architecture, enabling policy learning via direct environmental interaction.

In practice

• Apply OnDeFog in high frame-dropping rate environments.
• Use OnDeFog for datasets with low-reward data.
• Enhance real-world RL systems facing sensor failures.

Topics

• Online Decision Transformers
• Frame Dropping
• Reinforcement Learning
• Policy Generalization
• Robust AI Systems

Best for: Research Scientist, AI Scientist, Machine Learning Engineer

Related on AIssential

• Behavior-Invariant Task Representation Learning with Transformer-based World Models for Offline Meta-Reinforcement Learning — A novel OMRL framework uses behavior-invariant task representations and conservative value penalties to mitigate distribution shifts and enhance generalization in sparse-reward settings.
• Online Learning with Recency: Algorithms for Sliding-window Streaming Multi-armed Bandits — Sliding-window streaming MABs require balancing pure exploration and regret minimization under recency and memory constraints.
• GrowthHacker: Automated Off-Policy Evaluation Optimization Using Code-Modifying LLM Agents — LLM-based agents can autonomously optimize Off-Policy Evaluation code, significantly improving performance and reliability in data-driven systems.
• The Fundamental Choice in Reinforcement Learning: On‑Policy vs. Off‑Policy — Reinforcement learning's core choice is between on-policy learning from current actions and off-policy learning from diverse experiences.
• PowerOPD: Stabilizing On-Policy Distillation with Bounded Power Transformation — PowerOPD stabilizes on-policy distillation by using bounded power-transformed rewards, significantly improving performance and efficiency.
• Theoretical Grounding of Out-Of-Distribution Detection With Reinforcement Learning Optimizer — RL-guided optimizers can improve dynamic OOD detection by explicitly accounting for future environment changes.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Artificial Intelligence.