Information-Directed Sampling for Causal Bandits

2026-07-17 · Source: Artificial Intelligence · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, quick

Summary

Causal bandits with non-manipulable variables are explored in a new study, addressing scenarios where some variables influence rewards but cannot be directly manipulated. The research focuses on contextual causal bandits, observing context variables before action selection and additional variables post-intervention, assuming a known causal graph without latent confounding. Adopting a Bayesian formulation, the conditional probability tables of the observational distribution serve as the unknown parameter, enabling observations from one intervention to update reward estimates for others via shared causal mechanisms. The authors develop causal variants of Thompson Sampling and Information-Directed Sampling (IDS). For Thompson Sampling, an entropy-dependent sublinear Bayesian regret bound is established. For IDS, an entropy-dependent regret bound is derived, explicitly quantifying Monte Carlo approximation error and recovering the standard sublinear IDS rate when exact. High-probability confidence bounds for Monte Carlo estimates are also provided. Experiments on synthetic tasks show these methods outperform causal and non-causal baselines by more effectively exploiting cross-intervention information.

Key takeaway

For research scientists developing adaptive decision-making systems, this work demonstrates that incorporating non-manipulable variables into causal bandit frameworks significantly enhances learning efficiency. You should explore causal variants of Thompson Sampling and Information-Directed Sampling, particularly when structural relationships allow information sharing, to achieve superior regret performance compared to non-causal baselines. This approach offers a robust method for optimizing interventions in complex systems where not all influencing factors are directly controllable.

Key insights

Causal bandits with non-manipulable variables can leverage shared causal mechanisms for improved information-directed sampling and regret bounds.

Principles

• Causal graphs enable information sharing across interventions.
• Bayesian formulation updates reward estimates via shared mechanisms.
• Non-manipulable variables provide useful system information.

Method

Develops causal variants of Thompson Sampling and Information-Directed Sampling (IDS) for contextual causal bandits, using a Bayesian formulation with conditional probability tables as unknown parameters.

In practice

• Apply causal IDS to tasks with non-manipulable variables.
• Use Bayesian updates for cross-intervention information sharing.
• Quantify Monte Carlo error in regret bounds.

Topics

• Causal Bandits
• Information-Directed Sampling
• Thompson Sampling
• Bayesian Inference
• Regret Bounds
• Contextual Bandits
• Non-Manipulable Variables

Best for: AI Scientist, Research Scientist

Related on AIssential

• Certified Policy Optimisation for Nested Causal Bandits via PAC-Bayes Risk — Nested Causal Thompson Sampling (NCTS) offers certified policy optimization for hierarchical causal decision-making under distribution shifts.
• Correlation-Aware Contextual Bandits with Surrogate Rewards for LLM Routing — Contextual bandits for LLM routing can leverage correlated arms and noisy surrogate rewards for improved efficiency and robustness.
• DIY AI & ML: Solving The Multi-Armed Bandit Problem with Thompson Sampling — Thompson Sampling automates decision-making by balancing exploration and exploitation to optimize outcomes faster than A/B testing.
• Causal models for decision systems: an interview with Matteo Ceriscioli — Adaptable AI agents inherently possess causal knowledge, enabling robust decision-making under distribution shifts.
• Maximum Entropy Inverse Reinforcement Learning for Mean-Field Games with Average Reward — Recovering unknown reward policies in average-reward MFGs is feasible using maximum causal entropy and a novel contraction method.
• Reward-seekers will probably behave according to causal decision theory — Reward-seeking AI agents are likely to generalize Causal Decision Theory (CDT) behavior, reducing inter-agent collusion.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Artificial Intelligence.