Dissecting Performative Prediction: A Comprehensive Survey
Summary
This survey, "Dissecting Performative Prediction," by Sanguino Bautista, Lozano, and Quadrianto, comprehensively reviews the field of performative prediction (PP), which originated in 2020. PP describes a machine learning scenario where a deployed predictive model causes a distribution shift in the environment, leading to a mismatch between the model's expected and real data distributions. The authors introduce a new classification scheme for PP settings based on the available information about the "distribution map," $\mathcal{D}(\cdot)$, which formalizes this shift. The paper details different optimization targets: performative stability (a fixed point where the model minimizes risk on its self-induced distribution) and performative optimality (the global minimum of performative risk). It also surveys existing distribution map implementations, methods to address PP, and categorizes them. Finally, the survey highlights connections between PP and other fields like adversarial attacks, algorithmic recourse, and fairness, aiming to stimulate future research.
Key takeaway
Research Scientists developing predictive models should recognize that model deployment can inherently alter data distributions, necessitating a shift from traditional empirical risk minimization. You must consider whether to optimize for performative stability, which is often easier to achieve through iterative retraining, or performative optimality, which offers lower long-term risk but requires more sophisticated methods like performative gradient descent and explicit distribution map models. Carefully assess your access to distribution map information (e.g., cheap simulations vs. expensive real-world samples) to select the most appropriate optimization strategy, as this directly impacts model robustness and long-term accuracy.
Key insights
Performative prediction analyzes how deployed models induce data distribution shifts, requiring new optimization targets like stability and optimality.
Principles
- Model deployment can cause distribution shifts.
- Performative stability is easier to achieve than optimality.
- Distribution map knowledge dictates optimization strategy.
Method
Performative prediction problems involve two steps: obtaining information about the distribution map (e.g., via mathematical models or samples) and then solving an optimization problem to find performatively stable or optimal models, often iteratively.
In practice
- Use Repeated Risk Minimization for cheap sample access.
- Employ performative gradient descent with differentiable distribution maps.
- Consider outcome performativity for simpler model construction.
Topics
- Performative Prediction
- Distribution Shift
- Performative Optimality
- Machine Learning Optimization
- Strategic Classification
Best for: Research Scientist, AI Researcher, AI Scientist, Machine Learning Engineer
Related on AIssential
Editorial summary, takeaway, and curation by AIssential. Original article published by stat.ML updates on arXiv.org.