COPF: An Online Framework for Deployment-Stable Counterfactual Fairness in Evolving Graphs
Summary
COPF, a decision-layer framework, addresses deployment-stable fairness monitoring and control in online link recommendation on evolving graphs. It tackles the performative nature of such systems, where policy updates can cause fairness estimates to drift. COPF defines group-level opportunity gaps using exposure counterfactuals, making them estimable through explicit exploration and logging candidate propensities. The framework audits and controls fairness via residual outcome indistinguishability (OI) over a configurable auditor family, employing graph-aware doubly robust (GA-DR) estimators. A noisy transfer theorem supports Residual-OI's implication for bounding exposure-counterfactual group gaps. Instantiating an online multicalibration auditor with a primal-dual controller, COPF experiments on two TGB streams and a synthetic bipartite stream demonstrate its ability to reduce worst-case spikes in exposure-counterfactual group disparities with only a modest impact on ranking utility. The code is available on GitHub.
Key takeaway
For Machine Learning Engineers deploying online link recommendation systems, traditional fairness metrics often drift due to the performative nature of these evolving graphs. COPF offers a robust framework to monitor and control deployment-stable counterfactual fairness by explicitly estimating exposure gaps. Consider adopting such methods, like COPF's GA-DR estimators and multicalibration auditors, to mitigate worst-case disparity spikes and ensure more stable, equitable system behavior post-deployment, thereby improving long-term system integrity.
Key insights
COPF ensures deployment-stable counterfactual fairness in online link recommendation on evolving graphs by addressing performativity.
Principles
- Performative systems require explicit exploration for accurate fairness estimation.
- Fairness estimates can drift post-deployment in evolving recommendation systems.
- Residual outcome indistinguishability can bound exposure-counterfactual group gaps.
Method
COPF defines opportunity gaps, estimates them via exploration and propensity logging, then audits and controls fairness using residual outcome indistinguishability with GA-DR estimators.
In practice
- Use GA-DR estimators for robust fairness auditing in graph data.
- Implement explicit exploration to estimate counterfactuals in performative systems.
- Apply multicalibration auditors for online fairness control.
Topics
- Counterfactual Fairness
- Online Recommendation Systems
- Evolving Graphs
- Performative Prediction
- Doubly Robust Estimators
- Fairness Auditing
Code references
Best for: Research Scientist, AI Scientist, Machine Learning Engineer, AI Ethicist
Related on AIssential
Editorial summary, takeaway, and curation by AIssential. Original article published by Artificial Intelligence.