Phantoms and Disclosures: a Causal Framework for Auditing Synthetic Data

2026-02-07 · Source: stat.ML updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Cybersecurity & Data Privacy, Data Science & Analytics · Depth: Expert, extended

Summary

A new causal empirical auditing framework is introduced to detect and explain data disclosures in synthetic data, distinguishing between true disclosures, where a system directly reproduces user information, and phantom disclosures, where data is incidentally generated. The framework partitions input data into training and holdout sets, applying rigorous statistical hypothesis testing to determine if observed disclosures are consistent with strict privacy baselines like zero-learning or specific Differential Privacy (DP) bounds. This approach requires no model access, canary insertion, or reference model training, relying only on synthetic output and a held-out control set. It functions as a membership inference attack, providing tighter empirical lower bounds on privacy leakage than prior data-based methods. The framework is model-agnostic, computationally efficient, and effectively detects significant disclosures in non-private synthetic data while validating DP-SGD models by showing disclosures are primarily phantoms.

Key takeaway

For privacy auditors or ML engineers evaluating synthetic data for release, you must distinguish between true and phantom disclosures. Naive disclosure counts significantly overstate privacy risk, as over 35% can be phantoms. Implement this causal auditing framework to obtain statistically rigorous, empirical lower bounds on privacy leakage, validating formal privacy protections like Differential Privacy and ensuring accurate risk assessment before deployment.

Key insights

A causal auditing framework distinguishes true from phantom synthetic data disclosures using statistical tests and held-out data.

Principles

• Synthetic data quality often conflicts with privacy.
• "Phantom disclosures" inflate privacy risk metrics.
• Auditing synthetic data requires model-agnostic methods.

Method

Partition data into training and holdout sets. Generate synthetic data from training. Extract rare features. Quantify disclosures in synthetic data against both sets using statistical hypothesis tests (zero-learning, DP-bounded learning).

In practice

• Use Google Cloud DLP for PII detection.
• Apply Gemini Embedding 2 for semantic similarity.
• Employ n-gram matching for verbatim regurgitation.

Topics

• Synthetic Data Auditing
• Differential Privacy
• Membership Inference Attacks
• Generative AI Privacy
• Data Disclosure Detection
• Large Language Models

Best for: Research Scientist, CTO, VP of Engineering/Data, AI Scientist, AI Security Engineer, Machine Learning Engineer

Related on AIssential

• Phantoms and Disclosures: a Causal Framework for Auditing Synthetic Data — The framework distinguishes true from phantom data disclosures in synthetic data using statistical testing, requiring no model access.
• Your Synthetic Data Passed Every Test and Still Broke Your Model — Standard synthetic data metrics often miss critical issues like feature correlations, tail performance, and attribute inference risks.
• How probability models protect privacy — Differential privacy uses calibrated randomness to protect individual data while enabling accurate aggregate analysis.
• Auditing Model Bias with Balanced Datasets with Mimesis — The Mimesis library enables generating balanced, counterfactual data to isolate and audit model bias effectively.
• Equivalence Testing Under Privacy Constraints — DP-TOST offers a simulation-based framework for privacy-preserving equivalence testing of means and proportions.
• Disparate Impact in Synthetic Data Generation — Disparate impact in synthetic data generation occurs when utility varies across sensitive groups, ideally resolved by matching real and synthetic distributions.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by stat.ML updates on arXiv.org.