Differential Privacy of Gaussian Process Posterior Sampling

2026-06-10 · Source: stat.ML updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Cybersecurity & Data Privacy · Depth: Expert, extended

Summary

A study on Differential Privacy (DP) for Gaussian Process (GP) posterior sampling reveals that the intrinsic randomness of this process provides DP guarantees without requiring external noise. Researchers derived explicit Rényi-DP bounds, distinguishing between posterior-mean leakage and data-dependent posterior-covariance leakage, highlighting that effective ridge regularization is crucial for meaningful privacy. Empirical validation using membership-inference attacks demonstrated that leakage aligns with predicted dependencies on regularization, posterior variance, and the number of released sample paths. Utility experiments, particularly for downstream tasks like excursion-set estimation, showed that privacy-compatible regularization can maintain useful decisions with only modest utility loss, especially in noisy observation environments where the utility-optimal posterior is already substantially regularized. The findings also indicate that adding calibrated GP noise can further enhance these intrinsic DP guarantees.

Key takeaway

For AI Scientists developing or deploying Gaussian Process models with private training data, you should prioritize effective ridge regularization to achieve robust differential privacy. This intrinsic randomness-based approach offers meaningful privacy guarantees, particularly in noisy observation settings where utility loss is modest. Consider adding calibrated GP noise to further strengthen privacy without relying solely on posterior scale.

Key insights

Intrinsic randomness in GP posterior sampling provides differential privacy, critically dependent on effective ridge regularization.

Principles

• Intrinsic model randomness can inhibit data inference.
• Effective ridge regularization is essential for GP privacy.
• Posterior scale alone is insufficient for privacy.

Method

The method involves deriving explicit Rényi-DP bounds for GP posterior sample-path release, separating posterior-mean and data-dependent posterior-covariance leakage, and validating with membership-inference attacks and utility experiments.

In practice

• Use effective ridge regularization for GP privacy.
• Consider adding calibrated GP noise for stronger DP.
• Evaluate privacy-utility tradeoffs in noisy observation regimes.

Topics

• Differential Privacy
• Gaussian Processes
• Rényi-DP Bounds
• Ridge Regularization
• Membership Inference Attacks
• Excursion Set Estimation

Code references

• tmaciazek/gaussian_process_dp

Best for: Research Scientist, AI Scientist, AI Security Engineer

Related on AIssential

• Differential Privacy in 30 seconds — Differential Privacy protects individual data by clipping gradients and adding noise during model training.
• How probability models protect privacy — Differential privacy uses calibrated randomness to protect individual data while enabling accurate aggregate analysis.
• Let's Ask Gauss: Improved One-Run Privacy Auditing — A Gaussian distribution perspective on canary signals yields tighter privacy bounds in one-run DP auditing.
• Revisiting Privacy Amplification by Subsampling in Selective Release DPSGD — DPSR-CG rigorously re-evaluates privacy accounting for selective release in DPSGD to achieve high utility with strict guarantees.
• DP-CDA: An Algorithm for Enhanced Privacy Preservation in Dataset Synthesis Through Randomized Mixing — DP-CDA generates privacy-preserving synthetic datasets by class-specific random mixing and Gaussian noise, optimizing utility and privacy.
• Benchmarking Empirical Privacy Protection for Adaptations of Large Language Models — Distribution shifts critically impact practical privacy in differentially private LLM adaptations, often undermining theoretical guarantees.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by stat.ML updates on arXiv.org.