Revisiting Privacy Amplification by Subsampling in Selective Release DPSGD

2026-06-03 · Source: Takara TLDR - Daily AI Papers · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Cybersecurity & Data Privacy · Depth: Expert, quick

Summary

The DPSR-CG (Differentially Private Selective Release based on Clipped Gradients) algorithm is introduced to enhance privacy-preserving machine learning, specifically addressing utility degradation and slow convergence in Differentially Private Stochastic Gradient Descent (DPSGD). Prior work, like DPSUR, achieved high model utility but had compromised privacy guarantees due to overlooking sampling probability variations in its selective release mechanism. DPSR-CG re-evaluates this privacy analysis, proposing a novel algorithm with a rigorous, newly derived privacy analysis. Extensive experiments on datasets including MNIST, CIFAR-10, IMDB, and FMNIST demonstrate that DPSR-CG maintains strict privacy guarantees while achieving exceptional model performance.

Key takeaway

For Machine Learning Engineers developing privacy-preserving models, DPSR-CG offers a robust solution to the long-standing trade-off between utility and privacy. If you are struggling with utility degradation or slow convergence in DPSGD, consider implementing DPSR-CG. Its rigorous privacy analysis and demonstrated performance on datasets like CIFAR-10 and IMDB suggest it can deliver exceptional model performance while maintaining strict privacy guarantees.

Key insights

DPSR-CG rigorously re-evaluates privacy accounting for selective release in DPSGD to achieve high utility with strict guarantees.

Principles

• Rigorous privacy accounting is crucial for selective release mechanisms.
• Sampling probability variation impacts privacy guarantees in DPSGD.

Method

DPSR-CG proposes a novel algorithm by re-evaluating the privacy analysis of selective release mechanisms and incorporating clipped gradients for improved utility and strict privacy.

In practice

• Apply DPSR-CG for improved utility in DPSGD models.
• Test DPSR-CG on image and text datasets like MNIST or IMDB.

Topics

• Differentially Private Stochastic Gradient Descent
• Differential Privacy
• Privacy Amplification
• Selective Release
• DPSR-CG
• Model Utility

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, AI Security Engineer

Related on AIssential

• DP-CDA: An Algorithm for Enhanced Privacy Preservation in Dataset Synthesis Through Randomized Mixing — DP-CDA generates privacy-preserving synthetic datasets by class-specific random mixing and Gaussian noise, optimizing utility and privacy.
• COPF: An Online Framework for Deployment-Stable Counterfactual Fairness in Evolving Graphs — COPF ensures deployment-stable counterfactual fairness in online link recommendation on evolving graphs by addressing performativity.
• PPEDCRF: Privacy-Preserving Enhanced Dynamic CRF for Location-Privacy Protection for Sequence Videos with Minimal Detection Degradation — PPEDCRF protects video location privacy by perturbing only sensitive background regions, preserving foreground detection utility.
• Differentially Private Conformal Prediction — DPCP offers statistically efficient, end-to-end differentially private conformal prediction without data splitting.
• Differential Privacy in 30 seconds — Differential Privacy protects individual data by clipping gradients and adding noise during model training.
• Asymptotically Fair and Truthful Allocation of Public Goods — PPGA offers an asymptotically truthful and core-finding algorithm for fair public-good allocation, addressing scalability and implementation challenges.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Takara TLDR - Daily AI Papers.