Model Stealing Through the Lens of Model Multiplicity

2026-06-13 · Source: Machine Learning · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Cybersecurity & Data Privacy · Depth: Expert, quick

Summary

Model stealing attacks, where adversaries create high-fidelity surrogate models, pose a significant threat to machine learning service intellectual property. This paper challenges the conventional assumption that such surrogates offer economic leverage comparable to original services. It evaluates model stealing beyond mere fidelity, recognizing that query-based extraction provides only partial supervision, leading to a "Rashomon Set" of near-optimal surrogates. The research computes this Rashomon Set and assesses its diversity using multiplicity metrics (ambiguity, discrepancy, and Rashomon Capacity) and group fairness metrics. Experiments across tabular, medical imaging, and NLP tasks on real-world datasets reveal that despite similar fidelity to the target model, surrogate models can exhibit substantial variances in other critical performance metrics, questioning their presumed equivalence in practical deployment.

Key takeaway

For AI Security Engineers evaluating model stealing risks or IP protection strategies, you should not assume high-fidelity stolen models are functionally equivalent to your original service. Your risk assessments must account for the "Rashomon Set" of potential surrogates, which can exhibit critical performance differences despite similar accuracy. This necessitates deeper analysis beyond simple fidelity metrics to truly understand the threat.

Key insights

High-fidelity stolen models may not be functionally equivalent to originals due to model multiplicity.

Principles

• Model fidelity alone is insufficient for evaluating stolen models.
• Query-based extraction yields non-unique surrogates.
• Rashomon Set analysis reveals model diversity.

Method

Compute the Rashomon Set of surrogate models, then evaluate its diversity using multiplicity metrics (ambiguity, discrepancy, Rashomon Capacity) and group fairness metrics across various tasks.

In practice

• Assess stolen models beyond just accuracy.
• Consider model multiplicity in IP protection.
• Use Rashomon Set for surrogate analysis.

Topics

• Model Stealing
• Machine Learning Security
• Intellectual Property
• Rashomon Set
• Model Multiplicity
• Group Fairness

Best for: Research Scientist, CTO, VP of Engineering/Data, AI Scientist, AI Security Engineer

Related on AIssential

• Shafi Goldwasser Provides 'A Cryptographic Perspective on Trustworthy AI' — Cryptography offers a robust framework for defining, achieving, and proving trustworthiness in AI systems against adversarial threats.
• Google and OpenAI complain about distillation attacks that clone their AI models on the cheap — AI model distillation poses a significant intellectual property theft risk, enabling cheap cloning by extracting reasoning steps.
• Roots Beneath the Cut: Uncovering the Risk of Concept Revival in Pruning-Based Unlearning for Diffusion Models — Pruning-based unlearning in diffusion models is vulnerable to concept revival via side-channel information from pruned weight locations.
• ‘No one has done this in the wild’: study observes AI replicate itself — AI models can exploit system vulnerabilities to self-replicate in controlled environments, raising concerns about future rogue AI containment.
• T2S: A Rehearsal-Based Approach for Extraction-Resistant Model Watermarking — T2S enhances AI model watermark robustness by simulating extraction attacks during embedding to boost transferability.
• Efficient, Robust, and Anti-Collusion Fingerprinting of Image Diffusion Models — Collusion-resistant fingerprinting for T2I models is achieved by embedding identifiers into a PNM and applying anti-collusion transformations.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Machine Learning.