Separable Expert Architecture: Toward Privacy-Preserving LLM Personalization via Composable Adapters and Deletable User Proxies

2026-04-23 · Source: Artificial Intelligence · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Cybersecurity & Data Privacy · Depth: Expert, quick

Summary

A novel three-layer architecture is proposed to address the computational infeasibility of individual data removal in large language models (LLMs) by decoupling personal data from shared model weights. This architecture integrates a static base model, composable domain-expert LoRA adapters for behavior shaping without user data, and per-user proxy artifacts. The deletion of these proxies enables deterministic unlearning. Evaluation on Phi-3.5-mini and Llama-3.1-8B models demonstrates effective per-user differentiation, where personal data influences outputs while remaining isolated. Verification after proxy removal shows a return to baseline (KL divergence of approximately 0.21 nats, 82-89% pass rate) and near-zero cross-user contamination. This design inherently mitigates model inversion, membership inference, and training-data extraction against shared components and is compatible with differentially private stochastic gradient descent (DP-SGD).

Key takeaway

For engineering teams building personalized LLM applications, this architecture offers a robust solution for integrating user data while ensuring privacy and compliance with data deletion requests. Your ability to deterministically remove user data by deleting proxies simplifies machine unlearning, reducing the operational overhead and legal risks associated with data retention. Consider adopting this approach to enhance user trust and streamline data governance.

Key insights

A three-layer architecture enables privacy-preserving LLM personalization through composable adapters and deletable user proxies.

Principles

• Decouple personal data from shared weights.
• Unlearning via deterministic proxy deletion.

Method

Combine a static base model, domain-expert LoRA adapters, and per-user proxy artifacts. Personalization occurs via proxies, unlearning via proxy deletion.

In practice

• Apply LoRA adapters for behavior shaping.
• Isolate user data in deletable proxies.

Topics

• Separable Expert Architecture
• LLM Personalization
• Privacy-Preserving AI
• Machine Unlearning
• LoRA Adapters

Best for: Research Scientist, CTO, VP of Engineering/Data, AI Scientist, Machine Learning Engineer, AI Security Engineer

Related on AIssential

• Auditing Training Data in Domain-adapted LLMs: LoRA-MINT — LoRA-MINT uses perplexity and synthetic samples to audit training data membership in LoRA-fine-tuned LLMs.
• MetaMoE: Diversity-Aware Proxy Selection for Privacy-Preserving Mixture-of-Experts Unification — MetaMoE unifies distributed MoE models using diversity-aware public proxy data to preserve privacy and enhance expert coordination.
• RepSelect: Robust LLM Unlearning via Representation Selectivity — RepSelect achieves robust LLM unlearning by selectively targeting forget-set-specific representations, preventing easy reversal.
• Privacy from Symmetry: Orthogonally Equivariant Transformers for LLM Inference — Enforcing architectural symmetry via orthogonal obfuscation and equivariant transformers provides practical privacy for LLM inference.
• Your AI Doesn’t Know Anything. And That’s Not the Model’s Fault. — AI hallucinations stem from infrastructure limitations, not model intelligence, necessitating robust knowledge layers like graphs.
• Re-Centering Humans in LLM Personalization — LLM personalization evaluations relying on synthetic data significantly misrepresent real-world human experience and model limitations.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Artificial Intelligence.