Set our LLM data retention policy now, or wait for an incident to force it?
The EU AI Act mandates deterministic execution replay by August 2, 2026, but a twenty-million-log court order proves zero-retention policies offer no protection, leaving customer data exposed.
The question
We send customer data to multiple LLM providers daily — chat history, embeddings, RAG context, function-call inputs. Our retention policy today is the providers' defaults. The EU AI Act traceability obligations and our SOC2 audit are both pushing toward a documented policy. Do we author and enforce a unified policy now — before the next incident or audit forces it — and what should it commit to?
Counsel's position
Proactively author and enforce a unified, minimal LLM data retention policy now, committing to the shortest period necessary for business and regulatory needs.
Verdict
The verdict: Proactively author and enforce a unified, minimal LLM data retention policy now, committing to the shortest period necessary for business and regulatory needs.
How the criteria decide
3 of 3 criteria resolved on cited evidence.
| Criterion | Favours | Evidence |
|---|---|---|
| LLM data retention and provider DPA configuration | Both equally | A twenty-million-log court order bypassed zero-retention policies The twenty-million-log order didn’t happen because a company lied about its retention policy. It happened because a policy — any policy — is a sentence a court can rewrite. Artificial Intelligence on Medium Major enterprise AI services exclude customer data from training OpenAI, Google, Microsoft, Anthropic and xAI all offer commercial or enterprise services in which customer data is not used for model training by default or without the customer’s permission. AI harnesses capture user trajectories to train vendor models Unlike in SaaS, where this data lived in databases accessible only to the customer, trajectories can be fed back into a model to improve AI. |
| GDPR and EU AI Act data traceability for AI workflows | Both equally | EU AI Act mandates deterministic execution replay by August 2, 2026 If you cannot deterministically replay the exact line of machine reasoning from the initial user instruction to the final API payload, you aren’t deploying an enterprise agent. Artificial Intelligence in Plain English - Medium AI Act and GDPR compliance require separate EU representatives The AI Act does not care whether the underlying data was lawfully transferred under GDPR Chapter V. Each framework operates independently, and neither provides a safe harbor for compliance with the other. |
| Audit logging for LLM data flows | Both equally | EU AI Act mandates deterministic execution replay by August 2, 2026 If you cannot deterministically replay the exact line of machine reasoning from the initial user instruction to the final API payload, you aren’t deploying an enterprise agent. |
EU AI Act mandates deterministic execution replay by August 2, 2026
The EU AI Act evaluates the emergent trajectory of agentic pipelines, requiring organizations to capture and reconstruct the state and intermediate tool outputs at every node.
A twenty-million-log court order bypassed zero-retention policies
Vendor "zero retention" policies are contractual promises that can be overridden by judicial mandates, whereas architectural controls structurally prevent access.
AI Act and GDPR compliance require separate EU representatives
Training or deploying AI systems with EU personal data triggers overlapping but distinct obligations under the GDPR and the EU AI Act.
Major enterprise AI services exclude customer data from training
The primary data risk in generative AI stems from secondary use and provider data policies, not the inherent nature of machine learning inference.
AI harnesses capture user trajectories to train vendor models
The software interfaces used to interact with AI models capture proprietary enterprise data, which can be fed back into the models as training data.
Read another verdict
- Start with a small test, or take on the whole process at once?
- Our competitors advertise AI and we don't — match them, or hold the line?
- Our people already put client files into ChatGPT — ban it, frame it, or supply a tool?
- Our most experienced person retires in two years — how do we keep what they know?
- We can't hire the experienced people we need — automate, train up, or outsource?
- Slow our EU AI Act prep now the deadline's moved to 2027?
- Use AI to flatten middle management this year?
- Let an AI agent act on its own — or keep a human in the loop?