The AI Data Privacy Checklist I Run Before Shipping a Feature

A prompt leaves your process as a string. It comes back as a string. In between it crosses a boundary you do not control, lands in a log you did not configure and sits there for a number of days you probably cannot name. That number is what AI data privacy reduces to in practice. Not the ethics panel. The number, and who can read it.

Engineering leads get handed this problem sideways. Legal forwards a questionnaire. Someone asks whether the feature is GDPR compliant, as if that were a boolean. Nobody tells you that the fourteen API endpoints your service calls have at least four different retention behaviours between them.

So this is the version I would want. Not a comparison of regimes. A list of things to configure, verify and write down.

What happens to a prompt after it leaves your process

Start with the two questions that get collapsed into one. Is my data used for training, and how long is it stored. They are separate, and the answers diverge per endpoint inside a single provider.

Training first, because it is the easy one. OpenAI has stated since 1 March 2023 that API inputs and outputs are not used to train or improve its models unless a customer opts in. Anthropic's commercial terms say retained data is never used for model training without express permission. Google says the same for billing-enabled Gemini projects. The free tier is a different product: content sent through unpaid Gemini quota can be used to develop Google products, with human review in scope.

Retention is where it gets untidy. Most OpenAI endpoints keep abuse-monitoring logs for up to 30 days. But /v1/conversations, /v1/conversations/items and /v1/chatkit/threads hold data until you delete it, while /v1/audio/transcriptions, /v1/audio/translations and /v1/moderations store no abuse logs at all. Zero Data Retention, when approved, covers /v1/chat/completions, /v1/responses, /v1/embeddings, /v1/images/generations and the audio endpoints, and it forces the store parameter to false whatever your request says.

Anthropic's arrangement is per organisation and has to be enabled by an account team, not toggled in a console. Under it, prompts and responses are not stored at rest once the response returns. Then read the exceptions. Batch processing retains for 29 days. Files persist until deleted or expired. Code execution containers hold data up to 30 days. Content flagged by automated trust and safety systems may be kept for up to two years regardless of your arrangement. Two models designated Covered Models require 30-day retention and will return a 400 invalid_request_error to a zero-retention organisation that has not enabled retention on that specific workspace. And CORS is unsupported for those organisations, so browser calls must route through your own backend.

Gemini logs on paid projects default to 55 days, adjustable to 7, 14 or 28. Zero retention is available on approval, with named holes: Search grounding and Maps grounding carry 30-day retention and no opt-out, and the Interactions API stores by default until you set store to false. Azure's Modified Abuse Monitoring is a gated application with a business justification, not a checkbox, and only for enterprise agreement customers.

None of this is hidden. It is just spread across seven pages that change, and nobody on the team owns reading them.

If you cannot answer which rows belong to this person with a single query, you do not have an erasure process. You have a hope.

Personal data in prompts, and the vector store nobody classified

Prompts are an ingestion channel you do not control. A support agent pastes an email thread. A user drops a screenshot of an invoice into a free-text box. Your prompt template was clean; the runtime payload was not.

The retrieval layer is worse, because it accumulates. A pgvector table or a Qdrant collection built from internal documents will contain names, staff numbers, addresses and complaint narratives within weeks, and almost nobody classifies it as a personal data store at design time. It just looks like an index.

Embedding is not anonymisation, and this has been demonstrated rather than argued. Morris and colleagues at EMNLP 2023 showed that an iterative inversion method recovered 92% of 32-token inputs exactly from their embeddings alone. A float array is a reversible representation of the text that produced it.

The EDPB's draft Guidelines 02/2026 on anonymisation, adopted in July 2026 and open for consultation until 30 October 2026, set a three-part test: data is anonymous only when singling out, linkage and inference are all sufficiently unlikely. They follow the Court of Justice ruling in C-413/23 P, EDPS v SRB, of 4 September 2025, which treats identifiability as relative to who holds the data and what else they can reach. Your vector store fails that test on the first limb.

The fix is boring and has to happen at write time. Every chunk carries a subject identifier, a source system, a lawful basis and a retention date. Retrofitting provenance onto four million embedded chunks is a quarter of work nobody budgets for.

AI data privacy and the erasure problem

Article 17 gives a data subject the right to have their personal data erased. Your model has been fine-tuned. The question is whether that request reaches the weights.

Two regulators have taken visibly different positions. The Hamburg data protection authority's discussion paper of 15 July 2024 argued that an LLM does not store personal data, because trained parameters lack the direct association to individuals that the definition requires, so access, rectification and erasure claims attach to the input and the output of the deployed system rather than to the model. The EDPB's Opinion 28/2024, adopted 18 December 2024 at the Irish DPC's request, declined to go that far: an AI model trained on personal data is not automatically anonymous, and anonymity has to be assessed case by case against the likelihood of extracting training data, including unintentionally, through queries.

Neither position has been settled by a court that binds anyone. Both have been live for nearly two years.

What is settled is the cost of the naive remedy. Exact unlearning means retraining without the record. Llama 2 70B consumed roughly 1.7 million A100 GPU-hours. You are not doing that because one person in Düsseldorf filled in a form.

So the architecture has to answer the question instead. Keep person-level data out of the weights. Fine-tune on task structure, de-identified patterns or synthetic examples, and hold the person-specific content in retrieval where deletion is a DELETE, an index rebuild and an output filter. That gives you a defensible answer with a runbook behind it. The alternative is a paragraph explaining why erasure is technically infeasible, which is a position, not a process.

Residency, and when self-hosting is the only honest answer

Residency is where the compliance question stops being about clauses and starts being about hardware.

China's Provisions on Promoting and Regulating Cross-Border Data Flows, issued 22 March 2024, set the tiers plainly. Under 100,000 individuals in a calendar year and no standard contract is required. Between 100,000 and one million non-sensitive records, or under 10,000 sensitive records, you need a standard contract or certification. Above that, a CAC security assessment, valid three years. Nigeria's General Application and Implementation Directive took effect on 19 September 2025 and replaced the 2019 regulation and its 2020 framework, which means the compliance posture Nigerian teams built five years ago is no longer the operative one.

On a production floor in the Pearl River Delta I watched a line supervisor explain a defect cluster by pasting the shift roster into a chat window, because the roster was the fastest way to say which cell ran that batch. Names, employee numbers, three shifts. That is not a policy failure. That is what a useful tool looks like to someone under quota pressure, and it is why the control has to sit in the code path rather than in a training deck.

Sometimes the only compliant design is that the bytes never leave. vLLM and LMDeploy for serving, Ollama for the developer box, open weights from Qwen, DeepSeek or Gemma 4 that are genuinely good enough for extraction, classification and retrieval-augmented answering. In Huaqiangbei I have seen memory-modified 48GB RTX 4090 cards sold openly across the counter, which changes the arithmetic on what a single-box deployment can hold. Self-hosting is not ideology. It is what you do when the transfer mechanism does not exist.

Logging is where the compliance work quietly dies

You can get the provider configuration perfect and then leak everything into your own observability stack.

Trace tools have grown proper controls. Langfuse runs a masking function over trace data before it leaves your application, and since June 2026 mask_otel_spans applies that masking to every span the Python SDK exports, including spans produced by third-party OpenTelemetry instrumentation you never wrote. Self-hosted deployments can also mask server-side at the worker container as a centralised backstop. Arize Phoenix sits in similar territory. Use both layers. Client-side masking fails safe, server-side catches what you forgot.

The dangerous logs are the ones nobody thinks of as logs. The exception handler that serialises the request body. The retry middleware. The gateway access log with the prompt in a query string. The analytics event a data scientist added in a hurry.

And a retention policy is not the same thing as retention reality. On 13 May 2025 a US magistrate judge ordered OpenAI to preserve and segregate output log data that would otherwise have been deleted. That order was lifted on 9 October 2025, with obligations continuing for accounts the plaintiffs had specifically flagged. Litigation in a country you have never sold into can suspend the deletion schedule written into your agreement. Which leaves one reliable control: the sensitive value should not have entered the log in the first place.

The strongest argument against all of this

Here is the case against, and it is stronger than privacy people admit. Enforcement has produced almost nothing.

The Italian Garante fined OpenAI 15 million euros on 20 December 2024 over the legal basis for training, transparency and age assurance. On 18 March 2026 the Court of Rome annulled it, with full reasoning published on 28 May 2026, on the ground that the Garante had no jurisdiction once OpenAI established an Irish entity and the one-stop-shop mechanism applied. The flagship generative AI penalty in Europe currently collects nothing. A rational engineering lead could look at that, price the expected cost of non-compliance near zero and ship.

Three things break that reasoning.

The annulment was procedural. It moved the file to Dublin, it did not approve the processing. The substantive questions are still open and now sit with a regulator handling them for the whole single market.

The bill you actually pay is not a fine. It is the enterprise deal that stalls in security review because you cannot produce a data flow diagram. It is the region you cannot launch in. It is the retrofit six months after general availability. I have seen far more AI features killed by a procurement questionnaire than by any authority.

And erasure debt compounds. Every month you embed unclassified personal data is another month of rows you will not be able to find when someone asks.

The pre-ship checklist

Run this before the feature is live, not after. It takes about a day.

1. List every endpoint the feature calls, including tool-use and file endpoints, and write the current retention window next to each from the provider's own documentation. Not a blog summary. The doc, this week.

2. Confirm which contract you are actually on. Consumer, standard commercial or an executed agreement with zero retention. Check the console and the signed terms, because the marketing page describes a product you may not have bought.

3. If you hold zero retention, enumerate the features that silently step outside it. Batch, files, code execution containers, grounding tools and stateful conversation objects are the usual suspects.

4. Classify what can appear in a prompt, including anything a user can paste. Treat every free-text field as an uncontrolled ingestion point.

5. Put a masking function between your application and both the model call and the trace exporter. Field-level redaction where you know the shape, regex as the backstop for emails, phone numbers and identifier patterns.

6. Tag every vector chunk at write time with subject identifier, source, lawful basis and retention date. This is the single highest-value item on the list.

7. Prove erasure end to end on a real record. Delete it, then go looking in the vector store, the cache, the traces, the evaluation datasets and the backups. Write down what survived, because something will.

8. Never fine-tune on data you may later be asked to erase. If you already have, know which release contains it.

9. Decide residency before you choose a provider, not after. If the bytes cannot cross a border, price the self-hosted path properly instead of hoping a clause covers it.

10. Add a CI rule that flags logging calls touching request or response bodies. Semgrep handles this in an afternoon.

11. Run an adversarial pass for leakage before launch. garak for extraction probes, a guardrail layer for output filtering, and treat a successful extraction as a release blocker.

12. Write the one-page assessment. Purpose, data categories, recipients, retention, transfer mechanism, risks, mitigations, residual risk, who signed and when. GDPR Article 35 requires this for high-risk processing. Do it even when it does not apply, because filling in the retention row is what exposes the endpoint nobody checked.

The part I cannot close out

Everything above puts the erasure guarantee at the retrieval layer, because retrieval is the layer an engineer can actually control. A row disappears. An index rebuilds. You can demonstrate it to an auditor.

That is a compliance answer more than a privacy answer.

If a model memorised somebody's details during pretraining, from a scraped forum post in 2019 that they have spent years trying to bury, deleting your vector row does exactly nothing for them. The person is still in there. Hamburg says the weights hold no personal data. The EDPB says prove it, one model at a time. Nobody has, and the checklist I just gave you has no step twelve-and-a-half for the thing that actually matters to the person on the other end.

Tools referenced

vLLM, reviewed here: vLLM review.

Ollama, reviewed here: Ollama review.

LMDeploy, reviewed here: LMDeploy review.

DeepSeek, reviewed here: DeepSeek review.

Google Gemma 4, reviewed here: Google Gemma 4 review.

Langfuse, reviewed here: Langfuse review.

Sources

Anthropic, API and data retention (ZDR scope, feature eligibility, flagged-content retention): https://platform.claude.com/docs/en/manage-claude/api-and-data-retention

OpenAI, Data controls in the OpenAI platform (per-endpoint retention and ZDR eligibility): https://developers.openai.com/api/docs/guides/your-data

Google, Zero data retention in the Gemini Developer API: https://ai.google.dev/gemini-api/docs/zdr

Google, Gemini API data logging and sharing policy (55-day default retention): https://ai.google.dev/gemini-api/docs/logs-policy

EDPB Opinion 28/2024 on data protection aspects of AI models, adopted 18 December 2024: https://www.edpb.europa.eu/our-work-tools/our-documents/opinion-board-art-64/opinion-282024-certain-data-protection-aspects_en

EDPB, Guidelines on anonymisation and on web scraping for generative AI (July 2026): https://www.edpb.europa.eu/news/edpb-sheds-light-on-anonymisation-and-web-scraping-for-generative-ai-and-adopts-final-version_en

Hamburg Commissioner for Data Protection, Discussion Paper on LLMs and Personal Data, 15 July 2024: https://datenschutz-hamburg.de/fileadmin/user_upload/HmbBfDI/Datenschutz/Informationen/240715_Discussion_Paper_Hamburg_DPA_KI_Models.pdf

Morris et al., Text Embeddings Reveal (Almost) As Much As Text, EMNLP 2023: https://aclanthology.org/2023.emnlp-main.765/