User data and AI providers: what you are actually sending

When a product gets its first model-backed feature, the data question usually comes down to “will the provider train on our data?” It’s a fair question, but it’s one of several. Every model request is a transfer of user data to a third party, and it deserves the same attention as wiring up a payment processor or an email service.

Look at what actually goes out

Don’t start with the provider’s policy. Start with your own requests. Turn on logging for a day and look at what ends up in the prompt. There’s usually more than anyone expected:

• the entire conversation history, when only the last message matters;
• the full user profile “for context,” email, phone and address included;
• a whole uploaded document, when the relevant part is one paragraph;
• internal IDs, tokens and system fields that rode along with an object from the database.

The rule is simple: send the model only what the answer needs. That’s about privacy and about quality. Extra context distracts a model the same way it distracts a person.

Check the provider’s retention settings

Major providers usually have different terms for API access than for their consumer chat apps, often better ones. But “usually” isn’t an answer. Read your provider’s current terms and find answers to four questions:

• is API data used to train models, and is that off by default;
• how long are requests and responses kept, and why (often for abuse monitoring);
• can retention be shortened or turned off, and what does that take;
• in which region is data processed, and can you choose.

Separately check everything built on top of the model: saved conversations, files uploaded for search, caches, tracing logs. These keep data longer than the request itself and are configured separately. Write the answers down and revisit them when you switch providers or turn on a new feature.

Strip it before sending

Plenty of tasks need no personal data at all. To classify a support request, the model doesn’t need the customer’s name or the card they paid with.

A practical approach is to swap sensitive values for placeholders before sending and restore them afterwards:

• names, emails, phone numbers → [NAME_1], [EMAIL_1];
• card, document and account numbers: remove entirely, the model never needs them;
• addresses: keep the city if it matters for the answer, drop the rest.

Automated detection isn’t perfect. Regexes miss unusual formats, and a model-based filter needs the data sent to it in the first place. So the best protection is not putting extra data in the prompt to begin with. Masking is the second layer.

Consent and transparency

Users should understand that their text or file is processed by an outside service. Most personal data laws require it, and it’s a matter of trust anyway.

• State in your privacy policy which features use external models, what data goes there and to whom.
• For sensitive features, such as medical documents, finances or private correspondence, ask for explicit consent at the moment of use, not just a checkbox at signup.
• Give people a way to skip the AI feature and still use the product.
• If you have business customers, be ready to show them the list of subprocessors handling their data. The model provider belongs on it.

Your own storage counts too

The prompt and response logs you keep for debugging and evals contain the same user data. They need a retention period, restricted access and deletion along with the account. Otherwise the provider keeps requests for a month while you keep them forever in an open spreadsheet.

The upshot

Data in AI features is handled like any other personal data: know what you send, send the minimum, understand who keeps it and for how long, and tell users honestly. None of it is hard. The hard part is remembering to do it before a customer or a regulator asks.

SUBXCAT can audit what data your product sends to models and set up minimization and retention.