How to keep personal data out of AI prompts

Why this matters

How to keep personal data out of AI prompts starts with one rule: the model should see the task, not the person. A prompt is a copy. When you paste a support ticket or send customer text through an API, the text leaves your app and goes to another system. It may sit in request logs, abuse review queues or conversation history. Even if the vendor does not train on your data, the copy exists in places you do not control.

Personal data arrives in messy forms. Names hide in signatures, emails sit in free text, phone numbers appear in notes, and account IDs turn up in file names. You cannot rely on the user to clean this. You need a control between the user and the model.

Retrieval has the same problem. If the prompt builder pulls a whole CRM record, the model sees every field, including data the task does not need. Prompt hygiene is mostly data minimization, not clever wording.

Before you start

List every path where text reaches a model. Include the chat widget, the internal assistant, API jobs, spreadsheet add-ins, browser extensions and evaluation scripts. For each path, note who can send data and what gets stored.

Define three buckets: allowed, mask and prohibited. Allowed can include business context with no identifiers. Mask covers names, emails, phone numbers and account numbers. Prohibited covers government IDs, payment card data, health details and any combination that identifies a person. Use examples from your own workflows, not generic labels.

Pick where the control lives. The best place is a single prompt gateway that all calls pass through. If that is too much at first, start with the prompt builder for the highest risk path. Also check the vendor console: turn off training on your data where the option exists, set retention to the shortest period that works, and restrict who can view conversations.

How to do it

Build prompts from typed fields. Ask for customer tier and order date instead of pasting the order email. If a free text field is needed, mark it as free text and route it through redaction.

Redact in code before the model call. Use pattern rules for emails, phone numbers, account IDs and names where you have a list. Replace each match with a stable token such as PERSON_1 or EMAIL_1. Keep the token map in your application, not in the prompt. Send the model the tokens and rehydrate the answer only if the user is allowed to see the original data.

For names and addresses, pattern rules are not enough. Use a named entity tool or a curated list of customer names, then review samples. Do not ask the model to redact its own input. A model can miss a name in a signature or invent a replacement that changes the meaning.

Limit retrieval. Filter by tenant, date range and record type before you build context. Return only the fields the task needs. Drop chat history you no longer need. If a document upload is allowed, strip metadata and reject files that contain prohibited data types.

1. Map every prompt path. Walk through chat, API, batch jobs and side tools. Write down where text is entered, what is stored and who can read it.
2. Classify data with examples. Build a short list of allowed, masked and prohibited data using real workflow examples. Give each category an owner.
3. Build prompts from fields. Replace pasted records with a small set of typed inputs. Mark free text fields and send them through the same redaction step.
4. Redact before the call. Put a service in front of the model that finds and replaces identifiers. Keep the token map in your app and log only counts, not values.
5. Use fake data for tests. Create a test set with made up names, emails and account numbers. Run it through the gateway and confirm the outgoing payload is clean.
6. Filter retrieval by need. Add tenant and field filters to the context builder. Return the smallest record that answers the question.
7. Lock down chat and uploads. Disable file upload where it is not needed. Add warnings and client side checks, then enforce the same rules on the server.
8. Review prompts like code. Store prompt templates in version control. Require a second reviewer for changes that touch customer data.

How to check it worked

Inspect the outgoing request payload, not the model reply. A model can produce a clean answer from a dirty prompt. Use a proxy or gateway log that records the final prompt after redaction, with values hashed.

Run a small test set through each path. Include a name in an email signature, a phone number in a paragraph, an account ID in a file name and a record with many fields. Check that the tokens appear and the original values do not.

Review samples by hand each month. Pick prompts from each path and read them. Look for combinations that identify a person even when each field looks harmless. Check that the token map is not stored with the prompt.

Spot check vendor settings after any console change. Confirm retention and training options still match your policy. Test what happens when a user pastes a prohibited item: the request should be blocked with a clear message, not sent and cleaned later.

Mistakes to avoid

Cleaning the chat box and forgetting the API. Most leaks come from background jobs, evaluation scripts and copy paste into a separate tool. Put the control at the gateway so every path uses it.

Asking the model to remove personal data. It may miss an identifier or change the meaning. Do the removal before the call with rules you can test.

Keeping the token map in the prompt. That defeats the purpose. Keep it in your application and expire it when the task ends. Also avoid using production records as few shot examples; use synthetic examples.

Over redacting. If you replace every number, the model cannot do the task. Mask only what identifies a person and keep the business facts. If you need help mapping paths and checking data readiness, Cloudverg's AI readiness assessment at $5,500/month is one option among others.

Frequently asked questions