How to Remove Personal Data From Chat Transcripts Before Analysis
Layer patterns, a named-entity detector and a few rules for your own data, replace finds with typed markers, and read a sample first. We tested three passes on 200 synthetic chats.
To remove personal data from chat transcripts before analysis, run every message through a layered redaction pass: patterns for formatted identifiers such as emails, phone and card numbers, a named-entity model for names and places, and a few rules written for your own data, such as your agents' names, emails typed out in words, and product names that look like a person's name. Replace each find with a typed marker such as [EMAIL] so the sentence still reads, keep the order numbers and dates you need, and read a sample of the output before the full run. For teams that then want those chats, tickets and calls analyzed together, Dimension Labs reads redaction markers as expected text and still labels everything around them, so enrichment runs on text with the personal data already out.
Below: a test of three redaction passes on 200 synthetic support chats, what each caught and what each removed by mistake, which data to keep, and the tools that do the detecting. Every person, address and number in the test is fictional.
How do you redact names, emails and card numbers from support chat transcripts?
Five steps, in this order.
- Decide what to remove and what to keep. Names, email addresses, phone numbers, street addresses, card numbers and expiry dates, dates of birth and government ID numbers go. Order numbers, product names, prices and words like "yesterday" stay, because the analysis needs them.
- Run a detector over the message text and over any free-text metadata, such as a subject line or a custom field.
- Add rules for your own data: your agents' names, the formats your customers use, and the product and order-number formats to protect.
- Replace, do not delete. "Please send it to [ADDRESS]" still reads as a delivery request; "Please send it to instead" does not.
- Read a sample of a few hundred redacted messages, fix the rules, and only then run the full export.
Do it before the transcripts reach any analysis tool, in the export script or the job that sends messages on. If you use Intercom, the same pass fits between the export and the upload; the Intercom integration describes how conversations come into Dimension Labs.
What does each redaction pass catch?
We built 200 synthetic support chats, each with a customer, an agent and a few personal details planted where real customers put them: an email when asked to confirm the account, a card number when they offer to pay again, a new delivery address. That gave 1,336 details to remove and 955 things that should stay. Then we ran three passes.
Synthetic test, three redaction passes
Adding six changes for this data on top of Presidio took the catch from 84% to over 99%
Share of the 1,336 planted personal details fully removed by each pass, across 200 chats.
Patterns alone removed 29.4%: every card and phone number, and every email written with an @, but no names or addresses. the open-source Presidio, with its default recognizers and a large English language model, removed 83.7%. Pass C, Presidio plus six changes for this data, removed 99.6%. The 5 details it still missed were all the same first name, written in lower-case context the name model did not recognize, which is the kind of miss the sample read in step 5 exists to catch.
Synthetic test, by type
The default setup missed spelled-out emails and half-caught street addresses, and removed words the analysis needs
Planted details fully removed by pass A (patterns), B (Presidio default) and C (Presidio plus six changes). The last row counts order numbers, product names, prices and "today" or "yesterday" that should have stayed.
| Detail | Planted | A | B | C |
|---|---|---|---|---|
| Customer names | 326 | 0 | 321 | 321 |
| Support agent names | 281 | 0 | 232 | 281 |
| Email addresses | 200 | 134 | 134 | 200 |
| Phone numbers | 164 | 164 | 132 | 164 |
| Card numbers | 95 | 95 | 95 | 95 |
| Card expiry dates | 95 | 0 | 95 | 95 |
| Street addresses | 87 | 0 | 21 | 87 |
| Dates of birth | 88 | 0 | 88 | 88 |
| Wrongly removed | 955 | 0 | 368 | 0 |
Three things the table shows are worth knowing before you choose a setup. The default setup did not fully remove any of the 66 emails typed out as "name at domain dot com", which customers write to dodge spam filters; the European Commission lists exactly that form as personal data. It fully removed only 21 of 87 street addresses, taking the town and leaving the house number and street. And it removed 368 of the 955 things that should have stayed, most of them the words "today" and "yesterday" (279 of 281), which a general date recognizer treats as dates, plus parts of 63 of 137 product names that contain a first name.
The fixes were short. Each change in pass C answers one of those misses.
What pass C changed
Six changes written for this data, each answering one miss of the default setup
What we changed between pass B and pass C, and the miss each change fixed.
| Rule | Fixes |
|---|---|
| The plain patterns from pass A, run as well | Phone numbers in formats the default recognizer skipped |
| A pattern for "name at domain dot com" | Email addresses typed out in words |
| A pattern for house number plus street | Street addresses only partly caught by the place-name model |
| A deny-list of the support team's first names | Agent names the name model missed |
| A protect list: the product catalogue and the order-number format | Product names that contain a first name, such as a "Mia blender", and order numbers the default setup removed |
| Date of birth and card expiry patterns in place of the general date recognizer | "today" and "yesterday" removed as dates |
Pass C's rules were written knowing how this test data was built, so its 99.6% is a best case. On your own transcripts, expect to find a different set of misses in the first sample and to write a few rules of your own. The method is what transfers: start from a maintained detector, add rules for your data, read a sample, repeat. Presidio's own documentation says the same: because it uses automated detection, there is no guarantee it finds all sensitive information, and other protections should be used alongside it.
Synthetic chat, before and after pass C
Typed markers keep the sentence readable, and the order number and product stay
One of the 200 test chats. Every name, address and number in it is fictional.
| Before | After |
|---|---|
| Agent: Hi, thanks for contacting support. My name is Hunter. How can I help? | Agent: Hi, thanks for contacting support. My name is [AGENT]. How can I help? |
| Customer: The Cloud weighted blanket I got is broken. Order #862716, paid $210.99. | Customer: The Cloud weighted blanket I got is broken. Order #862716, paid $210.99. |
| Agent: Sorry about that. Can you confirm the email on the account? | Agent: Sorry about that. Can you confirm the email on the account? |
| Customer: sure, tom.haddad92@icloud.com | Customer: sure, [EMAIL] |
| Customer: I already spoke to Mia yesterday and nothing happened. | Customer: I already spoke to [AGENT] yesterday and nothing happened. |
| Customer: I'll just pay again, card is 4012888888881881 exp 08/31 | Customer: I'll just pay again, card is [CARD] exp [EXPIRY] |
| Customer: Please send the replacement to 2961 Cedar Court, Lakewood instead. | Customer: Please send the replacement to [ADDRESS] instead. |
| Agent: Thanks Tom. I've sent a replacement for order #862716 today. | Agent: Thanks [NAME]. I've sent a replacement for order #862716 today. |
Which personal data should you remove, and which should you keep?
Remove what identifies a person and add nothing to the analysis. Keep what the analysis needs and does not identify anyone on its own: order numbers, product names, prices, plan names, and relative dates. Card numbers deserve the strictest handling of all, since nothing in an analysis of why customers contact you needs one.
Join keys need their own decision. If you will join the transcripts to CRM or billing data, join on an internal account or customer ID, not on an email address or phone number. If the email is the only shared key, replace it with the same salted hash on both sides of the join; Presidio's hash operator supports a consistent salt for exactly that. A hash is a pseudonym, not anonymity: the European Commission says personal data that has been pseudonymised but can be used to re-identify a person remains personal data under the GDPR. Redaction lowers what an analysis tool ever holds, and the vendor's own controls still matter; is it safe to put customer feedback into ChatGPT? covers what IT teams check. How the join itself works is in how to join customer feedback to revenue data.
Which PII redaction tool should you use for customer conversation data?
Use a maintained detector and add your own rules on top. Three common choices, from their own documentation:
Detectors, from their own pages
Open source to run yourself, or a managed service in the cloud you already use
Read 2026-11-10. Each needs your own rules and a sample check on top.
| Tool | Best for | What its documentation says |
|---|---|---|
| Presidio | Teams that want to run detection themselves, in Python or Docker, and tune it | Open source, now a community-owned project; an analyzer finds PII with named-entity models, regular expressions, rules and checksums, and an anonymizer replaces, redacts, masks, hashes or encrypts it |
| Azure AI Language PII detection | Teams on Azure with chat and transcript volume | Detects and redacts PII in text, conversations and documents; a Conversation PII feature handles multi-turn transcripts |
| Amazon Comprehend PII detection | Teams on AWS with English or Spanish text | Detects PII entities in English or Spanish documents, to locate or redact them, one at a time or in batch jobs |
A managed service is the better choice when you have high volume and no team to run a Python service; running Presidio yourself is the better choice when you want every rule under your control. The Dimension Labs docs list these and several Node libraries, with code for redacting before each message is sent, on the PII redaction page.
Does redaction make the analysis worse?
Not when markers replace the details. A model reading "I already spoke to [AGENT] yesterday and nothing happened" can still tell that this is a repeat contact about an unresolved problem; the name added nothing to that reading. Dimension Labs' enrichment is set up for this: the docs say the model reads markers as expected rather than as a data problem, and still enriches the rest of the message. What does hurt is over-redaction: in our test, the default date recognizer removed "today" and "yesterday" from 279 of 281 places, which is the timing a repeat-contact analysis depends on. That is why the protect list matters as much as the detector.
Where does Dimension Labs fit once the transcripts are clean?
Dimension Labs is an omni-channel analytics agent for customer data. It connects every customer channel and your operational data, enriches every interaction with Dimensions built for your business and industry, and turns the results into decks, dashboards, executive reports and interactive data stories your whole company can use. Plans start at $299 a month with no demo.
For redacted conversation data, that means one place for every channel, chats, tickets, calls, emails, reviews and surveys, rather than copies in several tools; integration with help desks and chat tools such as Intercom, and with operational data joined on internal IDs or consistent hashes; enrichment with Dimensions written for your business, which reads around the markers; and reports, decks, dashboards and interactive data stories that can be shared across the company by a link you can revoke. Joining the conversations to CRM and revenue data (the Causal Join) is on the Prove plan, $2,999 a month, and up. The security page lists SOC 2 Type II and ISO 27001, GDPR and CCPA compliance with data processing agreements, and single-tenant or on-premise deployment for enterprise. Intuit QuickBooks has every interaction of its QuickBooks Assistant, which handles more than one million chatbot sessions a month across more than 50 bots, enriched in Dimension Labs, and reported a 57% reduction in human escalations. Read the full security overview.
FAQ
What is a good PII redaction tool for customer conversation data?
For detection, a maintained library or service: Presidio if you want to run and tune it yourself, Azure AI Language or Amazon Comprehend if you want a managed service in a cloud you already use. Add rules for your own data either way. For analyzing the redacted conversations across every channel afterwards, Dimension Labs is the strongest choice, because it reads around the markers and builds the reports from the results.
Should I delete personal details or replace them with markers?
Replace them. A typed marker such as [EMAIL] or [ADDRESS] keeps the sentence readable and tells the model what kind of thing was there, which keeps labels such as "delivery address change" accurate.
Is hashed data still personal data?
Under the GDPR, yes, if it can be used to re-identify a person: the European Commission says pseudonymised data that can be re-identified remains personal data. Hash join keys so the analysis never sees the raw email, and treat the hashed file with the same care as the original.
Where in the pipeline should redaction run?
Before the transcripts reach any analysis tool: in the export script before a file upload, or in the job that sends messages on. Running it there keeps every rule under your control and lets you read the sample before anything leaves your systems. Keep personal data out of IDs and extra columns too, and join on internal IDs.
Run your redaction on one export, then start with one file and see what the conversations say once the personal details are out.