NeFut Logo NeFut
中 Admin Login

[CS.AI] Redacting Personal Data: Speed and Accuracy Comparison of Regex, Presidio, and Hybrid Designs

Published at: 2026-11-06 22:00 Last updated: 2026-11-07 12:32
#AI #Machine Learning #LLM

Enterprises often feed free‑form text directly to large language models (LLMs), and such text frequently contains personally identifiable information (PII) such as names, phone numbers, email addresses, home addresses and dates of birth. Once a prompt is sent, the PII is copied into provider systems, application logs and observability traces. Redacting PII before the call is an obvious control, yet few teams have measured its latency cost or leakage rate. This paper evaluates four redaction designs that can be plugged into any LLM pipeline:\n\n1. A pure regular‑expression (regex) redactor.\n2. Microsoft Presidio running on a spaCy model.\n3. A hybrid where Presidio finds person names while regex handles structured identifiers (phone numbers, emails, etc.).\n4. A fast hybrid that calls spaCy’s named‑entity component directly.\n\nAll four share a known‑entity rule that removes the person the system already knows it is handling, and a consistent placeholder scheme so the LLM can still distinguish individuals after redaction.\n\nThe designs are benchmarked on a workflow called ProfileAssist, which turns resumes into structured profiles. The test set consists of 100 synthetic resumes (average 523 characters) containing about 900 PII values and 600 business terms that must survive.\n\n- Regex achieved a median latency of 0.20 ms but only 88.9% PII recall, because it cannot discover names not known in advance (e.g., references).\n- Presidio, the hybrid, and the fast hybrid each achieved 100% PII recall and 100% business‑term preservation, with median latencies of 37.59 ms, 30.23 ms and 13.48 ms respectively.\n- Compared with a 9 120 ms median LLM call, even the slowest option adds only 0.41% to the request latency.\n\nWe therefore recommend the fast hybrid as the default for LLM pipelines that handle personal data, keeping regex as a safety net rather than the sole control.\n\nReview: The fast hybrid strikes a practical balance between low overhead and complete redaction, making it suitable for production environments where both performance and compliance are critical.

Original Source: https://arxiv.org/abs/2611.07633

[h] Back to Home