Scrubbing PII From LLM Logs Before They Persist

The log you built to make your AI auditable is the same log that turns every customer’s personal data into a single, unencrypted, indefinitely-retained target. Redact before write, not after breach.

Every serious LLM deployment ends up logging prompts and completions. You need them to debug hallucinations, to reconstruct what the model was asked when a decision went wrong, to satisfy the record-keeping expectations that now attach to AI systems in regulated use. That instinct is correct. The problem is what those logs contain. Users paste account numbers, medical details, names, addresses and passport numbers into prompts because the interface invites conversation, and the model’s completion frequently repeats the same data back. Left alone, your observability layer quietly becomes the largest concentration of unstructured personal data in the estate — and, because it is a log, it is often the least protected.

The log is the breach you have not had yet

A log store is a soft target by design. It is replicated to a SIEM, shipped to a managed observability vendor, fanned out to cold storage, and read by more engineers than any production database. Access controls that would be unthinkable on the customer table are routine on the logs that describe it. Under the GDPR, none of that changes the legal character of what you are holding: a prompt containing a name and an IBAN is personal data, and writing it to a log is processing. Data minimisation (Article 5(1)(c)) and storage limitation (Article 5(1)(e)) apply to the log exactly as they apply to the database. A breach of the log store triggers the same Article 33 notification clock — 72 hours to the supervisory authority — and the same Article 34 obligation to tell the affected individuals.

This is also a due-diligence liability, not only a security one. An acquirer’s technical review will ask what your log retention actually holds, and “years of unredacted customer conversations” is precisely the kind of finding that shows up in technology due diligence as an indemnity line. The observability you need for AI governance and the sensitive-data lake you must not accumulate are the same pile of files. The job is to keep the first without creating the second.

Redact at the boundary, not at the query

The common mistake is to persist everything raw and rely on masking at read time — redact in the dashboard, restrict who can run the query. That is masking theatre. The plaintext is still on disk, in every replica and every backup, and every one of those copies is discoverable, subpoena-able and stealable. The only redaction that reduces your actual exposure happens before the write, in the logging path itself, so that no sensitive value ever lands on persistent storage in the clear.

Concretely, that means a redaction step sitting between your application and its log handlers, applied to the prompt, the completion and any metadata field that can carry free text. It runs synchronously, in-process, on the record before it reaches disk, the SIEM or the vendor. Detection is the hard part; the masking itself is trivial once you know where the personal data is.

Detection: structured fields are easy, free text is not

Where your prompts have structure — a form field labelled iban, a customer ID passed as an argument — you should never have let raw values into the log in the first place; strip or tokenise them at the point you construct the record. The difficulty is the free-text prompt, where a name, a card number and a home address arrive mid-sentence in whatever the user typed. That is a named-entity recognition problem, and it is worth using a purpose-built detector rather than a wall of regular expressions. Microsoft’s open-source Presidio is the pragmatic default: an analyzer that combines pattern recognisers (credit cards, IBANs, IP addresses, national identifiers) with an NLP model for the entities that only context reveals, such as personal names and locations.

No detector is perfect. Recognisers miss, and they over-flag. Tune the entity list to your jurisdiction and your data, measure the miss rate on real samples, and treat detection as a control you monitor rather than a box you tick. But a good detector applied at the boundary removes the overwhelming majority of the exposure, and that is the difference between a manageable residual risk and a reportable incident.

"""Redact PII from log records before any handler persists them.

Attach this filter to the handlers that write LLM prompts and completions.
Detection runs once, in-process, at the logging boundary, so nothing
sensitive reaches disk, the SIEM or a managed observability vendor.
"""
import logging
from functools import lru_cache

from presidio_analyzer import AnalyzerEngine
from presidio_anonymizer import AnonymizerEngine
from presidio_anonymizer.entities import OperatorConfig

# Entities we treat as personal data in free-text prompts and completions.
TARGET_ENTITIES = [
    "PERSON", "EMAIL_ADDRESS", "PHONE_NUMBER", "CREDIT_CARD",
    "IBAN_CODE", "IP_ADDRESS", "LOCATION", "UK_NHS",
]

# Load secrets from your vault; never hard-code them.
# AES key must be 16, 24 or 32 bytes (AES-256 here). Salt fixes the hash so
# the same value always yields the same pseudonym, which lets you correlate.
_AES_KEY = load_key_from_vault("llm-log-redaction-key")
_HASH_SALT = load_salt_from_vault("llm-log-redaction-salt")


@lru_cache(maxsize=1)
def _engines():
    # Engines load NLP models, so build them once per process.
    return AnalyzerEngine(), AnonymizerEngine()


def scrub(text: str) -> str:
    analyzer, anonymizer = _engines()
    found = analyzer.analyze(text=text, entities=TARGET_ENTITIES, language="en")
    # Encrypt names and account identifiers so they can be recovered under a
    # lawful, logged request; hash everything else for correlation only.
    operators = {
        "PERSON": OperatorConfig("encrypt", {"key": _AES_KEY}),
        "IBAN_CODE": OperatorConfig("encrypt", {"key": _AES_KEY}),
        "DEFAULT": OperatorConfig("hash", {"hash_type": "sha256", "salt": _HASH_SALT}),
    }
    return anonymizer.anonymize(
        text=text, analyzer_results=found, operators=operators
    ).text


class PiiRedactionFilter(logging.Filter):
    def filter(self, record: logging.LogRecord) -> bool:
        # Resolve args into the message first, then clear them, so no raw
        # value survives in record.args to be formatted again downstream.
        record.msg = scrub(record.getMessage())
        record.args = ()
        for field in ("prompt", "completion"):
            value = getattr(record, field, None)
            if isinstance(value, str):
                setattr(record, field, scrub(value))
        return True  # keep the record; it is now safe to persist

Reversible tokenisation, only where it is lawful

Redaction has a cost: you lose the ability to investigate a specific customer’s session later. Sometimes that loss is acceptable and you should simply hash or drop the value. Sometimes an investigation, a fraud case or a regulator’s request genuinely needs the original back. That is what reversible tokenisation is for — encrypt the entity rather than destroy it, so the ciphertext sits in the log and the plaintext can be recovered under a controlled, logged, key-holding process. Presidio’s encrypt operator and its matching decrypt path do exactly this with an AES key.

Reversibility is a governance decision before it is an engineering one. The recovery key must live in a secret store, not in the log’s neighbourhood, with access that is separately authorised and audited — the same key-hygiene discipline that applies when you are hashing and scoping API keys so they are not a liability. Encrypting in the log without ringfencing the key just relocates the plaintext problem. And under the GDPR, encrypted-but-recoverable data is pseudonymisation (Article 4(5)), not anonymisation: it remains personal data, still in scope, still counting towards your minimisation and retention obligations. Reserve reversibility for the entities and the use cases that can actually justify holding recoverable personal data, and hash the rest.

Latency, and the objection you will hear

Running NLP detection on every log line adds milliseconds, and someone will object that this slows the hot path. Two answers. First, LLM inference is already measured in hundreds of milliseconds to seconds; a redaction pass is noise against that budget, and it runs on the log record, not the model response the user is waiting for. Second, if the volume genuinely bites, redact asynchronously in the logging pipeline before persistence rather than in the request thread — but never after the raw record has already been written to durable storage, because at that point the redaction is cosmetic. The point is not to slow anything down. The point is that the plaintext must be masked before it reaches a disk you cannot fully control.

The observability that makes an AI system governable and the data hoard that makes it a target are the same log. Which one you have built depends entirely on what happens in the microseconds before the write.