Logs without personal data

Logs tend to grow on their own. Someone adds console.log(req.body) to catch a bug, the bug gets fixed, the line stays. Six months later the logs hold passwords from the login form, phone numbers from profiles and tokens from headers, and access to them belongs to every developer, the monitoring service, and the contractor who was given a look at one error.

Logs are a database of personal data just like your main one, only less protected and with no deletion policy. It’s easier not to put things there than to work out later where they all went.

What to log

A good log line answers “what happened, where, and how did it end”, not “who is this person”:

• the request ID;
• the route as a template: /orders/:id, not /orders/8412?token=...;
• method, status code and duration;
• the internal user ID, an opaque one from the database, not an email;
• for errors, the type, message and stack trace;
• key product events: order created, payment succeeded, subscription cancelled.

That’s enough to investigate almost any problem. If you need to know who the user is, look it up in the main database, which has access controls and an audit trail.

What never to log

• passwords, verification codes and one-time links;
• tokens, API keys, Authorization and Cookie headers;
• card numbers and payment details;
• full request and response bodies;
• AI prompts, if user text ends up in them.

If you can’t avoid emails and phone numbers, mask them: a***@example.com, the last two digits of a number. An IP address is personal data in many jurisdictions too. If you need it to fight abuse, keep it briefly or truncated.

Allowed fields, not forbidden ones

A list of things you mustn’t log always lags behind: tomorrow someone adds a passport_number field and nobody remembers to add it. The more reliable approach is the reverse: log objects only through an explicit set of allowed fields. Redaction in the logger stays as a safety net for whatever slips through:

const logger = pino({
  redact: {
    paths: ["req.headers.authorization", "req.headers.cookie", "*.password", "*.token"],
    censor: "[redacted]",
  },
});

Check your error tracker separately. Many collect request bodies, cookies and local variables by default. They have scrubbing settings, and those are worth turning on from day one.

Secrets in URLs

A password reset link with a token in the query string ends up in the logs of your proxy, load balancer and CDN, including ones you don’t control. If a token in the URL is unavoidable, make it single-use and short-lived, and log URLs without their query strings.

The request ID

Every incoming request gets a unique ID at the very first hop: at the proxy or in the first handler. From there it’s passed to every internal call, carried into any background jobs the request enqueues, and written into every log line.

That gives you two things. First, one ID shows the full history of a request across every service. Second, you can return it in a response header and show it on the error page. Someone writes to support saying “I got an error, the code is such-and-such”, and you find it in seconds without asking for their email or searching by timestamp.

How long to keep it

Detailed logs are for investigating recent problems, and a couple of weeks is usually enough. Aggregated metrics, such as request counts and error counts, can be kept as long as you like, since there are no people in them. An audit log, recording who changed what, is a separate thing with its own rules and a longer retention, and shouldn’t be mixed with debug logs.

Short retention solves one more problem: when a user asks for their data to be deleted, the logs clear themselves without a separate operation.

Who reads them

Access to logs is access to data, just like access to the database. Grant it as needed, review it, and give outside contractors a specific excerpt rather than the whole stream.

SUBXCAT can review what your project currently writes to its logs and set up redaction and request IDs.