$ cat ~/field-notes/audit-logs-that-answer-questions.md
Audit Logs That Can Actually Answer Who Changed What
A customer asks who removed an administrator from their workspace. Your application logs show requests, durations, and a successful database update. None identifies the affected membership or explains whose authority performed the change.
Debugging logs can tell you that code ran. An audit trail needs to explain an action in the application's own terms.
Start with the Investigation Question
For a membership change, a useful record could look like this:
{
"event_id": "audit_8c91",
"occurred_at": "2026-11-04T16:20:00Z",
"tenant_id": "workspace_42",
"actor_type": "user",
"actor_id": "user_17",
"action": "membership.role_changed",
"target_type": "membership",
"target_id": "membership_98",
"outcome": "succeeded",
"operation_id": "operation_712",
"changes": { "role": { "from": "admin", "to": "member" } }
}
This schema is an application example. Build it from the questions your team needs to answer, rather than serializing every request object.
OWASP's logging guidance emphasizes recording the event, actor, context, and result while excluding secrets and handling untrusted values safely. Those principles apply even when the final record is JSON rather than a text line.
Derive Identity from Trusted Context
The actor and tenant must come from authenticated server state and the authorization decision. A client-submitted actor_id is a claim, not evidence.
Represent service accounts and background jobs explicitly. If a support operator acts on behalf of a customer, preserve both the real operator and effective subject rather than making the action look like an ordinary customer request.
Record stable IDs even if you also store display names. Names change. Decide what should remain discoverable after an account is renamed or deleted, with the same retention and privacy policy used for the rest of the record.
Couple Success to the Change
For a local database action, commit the business update and successful audit record together:
transaction:
authorize the membership change against current state
update the membership
insert audit event with the actual before and after roles
commit
If the audit insert fails and this action requires a complete trail, the change must fail too. Otherwise a full log disk or unavailable logging service can quietly create unaudited changes.
If records must reach another system, commit an outbox event locally and export asynchronously. Give it a stable event ID so retries do not create multiple apparent actions.
Denied attempts and rolled-back changes need their own recording path: an audit insert inside a transaction that rolls back will disappear. Separate "attempted," "denied," and "succeeded" so an investigator cannot mistake intent for a committed result.
Choose Fields Instead of Dumping Objects
An account update may contain a role, a password-reset token, and an email address. Recording the whole payload creates an unnecessary store of sensitive material.
Use a per-action allowlist. For a role change, record roles and membership identity. For a credential rotation, record the credential's identifier and outcome, never the old or new secret. Redact at the point of event construction so downstream exports do not have to guess.
Structured serialization also prevents a submitted newline from masquerading as a separate event. Bound field lengths and treat labels as untrusted when showing them in an investigation UI.
Protect History and Access
Restrict ordinary application access to the operations it needs. An append-oriented application interface is useful, but a database administrator may still be able to change history. Be precise about the threat model before calling a system immutable.
If independent tamper resistance is required, consider a separately controlled destination and retention enforcement. A chain of hashes stored beside editable records cannot by itself stop an attacker from rewriting both the records and chain.
Scope investigation queries to the authorized tenant, record access to sensitive exports, and define retention before the table grows indefinitely.
Test the Questions
Seed a role change, a denied attempt, a retry, and an action by a service account. Then ask:
- Who changed this membership, and under which authority?
- Did the change commit, or was it merely attempted?
- What changed, and which operation connects the surrounding events?
- Can another tenant discover this record?
If those answers require reconstructing application code or searching raw HTTP bodies, improve the event design. An audit trail earns its storage by making specific historical questions answerable.