Skip to main content

Usage rules, rollout plan, and final notes

Part of the Universal Audit Log Specification. Open Mercato reference (§19), usage rules for Constellation apps (§20), the rollout plan (§21), testing requirements (§22), known cleanup (§23), spelling consistency notes (§24), and the final implementation recommendation (§25).

19. Open Mercato Reference​

19.1 What Open Mercato Does Well​

Open Mercato implements audit/history as a shared core module with useful patterns: central shared module, reusable history UI, explicit snapshots and computed diffs, separate action/access logs, and related-resource history support.

19.2 What Constellation Should Not Copy​

Constellation SHOULD NOT copy Open Mercato's command-bus-centric audit design. Constellation is tool-layer-centric, uses Prisma in apps / raw SQL in platform, and does not need undo/redo coupling. Borrow structural ideas, not the command bus.

20. Usage Rules for Constellation Apps​

20.1 Mandatory Rules​

  1. Every state-changing tool MUST write an audit entry.
  2. Audit writes MUST happen inside the same transaction as the domain mutation.
  3. Apps MUST use shared platform audit APIs.
  4. Apps MUST NOT create app-local audit tables for domain mutations.
  5. Repositories MUST NOT write audit rows directly.
  6. Route handlers MUST NOT bypass tools for auditable mutations.
  7. Failed and denied operations MUST be audited with outcome: 'FAILURE' or outcome: 'DENIED'.
  8. Background jobs and event handlers MUST use audit capture points defined in Section 6.1.

20.2 Diff Rules​

  • CREATE
    • changes contains { after: ... } or a normalised field map
  • UPDATE
    • changes contains only fields that actually changed
  • DELETE
    • changes contains { before: ... } or a normalised deletion snapshot
  • sensitive fields
    • MUST be redacted via the redaction framework (Section 18) before persistence

20.3 IP Address Rules​

  • Raw IP addresses MUST NOT be stored (see Section 9.4).
  • IP extraction MUST use the trusted proxy chain, not raw X-Forwarded-For.
  • Truncation to /24 (IPv4) or /48 (IPv6) is applied at write time.

20.4 Enforcement​

Audit coverage SHALL be enforced via automated checks that target mutating tools only. Read-only tools (e.g., queryAudit in audit.tools.ts) are explicitly exempt — they do not produce state changes and MUST NOT be forced to include meaningless audit calls.

  1. Typed wrapper convention: Mutating tool functions SHOULD use withAuditedMutation() which makes audit capture structurally unavoidable. Tools that use this wrapper are automatically compliant.
  2. Integration tests (primary enforcement): After running each mutating tool in the test suite, verify that an audit row exists with correct action, resource_type, resource_id, tenant_id, and outcome. This is the authoritative check — it validates behaviour, not file contents.
  3. Optional static check (advisory): A CI lint rule MAY flag mutating tool files that contain no audit call as a warning, but it MUST NOT block merges. The integration test in (2) is the merge gate. The static check uses a naming convention or annotation (e.g., @audit-exempt comment for read-only tools) to avoid false positives.
  4. These checks SHALL be part of the CI pipeline.

21. Rollout Plan​

Phase 1: Foundation​

  1. Extend audit.audit_entries migration with all new columns (organisation_id, parent_resource_type, parent_resource_id, context_json, entry_hash, previous_hash, session_id, user_agent, outcome, duration_ms, changed_fields).
  2. Apply range partitioning on created_at with monthly partitions.
  3. Create all indexes defined in Section 8.
  4. Define and document retention policy (Section 6.5); automate partition creation.
  5. Implement IP truncation at write time (Section 9.4); document retention alignment with architecture spec (Section 6.5).
  6. Implement createAuditor(), withAuditedMutation(), and auditBatch().
  7. Implement buildAuditDiff() with full specification (Section 10.4).
  8. Implement RedactionPolicy and default sensitive patterns (Section 18).
  9. Add keyset cursor support to queryAuditTrail() alongside existing offset/limit (Section 12.2).
  10. Establish operational monitoring baseline (Section 14).
  11. Add integration test enforcement for mutating tools (Section 20.4).
  12. Add tests for new schema fields, query filters, diff generation, and redaction.

Phase 2: Directory as Reference Implementation​

  1. Keep Directory as the reference implementation.
  2. Update audit-producing tools to use createAuditor() and the richer contract.
  3. Expand apps/directory/src/server/tools/audit.tools.ts for parent-resource, organisation, and outcome filters.
  4. Implement granular audit access control (Section 13): audit:read:own, audit:read:org, audit:read:tenant, audit:read:classified.
  5. Implement tamper-evident chain (Section 11) with per-tenant chaining.
  6. Implement audit-of-audit-access logging (Section 13.3).
  7. Validate RLS and permission behaviour for tenant-scoped audit queries.

Phase 3: Catalog Adoption​

  1. Update catalog tools to use normalised resourceType naming.
  2. Add before/after diffs to update and delete flows.
  3. Expose per-entry audit trail in Catalog UI once shared UI is ready.

Phase 4: Project-Tracker Migration​

  1. Remove stub/no-op audit paths.
  2. Replace src/lib/entity-audit.ts as the primary mechanism with explicit tool-layer writes.
  3. Implement /api/audit on top of shared platform queries.
  4. Connect existing AuditTrail UI to real shared audit data.

Phase 5: Shared UI Extraction​

  1. Generalise the audit trail component for reuse across apps.
  2. Introduce standard display adapters for actor/resource labels.
  3. Add related-history support where parent resource fields are present.
  4. Implement streaming export for compliance (Section 15).

Phase 6: Defence-Tier Hardening​

  1. SIEM integration (Section 17).
  2. Chain integrity verification job with alerting (Section 11.5).
  3. Archival automation: detach old partitions, export to cold storage (Section 6.5).
  4. Data residency controls for EU vs US defence tenants.
  5. pg_audit extension for superuser action logging (Section 7.4).

22. Testing Requirements​

22.1 Platform Tests​

  • Schema migration: all columns, partitioning, immutability triggers, JSON validation.
  • Query filters: all fields including organisation, outcome, changed_fields; keyset pagination stability.
  • buildAuditDiff(): property tests, nested flattening, array diffs, 64 KB truncation, redaction, ignoreFields.
  • Pseudonymisation: HMAC generation, actor mapping CRUD, GDPR deletion flow.
  • Ergonomic helpers: createAuditor(), withAuditedMutation(), auditBatch().
  • Redaction: default patterns, custom policies, double-exposure prevention in auditCritical().

22.2 Module Integration Tests​

For each module: every tool creates an audit row matching actor/action/resource/module/outcome; updates persist only changed fields; deletes preserve before-state; failed and denied operations produce appropriate outcome values; tenant isolation holds; organisation-scoped queries return only matching entries.

22.3 UI Tests​

Audit panel loads and paginates (keyset); diffs render correctly; related-resource history toggles; outcome badges display; empty and error states are stable.

22.4 Enforcement Tests​

CI check confirms every tool file contains an audit call. Integration suite verifies audit row existence after each tool execution.

23. Known Cleanup Required​

The following repo state should be corrected during rollout:

  1. apps/project-tracker/src/app/api/audit/route.ts
    • currently returns a stubbed empty result
  2. apps/project-tracker/src/lib/audit.ts
    • currently contains stub auth audit logging
  3. apps/project-tracker/src/lib/entity-audit.ts
    • currently logs debug output instead of canonical audit rows
  4. packages/platform/testing/README.md
    • still references identity.audit_log in an example; canonical table is audit.audit_entries

24. Spelling Consistency​

This specification and all implementation code SHALL use British spelling consistently, matching the existing Constellation codebase:

  • organisation (not organization)
  • normalised (not normalized)
  • behaviour (not behavior)
  • serialisation (not serialization)

Database columns, API fields, and TypeScript types SHALL all follow this convention: organisation_id, organisationId.

25. Final Recommendation​

Constellation should implement universal audit logging as:

  • a shared platform library capability
  • backed by the shared audit schema with range partitioning and defined retention
  • captured explicitly in each module's tool layer and other defined capture points
  • protecting PII surface via IP truncation and context_json prohibition, with actor identity retained per GDPR Article 17(3) exemptions
  • secured by granular access control with audit-of-audit-access
  • queried through shared platform APIs with keyset pagination
  • rendered through reusable audit-history UI components
  • hardened with tamper-evident per-tenant hash chains
  • monitored with operational metrics and SLOs
  • exportable for compliance reporting and SIEM integration

This is aligned with the approved Constellation architecture and uses Open Mercato appropriately as a pattern reference rather than a framework to copy.

Review Acknowledgments​