Skip to content

feat(observability): extend audit log, PostHog, and storage metering coverage#5269

Open
waleedlatif1 wants to merge 13 commits into
stagingfrom
worktree-audit-posthog-coverage
Open

feat(observability): extend audit log, PostHog, and storage metering coverage#5269
waleedlatif1 wants to merge 13 commits into
stagingfrom
worktree-audit-posthog-coverage

Conversation

@waleedlatif1

Copy link
Copy Markdown
Collaborator

Summary

Extends the three observability layers — audit log, PostHog product analytics, and usage metering — to cover resources and actions that were previously uninstrumented. Found via a codebase-wide coverage audit; every gap below was a real blind spot.

Audit log

  • Exfiltration trail (none existed): file downloads (workspace, public-share, v1 API), and table / workflow / workspace exports.
  • Credential access audited at the token-issuance boundary (CREDENTIAL_ACCESSED), success-only.
  • Full revenue/financial trail: invoice paid/failed, overage billed, charge disputes, credit fulfillment, subscription create/cancel/transfer, enterprise provisioning, plan/seat changes.
  • Session/account lifecycle via Better Auth hooks: login, blocked sign-in, logout, session revoke, account delete.
  • The entire v1/admin programmatic surface (member/role/export ops) and copilot tool handlers (skills, custom tools, tables) — both previously bypassed audit entirely.
  • Wired several defined-but-dead AuditAction constants (lock/unlock, table update/delete, etc.).

PostHog

  • Revenue events (payments, overage, credits, disputes, plan conversion, enterprise), org/seat lifecycle, KB search, copilot/skill/tool events.
  • Client-side workspace + organization group association so events roll up by account.

Metering

  • Storage now counted for KB documents and copilot files (was workspace-files only). Connector-synced KB docs remain correctly unmetered on both ingest and delete.

Hardening

  • Audit package nulls the actor FK for system actors (admin-api) with a readable label instead of silently FK-failing; adds an awaitable recordAuditNow for pre-delete hooks.
  • Every billing-webhook actor lookup is guarded so instrumentation can never fail payment processing.
  • All instrumentation is fire-and-forget and never blocks or breaks the primary operation. Ephemeral execution-file storage is intentionally not metered (no decrement path → would leak).

Type of Change

  • New feature (observability coverage)

Testing

  • Full typecheck, biome, and check:api-validation pass.
  • 2,617 touched-area tests + 22 audit-package tests pass.
  • Validated by a multi-agent adversarial review across every changed domain (FK safety, fire-and-forget, no double-emit, storage symmetry, behavior-preserving refactors).

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

…coverage

Instruments previously-uncaptured resources and actions across the three
observability layers (audit log, PostHog product analytics, usage metering):

- Exfiltration audit: file downloads (workspace/public-share/v1), table,
  workflow, and workspace exports.
- Credential access audited at the token-issuance boundary (success-only).
- Full revenue trail: invoice paid/failed, overage billed, disputes, credit
  fulfillment, subscription lifecycle, plan/seat changes.
- Session/login lifecycle audit via Better Auth hooks (login, blocked sign-in,
  logout, session revoke, account delete).
- v1/admin programmatic surface and copilot tool handlers instrumented.
- Dead audit/PostHog constants wired (lock/unlock, table, custom tool, etc.).
- Storage metering extended to KB documents and copilot files.
- PostHog person identify + workspace/organization group hygiene.

Audit package hardened to null the actor FK for system actors (admin-api) and
adds an awaitable recordAuditNow for pre-delete hooks. All instrumentation is
fire-and-forget and never blocks or breaks the primary operation.
@vercel

vercel Bot commented Jun 29, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
docs Skipped Skipped Jun 29, 2026 10:29pm

Request Review

@cursor

cursor Bot commented Jun 29, 2026

Copy link
Copy Markdown

PR Summary

Medium Risk
Touches authentication hooks, credential token routes, billing webhooks, and storage quota accounting; instrumentation is fire-and-forget but incorrect metering or actor resolution could skew billing or audit trails without breaking primary flows.

Overview
Closes observability gaps by wiring recordAudit and captureServerEvent across sensitive paths that previously had no trail: file egress (workspace, bulk zip, public share, v1 API) only on successful responses; OAuth/service-account token issuance as CREDENTIAL_ACCESSED; table/workflow/workspace exports (including async export jobs with userId on the payload); org/workspace membership and billing (plan switch, subscription transfer, seat reconcile with an optional actorId, threshold overage settlement); and the full v1/admin surface.

Better Auth gains before/after hooks for blocked sign-in/sign-up, successful logins (gated to real sign-in paths), sign-out, session revocation, and an awaited recordAuditNow before account deletion. Realtime debounced workflow variable flushes now attribute the coalesced writer and emit WORKFLOW_VARIABLES_UPDATED after persist.

PostHog adds typed revenue/org/credential/copilot events and enriches client workspace/organization group properties in WorkspaceScopeSync.

Storage metering now checks quota and increments on KB document create (post-commit), decrements in the same transaction as hard deletes via decrementStorageUsageInTx, and adds releaseDeletedFileStorage for atomic soft-delete + counter decrement (used on copilot file delete). Table rename/delete/archive and column adds accept an acting user for audit attribution.

Reviewed by Cursor Bugbot for commit f0a18ef. Configure here.

Comment thread apps/sim/app/api/auth/oauth/token/route.ts
Comment thread apps/sim/app/api/files/export/[id]/route.ts Outdated
Comment thread apps/sim/app/api/files/export/[id]/route.ts Outdated
@greptile-apps

greptile-apps Bot commented Jun 29, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR extends three observability layers — audit log, PostHog analytics, and storage metering — to cover previously uninstrumented surfaces including file downloads, credential access, billing events, and session lifecycle. It also hardens the audit package to null actor FKs for system/unknown actors instead of FK-violating silently, and introduces recordAuditNow for pre-delete hooks where a deferred insert would race a cascade delete.

  • Audit log: Adds CREDENTIAL_ACCESSED, FILE_DOWNLOADED, USER_LOGIN/LOGOUT, SESSION_REVOKED, ACCOUNT_DELETED, all Stripe billing events (INVOICE_*, SUBSCRIPTION_*, CHARGE_DISPUTE_*), and previously dead constants (TABLE_EXPORTED, WORKFLOW_EXPORTED, etc.); fixes actor FK nulling for system actors in both the not-found and catch branches of insertAuditLog.
  • Storage metering: KB documents and copilot files are now metered at ingest (incrementStorageUsage via meterCopilotIngest) and decremented atomically with row deletion (decrementStorageUsageInTx inside the delete transaction, releaseDeletedFileStorage for copilot soft-deletes).
  • PostHog: Adds 18 new typed event definitions and wires them across billing webhooks, copilot tools, and the credential token API; adds client-side organization group association alongside the existing workspace group.

Confidence Score: 5/5

Safe to merge — all instrumentation is fire-and-forget and the primary operations are unchanged; the storage metering path is atomic and idempotent.

Every audit write is non-blocking (fire-and-forget recordAudit, or a best-effort try/catch wrapper), so a failure in any of the new observability paths cannot break payment processing, file operations, or auth flows. The storage metering changes are the most structurally significant: releaseDeletedFileStorage uses an idempotency claim row to prevent double-decrements, and decrementStorageUsageInTx keeps the counter and the row deletion in the same transaction. The actor FK nulling fix for system actors is well-tested with new regression tests. The previously-flagged issues (GET audit ordering, actor misattribution on public shares, copilot delete ordering, catch-branch FK violation) are all resolved in this revision.

No files require special attention — all changed files are observability additions over existing, well-tested business logic.

Important Files Changed

Filename Overview
packages/audit/src/log.ts Correctly nulls actorId FK in both the not-found and catch branches; adds awaitable recordAuditNow for pre-delete use cases; introduces null actorId support for anonymous public-share events.
apps/sim/lib/billing/storage/tracking.ts Adds decrementStorageUsageInTx (in-transaction decrement) and releaseDeletedFileStorage (atomic soft-delete + decrement); idempotency guard via claiming the row before decrementing prevents double-decrement on retry.
apps/sim/lib/uploads/contexts/copilot/copilot-file-manager.ts deleteCopilotFile correctly settles storage accounting before blob deletion — if releaseDeletedFileStorage fails the blob is preserved for retry; quota check added to uploadCopilotFile; metering centralized in insertFileMetadata.
apps/sim/lib/knowledge/documents/service.ts KB document ingest and hard-delete now meter storage; hard-delete uses decrementStorageUsageInTx inside the deletion transaction so counter and rows stay in sync; connector-synced docs correctly excluded from metering.
apps/sim/lib/auth/auth.ts Auth lifecycle audited via before/after hooks: logout captured before session deletion (avoids race), login gated on isLoginPath to prevent refresh paths from double-counting, recordAuditNow used in beforeDelete to guarantee write before FK cascade.
apps/sim/app/api/auth/oauth/token/route.ts CREDENTIAL_ACCESSED audit correctly placed after refreshTokenIfNeeded in all three handlers (GET and both POST branches); fire-and-forget recordAudit never blocks token delivery.
apps/sim/lib/billing/webhooks/disputes.ts handleDisputeClosed refactored to audit all dispute closures regardless of outcome; shouldUnblock logic preserved; early return moved after customer lookup so actor can be resolved for the audit trail.
apps/sim/lib/billing/webhooks/invoices.ts resolveBillingActorId helper correctly falls back to referenceId on failure; INVOICE_PAYMENT_SUCCEEDED and INVOICE_PAYMENT_FAILED audits added after processing to avoid spurious records on failure paths.
apps/sim/lib/uploads/server/metadata.ts meterCopilotIngest centralizes copilot storage metering at row-creation time, covering fresh insert, soft-delete restore, and all ingest paths symmetrically with the delete decrement.
packages/audit/src/types.ts Comprehensive new AuditAction constants and AuditResourceType values added for auth, billing, subscriptions, files, tables, and workflows.
apps/sim/lib/billing/threshold-billing.ts Both personal and org overage billing transactions refactored to return a result object, enabling OVERAGE_BILLED audit and PostHog event to fire only when the transaction actually committed a charge.
apps/sim/lib/posthog/events.ts 18 new typed PostHog event definitions added; existing api_key and environment events made workspace_id-optional to accommodate personal-scope variants.

Reviews (12): Last reviewed commit: "fix(audit): null actor FK when the user ..." | Re-trigger Greptile

Comment thread apps/sim/app/api/files/public/[token]/content/route.ts
Address Cursor Bugbot review:
- GET OAuth token: emit CREDENTIAL_ACCESSED/credential_used after
  refreshTokenIfNeeded succeeds (matches POST), not before.
- File export: audit on each actual success exit via a shared helper —
  including the non-markdown serve redirect (previously unaudited) and
  after the zip is generated (previously before asset fetch).
Address Greptile P1: setting actorId to the file owner made every anonymous
external download read as a self-download, undermining the exfiltration trail.
recordAudit now accepts a null actor; the public content/inline routes record
actorId=null with the owner in metadata.sharedByUserId and rely on ip/user-agent
for the forensic trail. The misleading owner-attributed file_downloaded PostHog
event is dropped on these anonymous paths.
@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cursor review

Comment thread apps/sim/app/api/v1/admin/workflows/export/route.ts Outdated
Comment thread apps/sim/lib/knowledge/documents/service.ts
- Admin workflow/workspace ZIP exports now audit only after the archive is
  built (via a local helper), so a zip/build failure no longer logs an export.
- Remove redundant 'success-only placement' inline comments and tighten the
  remaining design-rationale notes (export helper now TSDoc).
@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cursor review

Comment thread apps/sim/lib/billing/webhooks/disputes.ts
Comment thread apps/sim/lib/billing/threshold-billing.ts
Comment thread apps/sim/lib/uploads/contexts/copilot/copilot-file-manager.ts
Address Cursor review:
- handleDisputeClosed now records CHARGE_DISPUTE_CLOSED for every closed
  dispute (won/lost/warning_closed), unblocking only on favorable outcomes.
  dispute.status in the metadata distinguishes the outcome, so lost
  chargebacks are no longer missing from the trail.
- Threshold overage now emits OVERAGE_BILLED + overage_billed even when credits
  fully cover the overage (settledVia: 'credits' vs 'stripe'), so credit-settled
  overages are audited instead of silently returning null.
Address Greptile P1: deleting the metadata row before the decrement meant a
decrement failure left the quota permanently inflated with no record to retry
from. Decrement first; only remove the metadata row once it succeeds.
@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cursor review

Comment thread apps/sim/app/api/table/[tableId]/export/route.ts
Comment thread apps/sim/lib/auth/auth.ts
Comment thread apps/sim/lib/uploads/contexts/copilot/copilot-file-manager.ts
…tion

- Copilot file delete now releases storage via a single transaction
  (releaseDeletedFileStorage): the soft-delete is the idempotency claim and the
  decrement shares the transaction, so neither a partial failure (inflated
  counter) nor a retry (double-decrement) can desync the quota. Resolves the
  conflicting Greptile/Cursor ordering findings.
- Policy-blocked sign-ups now record USER_SIGNUP_BLOCKED instead of
  USER_SIGNIN_BLOCKED, so account-lifecycle events aren't mislabeled.
@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cursor review

Comment thread apps/sim/app/api/v1/admin/workflows/export/route.ts
Address Cursor review: when every requested workflow fails to load, the admin
export still returned 200 but recorded a successful export of zero workflows.
Guard auditExport so an empty result records nothing.
@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cursor review

Comment thread apps/sim/lib/uploads/contexts/copilot/copilot-file-manager.ts
Address Cursor review: the blob was removed before releaseDeletedFileStorage, so
a release failure left the counter inflated and the metadata active with the blob
gone. Now the atomic soft-delete + decrement runs first and the blob is deleted
only if it succeeds, so a failure leaves the file fully intact and retryable.
@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cursor review

Comment thread apps/sim/lib/uploads/contexts/copilot/copilot-file-manager.ts
@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cursor review

Comment thread apps/sim/lib/knowledge/documents/service.ts Outdated
Address Cursor review: hardDeleteDocuments deleted the rows then decremented
best-effort, so a decrement failure left billed storage inflated with no row to
reconcile. Resolve each owner's subscription up front, then decrement inside the
same transaction that deletes the embeddings/documents (decrementStorageUsageInTx,
also now shared by releaseDeletedFileStorage), so the counter and the content
commit or roll back together. Connector docs remain excluded.
@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cursor review

Comment thread apps/sim/lib/auth/auth.ts
Address Cursor review: /verify-email could emit USER_LOGIN when verification
mints or refreshes a session, mislabeling (or double-counting) a non-sign-in as
a login. Restrict isLoginPath to genuine sign-in entrypoints.
@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cursor review

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit 59ee511. Configure here.

Comment thread packages/audit/src/log.ts
Address Greptile P1: the catch branch left the original actorId, so a system
actor like 'admin-api' (or a since-deleted user) would FK-violate the insert and
lose the row when the existence lookup errored. Mirror the not-found branch —
null the FK with a readable label — so the audit row always persists.
@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cursor review

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit f0a18ef. Configure here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant