Skip to content

fix(core): compact repeated message diff events - #48537

Closed
AceRothstein71 wants to merge 4 commits into
anomalyco:devfrom
AceRothstein71:session-diff-events
Closed

fix(core): compact repeated message diff events#48537
AceRothstein71 wants to merge 4 commits into
anomalyco:devfrom
AceRothstein71:session-diff-events

Conversation

@AceRothstein71

Copy link
Copy Markdown

Problem

SessionSummary.summarize attaches the full git patch text of a turn to the user message at summary.diffs (packages/opencode/src/session/summary.ts). Every later sessions.updateMessage on that same user message re-publishes MessageUpdated with the entire summary payload, and because the event log is immutable each publish writes a brand new row containing another full copy of every patch.

The result is unbounded growth: a single turn that touches a large file is re-serialized into the event log on every subsequent update of that message.

Measured impact

On a real opencode.db that had grown to 79 GB, sampling 400 message.updated.1 rows:

rows share average data size
carrying summary 25% 400.8 KB
everything else 75% 0.55 KB

That extrapolates to roughly 49 GB of the 79 GB database being duplicated patch text.

Fix

The patch text only needs to reach the projector once. When a user message is updated and its outgoing summary.diffs are byte-identical to what is already stored, the published event now carries the diff metadata (file, additions, deletions, status) with the heavy patch string omitted.

  • packages/opencode/src/session/session.ts compares the outgoing diffs against the stored row inside SQLite (json_extract(...) = json(?)) and strips patch only on an exact match.
  • packages/core/src/session/projector.ts restores the stored patches when, and only when, the incoming event is exactly that stripped form: same length, every item missing patch, and all metadata equal to the stored item.

Fails safe by construction

Any mismatch at all, including a mere JSON key ordering difference, makes the comparison fail and the full payload is published unchanged. The optimization is lost in that case; the data never is.

Backward compatible

Events already persisted carry real patch strings, so the restore guard never triggers for them and they project exactly as before. No schema change, no migration, no change to the public MessageUpdated payload type.

Cost

The projector performs one extra row read, and only for events that actually arrive in the compacted form.

Verification

packages/core      bun typecheck   exit 0
packages/opencode  bun typecheck   exit 0
packages/opencode  bun test test/server/session-diff-missing-patch.test.ts
                   4 pass, 0 fail, 11 expect() calls

test/server/session-diff-missing-patch.test.ts gains two cases alongside the two existing ones:

  • keeps stored turn diffs while compacting later message update events proves the compaction round-trips.
  • persists changed turn diffs through a fresh event replay proves that when a turn's diffs genuinely change, the new value is published, persisted and survives replay. This pins the failure mode an earlier revision of this change introduced, where diffs were skipped whenever any value was already stored rather than only when it was identical.

In a SQLite fixture reproducing the repeated-update pattern, the repeated event rows drop from 262,515 bytes to 284 bytes while the stored turn diff stays fully available through GET /session/<id>/diff.

Avoid re-appending a summarized user message's full patch array in each message.updated.1 event. Preserve the first durable diff event and projection value for replay compatibility while compacting later updates.\n\nThe focused SQLite regression fixture reduces the repeated event row from 262515 bytes to 284 bytes (99.89%).

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Record the chosen replay-compatible event compaction, failing-before and passing-after output, SQLite byte measurement, verification, and residual risk.

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Compact only patch bodies when SQLite-normalized summary diffs exactly match the projected value. Changed summaries retain their full new patches, and replay restores compact event patches without decoding every projection.\n\nThe real summarize regression covers changed diffs and fresh replay. The repeated fixture event falls from 262515 bytes to 360 bytes (99.86%).

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Keep the requested evidence report as a local untracked deliverable rather than adding it to the upstream branch.

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
@github-actions

Copy link
Copy Markdown
Contributor

Thanks for your contribution!

This PR doesn't have a linked issue. All PRs must reference an existing issue.

Please:

  1. Open an issue describing the bug/feature (if one doesn't exist)
  2. Add Fixes #<number> or Closes #<number> to this PR description

See CONTRIBUTING.md for details.

@github-actions github-actions Bot added the needs:compliance This means the issue will auto-close after 2 hours. label Sep 11, 2026
@github-actions

Copy link
Copy Markdown
Contributor

This PR doesn't fully meet our contributing guidelines and PR template.

What needs to be fixed:

  • PR description is missing required template sections. Please use the PR template.

Please edit this PR description to address the above within 2 hours, or it will be automatically closed.

If you believe this was flagged incorrectly, please let a maintainer know.

@github-actions

Copy link
Copy Markdown
Contributor

The following comment was made by an LLM, it may be inaccurate:

Found a related PR:

#42771 - fix(opencode): evict message diff patches from the event payload

This PR appears to address the same core issue: removing patch text from event payloads to reduce database bloat. It likely represents a previous approach or related work on the same problem of unbounded growth from repeated message diff events.

@AceRothstein71
AceRothstein71 marked this pull request as draft September 11, 2026 19:59
@github-actions

Copy link
Copy Markdown
Contributor

This pull request has been automatically closed because it was not updated to meet our contributing guidelines within the 2-hour window.

Feel free to open a new pull request that follows our guidelines.

@github-actions github-actions Bot removed the needs:compliance This means the issue will auto-close after 2 hours. label Sep 11, 2026
@github-actions github-actions Bot closed this Sep 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants