Skip to content

fix(ingest): persist document ids so a mid-question failure doesn't lose them - #88

Open
Agnik47 wants to merge 1 commit into
supermemoryai:mainfrom
Agnik47:fix/ingest-persist-document-ids
Open

fix(ingest): persist document ids so a mid-question failure doesn't lose them#88
Agnik47 wants to merge 1 commit into
supermemoryai:mainfrom
Agnik47:fix/ingest-persist-document-ids

Conversation

@Agnik47

@Agnik47 Agnik47 commented Aug 20, 2026

Copy link
Copy Markdown

Fixes #68.

Problem

runIngestPhase ingests a question's sessions one at a time. completedSessions is checkpointed inside the loop, but the accumulated ingestResult is only written after the loop finishes:

completedSessions.push(session.sessionId)
checkpointManager.updatePhase(..., "ingest", { completedSessions })   // session id persisted
...
checkpointManager.updatePhase(..., "ingest", { status: "completed", ingestResult: combinedResult })

If session 7 of 10 throws, the catch block writes status: "failed" and the document ids for sessions 1–6 are discarded with the in-memory object. On resume those sessions are skipped — they are in completedSessions — so their ids are never regenerated. The merge against existingResult cannot recover them: the failed attempt never wrote one.

Impact

ingestResult.documentIds is what indexing waits on before search is allowed to run:

const ingestResult = question.phases.ingest.ingestResult
await provider.awaitIndexing(ingestResult, question.containerTag, ...)

So after a mid-question ingest failure the run searches before the earlier sessions have finished indexing. For providers with asynchronous indexing (Supermemory, Zep, Mem0) that is a race producing quietly degraded retrieval — the memories exist but were not queryable when search ran — which reads as a provider quality problem rather than a harness bug. getSummary's indexingEpisodes under-reports by the same amount.

Fix

Seed combinedResult from the persisted result on entry instead of merging it in at the end, and write it alongside completedSessions on every iteration, so the two agree at every point in the loop.

Merging on entry is what keeps this from double-counting: a resumed attempt starts from the persisted ids and appends only the sessions it actually ingests, so the end-of-loop merge is no longer needed and is removed. No extra checkpoint writes are added — the per-session updatePhase call already existed, it just carries the ids now.

Tests

src/orchestrator/phases/ingest.test.ts (new). Every test reloads the checkpoint from disk, so it exercises the resume path rather than in-memory state.

  • ids of sessions before the failure survive on disk — fails without the fix (documentIds absent)
  • resuming after a failure ends with every session's ids exactly once — fails without the fix (only post-resume ids)
  • a single clean pass does not double-count — guards the merge-on-entry change
  • re-running a completed question is a no-op
  • taskIds stays absent for providers that don't report them
$ bun test src/orchestrator/phases/ingest.test.ts
 5 pass, 0 fail

$ bunx tsc --noEmit    # clean

Only ingest.ts and the new test file are touched. The new file is Prettier-clean; ingest.ts was already non-Prettier on main and is left as-is to keep the diff reviewable.

completedSessions was checkpointed inside the ingest loop while the
accumulated ingestResult was only written after the loop finished, so a
session that threw partway through a question discarded the document ids
of every session before it. Resume skipped those sessions, so their ids
were never regenerated and indexing waited on a short list, letting
search run before those sessions were queryable.

Seed combinedResult from the persisted result on entry and write it
alongside completedSessions each iteration, so the two agree at every
point in the loop. Merging on entry replaces the end-of-loop merge,
which would otherwise double-count on resume.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Ingest loses the document IDs of already-ingested sessions when a later session in the same question fails

1 participant