Ingest loses the document IDs of already-ingested sessions when a later session in the same question fails

[rajarshidattapy]

Description

runIngestPhase ingests a question's sessions one at a time, tracking completedSessions for resume, but only persists the accumulated ingestResult after the whole loop succeeds:

// src/orchestrator/phases/ingest.ts:65-107
for (const session of sessions) {
  if (completedSessions.includes(session.sessionId)) continue
  const result = await provider.ingest([session], { containerTag })
  combinedResult.documentIds.push(...result.documentIds)   // in memory only
  completedSessions.push(session.sessionId)
  checkpointManager.updatePhase(..., "ingest", { completedSessions })   // <-- persists the session id
}
...
checkpointManager.updatePhase(..., "ingest", { status: "completed", ingestResult: combinedResult })

completedSessions is checkpointed inside the loop; combinedResult.documentIds is not. If session 7 of 10 throws, the catch block writes status: "failed" and the document IDs for sessions 1–6 are discarded with the in-memory object.

On resume, sessions 1–6 are skipped (they are in completedSessions), so their IDs are never regenerated. The merge with a prior result at lines 87-99 does not help — existingResult is undefined, because the failed attempt never wrote one.

Impact

ingestResult.documentIds is what the indexing phase waits on:

// src/orchestrator/phases/indexing.ts:125-126, 155
const ingestResult = question.phases.ingest.ingestResult
await provider.awaitIndexing(ingestResult, question.containerTag, ...)

So after a mid-question ingest failure the run proceeds to search before the first six sessions have finished indexing, and getSummary's indexingEpisodes totals under-report by the same amount. For providers with asynchronous indexing (Supermemory, Zep, Mem0) this is a race that produces quietly degraded retrieval — the memories exist but were not yet queryable when search ran. The result looks like a provider quality problem, not a harness bug.

In the worst case (failure on the very last session of the last attempt sequence) a question can reach indexing: completed with an empty completedIds list while most of its data is still in the provider's queue.

Reproduction

Point a provider at a session that will fail (e.g. oversized content, or kill the network) partway through a multi-session question, then resume the run with the same ID and inspect checkpoint.json: phases.ingest.completedSessions lists the pre-failure sessions, while phases.ingest.ingestResult.documentIds contains only the post-resume ones.

Suggested fix

Persist ingestResult incrementally inside the loop, alongside completedSessions, so the two never disagree:

   completedSessions.push(session.sessionId)
-  checkpointManager.updatePhase(checkpoint, question.questionId, "ingest", { completedSessions })
+  checkpointManager.updatePhase(checkpoint, question.questionId, "ingest", {
+    completedSessions,
+    ingestResult: combinedResult,
+  })

and merge against the persisted value on entry rather than at the end.