Skip to content

map-memories: even time-slice allocation starves batches when backlog is large (zero progress loop) #356

Open
Open
map-memories: even time-slice allocation starves batches when backlog is large (zero progress loop)#356
Labels
bugSomething isn't working

Description

@octal-illumination
octal-illumination
opened on Aug 22, 2026

Short description

The map-memories dreamer task divides its whole task deadline evenly across the remaining batches. With MAP_BATCH_SIZE = 80 (hardcoded) and the schema-default timeout_minutes: 20, per-batch prompt budget = 1200s ÷ ceil(backlog/80). Any backlog above ~880 memories (≥12 batches) drops the budget to ≤100s. An agentic model executing map-memories through tool calls needs minutes per batch. Result: every batch times out, nothing is ever committed, and the loop repeats until the deadline decays — a zero-progress feedback loop that worsens as the backlog grows.

What happened?

Summary

The map-memories dreamer task divides its whole task deadline evenly across the remaining batches:

// dist/index.js (@cortexkit/opencode-magic-context@0.38.0), mapMemories():
const deadline   = startedAt + config.timeoutMinutes * 60 * 1000;
...
const remainingMs      = Math.max(0, args.deadline - Date.now());
const batchesRemaining = batches.length - i;
const sliceMs          = Math.max(1, Math.floor(remainingMs / batchesRemaining));
// → promptSyncWithValidatedOutputRetry(..., { timeoutMs: sliceMs })

With MAP_BATCH_SIZE = 80 (hardcoded) and the schema-default timeout_minutes: 20, per-batch prompt budget = 1200s ÷ ceil(backlog/80). Any backlog above ~880 memories (≥12 batches) drops the budget to ≤100s. An agentic model executing map-memories through tool calls needs minutes per batch. Result: every batch times out, nothing is ever committed, and the loop repeats until the deadline decays — a zero-progress feedback loop that worsens as the backlog grows.

This is the out-of-the-box configuration. At reproduction time our config contained no timeout_minutes key at all (config-at-starvation-repro.jsonc); we verified against upstream source that,

(a) the official setup flow never asks about or writes this key — dreamer-setup.ts persists only model and per-task schedule — and
(b) the effective default is plugin-internal (installed v0.38.0: schema .default(20); upstream HEAD: runtime ?? 20 in task-config.ts).

The only code path that writes it into a user config is a legacy v1→v2 migration gated on an old key our config never had. No user action produces this state; every fresh install lands in it.

Environment

Component Value
Harness OpenCode TUI, local server (opencode --port 5012)
OpenCode version 1.18.21
Plugin @cortexkit/opencode-magic-context@0.38.0 (pinned in opencode.jsonc "plugin" array)
Dreamer/sidekick/historian model nvidia/nvidia/nemotron-3-ultra-550b-a55b (NVIDIA NIM, free tier)
OS Linux (x86_64)
Backlog at observation remaining=916→921 unmapped inputs ⇒ batches=12 (ceil(921/80)=12, matches logs)

Steps to reproduce

  1. Fresh install / default config: do not set timeout_minutes (the official setup flow never writes it — verified in dreamer-setup.ts; the starving default is what every new user gets).
  2. Accumulate a memory backlog ≥ ~880 items (⇒ ≥12 batches at MAP_BATCH_SIZE=80 ⇒ ≤100s slices at the default budget; observed: 916–921 items → 12 batches).
  3. Run the dreamer on a model whose per-batch tool-loop cost exceeds deadline/batches (~100s here).
  4. Observe map-memories batch failed: "prompt timed out after <sliceMs>" repeating every tick with no committed mappings.

Actual behavior (primary evidence)

Plugin log (magic-context.log), all lines from one day, verbatim timestamps and slice values:

[2026-08-22T02:01:40Z] batch failed: "prompt timed out after 99984ms"
[2026-08-22T02:03:21Z] batch failed: "prompt timed out after 99915ms"
[2026-08-22T02:05:02Z] batch failed: "prompt timed out after 99804ms"
[2026-08-22T02:06:43Z] batch failed: "prompt timed out after 99712ms"
[2026-08-22T02:08:23Z] batch failed: "prompt timed out after 99621ms"
[2026-08-22T02:10:03Z] batch failed: "prompt timed out after 99523ms"
[2026-08-22T02:11:44Z] batch failed: "prompt timed out after 99237ms"
[2026-08-22T02:13:24Z] batch failed: "prompt timed out after 99059ms"
[2026-08-22T02:15:04Z] batch failed: "prompt timed out after 98658ms"
[2026-08-22T02:16:44Z] batch failed: "prompt timed out after 98336ms"
[2026-08-22T02:18:23Z] batch failed: "prompt timed out after 97914ms"
[2026-08-22T02:20:00Z] batch failed: "prompt timed out after 96230ms"

Twelve consecutive attempts, slices monotonically decaying 100s→96s exactly as the formula predicts (the shared 1200s deadline depletes while batchesRemaining stays constant because no batch ever succeeds). Task summaries throughout: map-memories: mapped=0 ... complete=false.

Backlog-growth proof of the feedback loop: remaining=916 (01:54Z) → remaining=921 (02:20Z) — the backlog grew during hours of retries because new memories were indexed while none were ever mapped.

Knob verification (proves the mechanism, not coincidence): after setting timeout_minutes: 60 and restarting, the very next timeout was 299979ms — i.e., 3600000ms ÷ 12 batches = 300000ms minus 21ms of pre-slice elapsed time. Byte-for-byte match with the traced formula.

Natural experiment corroborating the batch-count dependence: two days earlier, the same task with a small backlog ran as a single batch and completed cleanly:

[2026-08-20T20:40:54Z] map-memories: mapped=21 independent=9 batches=1 remaining=0 complete=true

With batchesRemaining = 1, the slice equals the entire remaining task window — ample time. Same model class of workload, only the batch count differs: 1 batch → success; 12 batches → 12 consecutive ~100s timeouts.

Expected behavior

Per-batch budgets should be derived from expected batch cost (or adapted), and/or the system should detect sliceMs < plausible batch cost early and fail loudly / reduce scope instead of silently burning the entire task window on doomed prompts. At minimum, progress made inside a timed-out batch should not be wholly discarded.

Suggested fixes (any one suffices to close the loop)

  1. Cost-aware slicing: estimate per-batch cost (rolling average of completed batch durations) and set timeoutMinutes guidance or split batches so cost_estimate ≤ slice.
  2. Adaptive batching: if measured pace exceeds slice, halve the batch (re-chunk the 80 inputs) instead of failing.
  3. Early-exit with warning: when remainingMs / batchesRemaining < MIN_PLAUSIBLE_BATCH_MS, abort the run up-front with an actionable message ("backlog too large for task budget; raise timeout_minutes").
  4. Docs: surface MAP_BATCH_SIZE / per-task budget interplay in the schema description for timeout_minutes.

Workaround used (works, but is manual)

~/.config/cortexkit/magic-context.jsonc:

"dreamer": {
  "tasks": {
    "map-memories": {
      "schedule": "0 2 * * *",
      "timeout_minutes": 120   // 7200s ÷ 12 batches = 600s/batch
    }
  }
}

This eliminated timeouts entirely (verified live: previous 300s slices produced 299979ms timeouts; after the change the first batch ran ~8 min and reached validation). But it only shifts the cliff — the starvation returns whenever backlog × per-batch cost outgrows the window again. Note that the existence of this workaround does not downgrade the issue: the defect is that the default configuration (key absent, which is what setup produces and what most users will run) has no viable path once the backlog outgrows the silent 20-minute budget.

Upstream tracker check (no duplicate)

no prior issue covers even-slice allocation, timeout_minutes adequacy, or batch-count starvation. The formula is still present at master HEAD (map-memories.ts:181-182) and in the latest release v0.38.1 (2026-08-21, changelog does not touch it). Tangentially related, not duplicates: #272 (dreamer progress visibility), #270 (sub-agent empty result).

Notes for maintainers

  • Two larger slices observed under default config (353823ms, 390045ms) are also formula-consistent: those runs evidently had fewer eligible inputs (≤3 batches ⇒ ~240 inputs), implying loadUnmappedInputs eligibility varies between runs. Worth documenting alongside the fix.

Attachment manifest (this package)

File Contents
batch-failures-default-20min-budget-full-lines.log 14 full failure lines under default budget — incl. the monotonic 99984→96230ms decay series and two large-slice outliers (353823/390045ms, formula-consistent with reduced eligible-batch counts)
batch-failures-60min-budget-full-lines.log 3 lines at 60-min budget: 299979ms == floor(3600s/12)−21ms (exact prediction match), then 297582/297439ms
map-memories-progress-summaries-all.log All task summaries: mapped=0 throughout; remaining 916→921 (backlog growth during zero-progress retries)
dist-slice-allocation-code.txt Verbatim excerpts from running @0.38.0 dist/index.js: mapMemories() loop, deadline line, MAP_BATCH_SIZE=80
config-current-after-workaround.jsonc Full config as of filing (timeout_minutes: 120 workaround in place)
config-at-starvation-repro.jsonc Exact on-disk config during the starvation reproduction (model refs fixed; no timeout_minutes key → default 20-min budget)
config-original-before-any-fix.jsonc Provenance snapshot: state before any same-day fix (broken model refs, no timeout key)
CONFIG-STATE-TIMELINE.txt Provenance chain linking each config snapshot to the failure regime it produced
environment.txt Environment matrix + reproduction timeline (UTC)
magic-context-issue-20260822-150700.md Official doctor --issue output (auto-collected diagnostics: config paths, plugin cache v0.37.0-cached/v0.38.1-latest, storage, sanitized log tail)

Diagnostics



Plugin version


0.36.0


OpenCode version


1.18.20


Platform


linux x64


Client


OpenCode TUI (CLI)


Log output (optional)


Metadata
Metadata
Assignees
No one assigned
    Labels
    bugSomething isn't working
    Type
    No type
    Projects
    No projects
    Milestone
    No milestone
    Relationships
    None yet
    Development
    No branches or pull requests
    Issue actions