You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
[Bug] memos-local-plugin 2.0.19: before_prompt_build cold-start hook always times out (15s); orphan-episode replay emits capture_failed 'recovered replay did not match any persisted trace rows' #2382
[Bug] memos-local-plugin 2.0.19: two startup-recovery path issues — before_prompt_build cold-start hook always times out (15s), and orphan-episode replay emits capture_failed / "recovered replay did not match any persisted trace rows"
Both issues are new in 2.0.19 (absent in 2.0.18) and only surface on cold start / startup recovery. They are unrelated in mechanism but both live in the startup path.
Issue 1 — before_prompt_build hook times out on every cold start (15 s)
Symptom: the gateway log shows, on the first turn of any fresh session (and on every cron run), and repeatedly during the day:
[hooks] before_prompt_build handler from memos-local-plugin failed: timed out after 15000ms
In our logs: 21 times on 09-15, 11 times on 09-16 (cron runs + new sessions). 0 occurrences on 09-11 .. 09-14 with 2.0.18 — so this is a regression in 2.0.19.
Code path (from dist/adapters/openclaw/index.js):
register() calls sharedRuntime.registerBindings(api) synchronously, without waiting for bootstrap.
ensureRuntime() is runtime ? runtime : (await bootstrapPromise, runtime) — i.e. it awaits the full core bootstrap (SQLite open, migrations, providers/embedder init, recovery close of orphan episodes) before returning.
If bootstrap is still in flight when the hook first fires, the hook awaits indefinitely and the host kills it at 15 s. The turn then proceeds without any memory context injected and every session pays a 15 s stall.
Expected: either register the hook only after bootstrap completes, or make the hook non-blocking with a bounded wait (return no context rather than hanging), or at minimum don't let a cold-start bootstrap inflate every first turn. Note agent_end was already made fire-and-forget for the same class of problem (comment in the code explains this) — before_prompt_build remains async-await because it must return prependContext, but a bounded readiness wait would make the timeout avoidable.
Impact: every new session / cron run loses its first memory-enhancement injection and stalls 15 s; on time-budgeted jobs (e.g. a 10-minute report cron) this stacks dangerously with slow model calls.
Issue 2 — orphan-episode replay throws capture_failed "recovered replay did not match any persisted trace rows" after every gateway restart
Symptom (from openclaw doctor after a restart):
ERROR [core.capture] subscriber.capture_failed episodeId="ep_..."
Error: {"name":"MemosError","message":"recovered replay did not match any persisted trace rows"}
runReflect then replays those recovered episodes: extractSteps → normalizeSteps → matches normalized steps against already-persisted trace rows by content signature (with a relaxed tool-timing fallback allowed for recovered replay).
If recoveredReplay && normalized.length > 0 && matchedRows.every(row => row === null) → throws MemosError(CONFLICT) and emits capture.failed. The comment says this is intentional: fail before spending LLM calls on an invalid snapshot.
Problem: for episodes recovered from crash/restart, the reconstructed steps frequently fail to match any persisted row (timestamps drift on rebuild, tool steps reorder, etc.), so the guard fires on essentially every restart's orphan set. It is currently only log noise — global side effects are bounded because the error prevents the invalid replay — but it makes doctor noisy and suggests the recovered-replay matcher is too strict for its own recovery rows.
Expected: relax the matcher / degrade gracefully for recovered snapshots (e.g. accept row-matching by turn identity rather than full content+tool timing, or skip reflection and just close the orphan cleanly without replay), so a routine restart doesn't produce capture_failed noise.
Relationship between the two
Both are 2.0.19 startup-path regressions. They do not block agent operation or normal memory writes, but together they add per-session latency (Issue 1) and per-restart error noise (Issue 2). Happy to provide full gateway logs or run targeted repro if useful.
[Bug] memos-local-plugin 2.0.19: two startup-recovery path issues — before_prompt_build cold-start hook always times out (15s), and orphan-episode replay emits
capture_failed/ "recovered replay did not match any persisted trace rows"Environment
@memtensor/memos-local-plugin2.0.19 (npm)Both issues are new in 2.0.19 (absent in 2.0.18) and only surface on cold start / startup recovery. They are unrelated in mechanism but both live in the startup path.
Issue 1 —
before_prompt_buildhook times out on every cold start (15 s)Symptom: the gateway log shows, on the first turn of any fresh session (and on every cron run), and repeatedly during the day:
In our logs: 21 times on 09-15, 11 times on 09-16 (cron runs + new sessions). 0 occurrences on 09-11 .. 09-14 with 2.0.18 — so this is a regression in 2.0.19.
Code path (from
dist/adapters/openclaw/index.js):register()callssharedRuntime.registerBindings(api)synchronously, without waiting for bootstrap.registerRuntimeBindingsregisters:ensureRuntime()isruntime ? runtime : (await bootstrapPromise, runtime)— i.e. it awaits the full core bootstrap (SQLite open, migrations, providers/embedder init, recovery close of orphan episodes) before returning.Expected: either register the hook only after bootstrap completes, or make the hook non-blocking with a bounded wait (return no context rather than hanging), or at minimum don't let a cold-start bootstrap inflate every first turn. Note
agent_endwas already made fire-and-forget for the same class of problem (comment in the code explains this) —before_prompt_buildremains async-await because it must returnprependContext, but a bounded readiness wait would make the timeout avoidable.Impact: every new session / cron run loses its first memory-enhancement injection and stalls 15 s; on time-budgeted jobs (e.g. a 10-minute report cron) this stacks dangerously with slow model calls.
Issue 2 — orphan-episode replay throws
capture_failed"recovered replay did not match any persisted trace rows" after every gateway restartSymptom (from
openclaw doctorafter a restart):Code path (from
dist/core/capture/capture.js):memory-core.jscloses orphanopenepisodes (recoveryReason: "lightweight_startup_close"/"lightweight_startup_close_backfill", issue algorithm.lightweightMemory.enabled: true does not actually skip evolution pipeline (reward/L2/L3/skill still run) #2063 path).runReflectthen replays those recovered episodes:extractSteps→normalizeSteps→ matches normalized steps against already-persisted trace rows by content signature (with a relaxed tool-timing fallback allowed for recovered replay).recoveredReplay && normalized.length > 0 && matchedRows.every(row => row === null)→ throwsMemosError(CONFLICT)and emitscapture.failed. The comment says this is intentional: fail before spending LLM calls on an invalid snapshot.Problem: for episodes recovered from crash/restart, the reconstructed steps frequently fail to match any persisted row (timestamps drift on rebuild, tool steps reorder, etc.), so the guard fires on essentially every restart's orphan set. It is currently only log noise — global side effects are bounded because the error prevents the invalid replay — but it makes doctor noisy and suggests the recovered-replay matcher is too strict for its own recovery rows.
Expected: relax the matcher / degrade gracefully for recovered snapshots (e.g. accept row-matching by turn identity rather than full content+tool timing, or skip reflection and just close the orphan cleanly without replay), so a routine restart doesn't produce
capture_failednoise.Relationship between the two
Both are 2.0.19 startup-path regressions. They do not block agent operation or normal memory writes, but together they add per-session latency (Issue 1) and per-restart error noise (Issue 2). Happy to provide full gateway logs or run targeted repro if useful.