Prompt recall always fails on self-hosted backends: 3s HOOK_RECALL_TIMEOUT_MS vs 9-17s cold /v4/profile

[Tomauskasz]

Summary

On a self-hosted Supermemory backend, prompt recall (UserPromptSubmit) never returns memories. Every turn emits:

◪ supermemory · recall unavailable; continuing without recalled context

The cause is a client-side cap, not auth, not the backend, and not the Codex hook budget: HOOK_RECALL_TIMEOUT_MS is 3000 ms, while a cold /v4/profile call against the self-hosted server takes 9–17 s. SessionStart is unaffected because it uses a 30 s cap, so the profile loads at session start and then never refreshes per prompt. That asymmetry makes the failure look like a relevance problem rather than a timeout.

Environment

• macOS 15 (Darwin 25.6.0)
• codex-cli 0.154.0
• codex-supermemory 1.0.18 (constants below verified against the published tarball)
• Self-hosted backend: supermemory-server-loopback-0.0.8 on http://127.0.0.1:6767
• recallMode: "direct"
• UserPromptSubmit hook timeout in ~/.codex/hooks.json: 90 s

Measurements

Single-container POST /v4/profile against the local server, same query:

Run	Status	Wall time
cold 1	200	8.90 s
cold 2	200	11.39 s
cold 3	200	17.28 s
warm (repeat query)	200	0.11 s

The response is correct in every case — 42 KB of profile and search results. It is simply slower than 3 s until the query is cached.

Recall does not make one such call. getAllReadTags produced 11 container tags for this repository, and getHookProfileWithSearchMany fires all 11 concurrently, each with its own 3 s AbortController:

recall: start: {"query":"...","tags":{... "allReads":[
  "repo_<name>__<id>", "coding_personal", "user_project_<id>",
  "claudecode_project_<id>", "codex_user_<id>", "opencode_user_<id>",
  "cursor_user_<id>", "repo_<name>", "codex_project_<id>",
  "opencode_project_<id>", "cursor_project_<id>"]},"recallMode":"direct"}

No recall: error line follows, so the failure is the !profileResult.success branch: all 11 requests abort, mergeProfileResults finds zero successes and returns success: false.

Warm-cache control: after warming one query with curl, recall.js run with that exact prompt returned ◪ supermemory · recalled 5 memories (267 tok). The next cold prompt failed again. Recall works whenever the backend happens to answer inside 3 s and fails otherwise, which on a self-hosted instance is almost always.

Root cause

src/services/hookRecallClient.ts:6

export const HOOK_RECALL_TIMEOUT_MS = 3000;

versus src/hooks/session-start.ts:15

const SESSION_START_HOOK_TIMEOUT_MS = 30_000;

Both are hardcoded. Neither is readable from ~/.codex/supermemory.json, and neither honours the host's configured hook timeout, which here is 90 s. The 3 s cap is the binding constraint and it is not reachable by configuration.

Verification of the fix

Raising HOOK_RECALL_TIMEOUT_MS to 30000 and leaving everything else unchanged:

◪ supermemory · recalled 5 memories (217 tok)
elapsed: 24s

Cold query, recallMode: "direct", 11-container fan-out, first attempt. That is the exact case that failed before.

Secondary finding: wasted work when recall is disabled

src/hooks/recall.ts calls getTags(cwd) (line 93 on main) before the recallMode short-circuits at lines 103 and 107. src/services/tags.ts contains 7 execSync git invocations. In recallMode: "advisory" and "off", the tags are computed and then discarded.

Measured cost of that dead work, per prompt:

• inside a git repository: 2.21 s, 2.29 s, 2.32 s
• outside a git repository: 1.14 s
• payload that exits before getTags (empty prompt): 0.06 s
• bare node -e "" baseline: 0.06 s

So advisory mode, which exists to avoid recall latency, still pays roughly 2 s per prompt for tags it never uses. Moving the two recallMode checks above getTags(cwd) removes it. Caching the git lookups would also help direct mode.

Suggested changes

1. Raise HOOK_RECALL_TIMEOUT_MS to match SESSION_START_HOOK_TIMEOUT_MS (30 s), or make it configurable via ~/.codex/supermemory.json. Self-hosted backends do not answer profile+search in 3 s cold.
2. Distinguish "no relevant memories" from "every request timed out" in the status line. The current message is identical for a timeout and for a genuinely empty result, which makes this hard to diagnose.
3. Move the recallMode === "off" and recallMode === "advisory" short-circuits above getTags(cwd).

Related: #59 notes under "Additional observation" that searching all legacy containers pushes the hook past its time budget. This issue is the same pressure measured directly, with the specific constant responsible.

Offer

I am running change 1 as a local patch and it has been reliable. Happy to open a PR for it, and for change 3, if you want either — say the word and I will send them.

[Tomauskasz]

Measured suggested change 3 (moving the recallMode short-circuits above getTags(cwd)) on the same machine.

Patch applied to src/hooks/recall.ts: the off and advisory checks now run immediately after the empty-query guard, before cwd, getTags, and getSessionId. The two original blocks are removed, and the early exit logs recall: short-circuit so the hook still leaves a debug breadcrumb.

recallMode: "advisory", three consecutive prompts inside a git repository:

	before	after
run 1	2.21 s	0.075 s
run 2	2.29 s	0.076 s
run 3	2.32 s	0.071 s

Bare node -e "" on this machine is 0.057 s, so after the change the hook is essentially process startup. Emitted directive text is unchanged, off still emits nothing, and direct is unaffected — a cold 11-container recall in direct mode after the patch still returned ◪ supermemory · recalled 5 memories (232 tok).

One datapoint for change 1 while I was there: that cold direct run took 28.4 s wall (recall: start 06:25:31.306 → recall: done 06:25:59.689). A 30 s cap clears it with only 1.6 s of headroom on an 11-container fan-out. If you take the timeout change, 30 s is the floor rather than a comfortable value — deriving it from the host's configured hook timeout would be sturdier than another hardcoded number.

Both changes are small and independent. Still happy to send them as PRs.