Pre-submission checklist
Bug Description
Summary
The overview/health model cards silently misreport a per-model slot as "Not called yet" whenever the slot's most recent system_model_status row has fallen further than 500 rows below the head of api_logs — even though the row exists and the slot is working.
findLatestPersistedModelStatus() applies limit: 500 before it filters by role, so it scans the newest 500 system_model_status rows and then looks for a matching role / provider / model within them. system_model_status rows are dominated by high-frequency llm and embedding calls (a busy install writes thousands per day), so a lower-frequency slot — skillEvolver is the one that shows this in practice — is crowded out of the window within a few days of uptime.
When no matching row is found, the endpoint falls back to the per-process in-memory counter from the LLM facade. That counter is null at boot, so the card reads "Not called yet" — and since it resets on every bridge restart, it is very easy to mistake for a slot that has never worked.
This is independent of the wiring bug in #2362 (a slot can be genuinely unused and be unreachable in the lookup); the two compound, which is what made it hard to attribute.
Root cause
apps/memos-local-plugin/core/pipeline/memory-core.ts
// findLatestPersistedModelStatus() — :5900
const rows = repos.apiLogs.list({
toolName: "system_model_status",
limit: 500, // :5913 — applied BEFORE the role filter below
offset: 0,
});
for (const row of rows) {
const out = JSON.parse(row.outputJson) as { role?: unknown; /* … */ };
if (out.role !== role) continue; // role filter runs after the limit
if (String(out.provider ?? "") !== provider) continue;
if (String(out.model ?? "") !== model) continue;
// …
}Because limit is applied first, the 500 rows are filled almost entirely with llm/embedding entries and the target role is never reached.
Suggested Fix
Filter by role and model in the query rather than in a post-scan over a bounded top-N — i.e. order by called_at DESC and select the newest row matching tool_name='system_model_status' and the wanted role/provider/model, and let the index do the work. That removes the window entirely and is also cheaper than deserializing 500 JSON payloads on every health poll. A pragmatic stopgap would be to raise/remove the limit, but the unbounded version has the same correctness problem on a large enough table.
How to Reproduce
- On an install with a distinct
skillEvolver model configured, let the bridge run until api_logs holds more than ~500 system_model_status rows whose role is llm or embedding after the most recent skillEvolver row.
GET /api/v1/health (or open the overview) — skillEvolver.lastOkAt is null and the card reads "Not called yet", despite skillEvolver rows existing in api_logs.- Confirm the row is present but out of window:
SELECT count(*) FROM api_logs
WHERE tool_name='system_model_status'
AND called_at > (SELECT max(called_at) FROM api_logs
WHERE tool_name='system_model_status'
AND json_extract(output_json,'$.role')='skillEvolver');
- Restart the bridge: the card still reads "Not called yet" (the fallback counter is reset), which is the misleading part.
Environment
- MemOS:
memos-local-plugin v2.0.16
- Host: Hermes agent adapter, Linux
- Observed on three independent production installs. Measured offsets between the head of
system_model_status and the newest skillEvolver row: 0 rows on one install (no rows at all in the retained window), 2,193 on the second, 4,896 on the third — the latter two both read "Not called yet" while holding 144 and 195 historical skillEvolver rows.
api_logs appears capped at 10,000 rows, so the retained history is roughly a week of busy traffic; the window can be exceeded within days of uptime.
Additional Context
Found while confirming #2362 on the same installs. #2362 explains why the slot stopped being invoked (dedicated bgReflectLlm never passed to the skill subscriber); this issue explains why the card cannot tell "never invoked" apart from "invoked a while ago" in the first place. Fixing the wiring alone will restore fresh rows and hide this for a while, but any install whose evolver goes quiet for a few thousand status rows will misreport again.
Relevant precedent in the same function: the surrounding comment notes the deliberate decision to drive card colour from in-memory stats so a stale historical error row cannot mask a freshly-booted process. The persisted lookup exists precisely to give that decision a floor — it just needs to be able to reach the row it is looking for.
Happy to provide raw row dumps, timestamps, or the health payloads on request.
Pre-submission checklist
Bug Description
Summary
The overview/health model cards silently misreport a per-model slot as "Not called yet" whenever the slot's most recent
system_model_statusrow has fallen further than 500 rows below the head ofapi_logs— even though the row exists and the slot is working.findLatestPersistedModelStatus()applieslimit: 500before it filters by role, so it scans the newest 500system_model_statusrows and then looks for a matchingrole/provider/modelwithin them.system_model_statusrows are dominated by high-frequencyllmandembeddingcalls (a busy install writes thousands per day), so a lower-frequency slot —skillEvolveris the one that shows this in practice — is crowded out of the window within a few days of uptime.When no matching row is found, the endpoint falls back to the per-process in-memory counter from the LLM facade. That counter is
nullat boot, so the card reads "Not called yet" — and since it resets on every bridge restart, it is very easy to mistake for a slot that has never worked.This is independent of the wiring bug in #2362 (a slot can be genuinely unused and be unreachable in the lookup); the two compound, which is what made it hard to attribute.
Root cause
apps/memos-local-plugin/core/pipeline/memory-core.tsBecause
limitis applied first, the 500 rows are filled almost entirely withllm/embeddingentries and the target role is never reached.Suggested Fix
Filter by role and model in the query rather than in a post-scan over a bounded top-N — i.e. order by
called_at DESCand select the newest row matchingtool_name='system_model_status'and the wantedrole/provider/model, and let the index do the work. That removes the window entirely and is also cheaper than deserializing 500 JSON payloads on every health poll. A pragmatic stopgap would be to raise/remove the limit, but the unbounded version has the same correctness problem on a large enough table.How to Reproduce
skillEvolvermodel configured, let the bridge run untilapi_logsholds more than ~500system_model_statusrows whose role isllmorembeddingafter the most recentskillEvolverrow.GET /api/v1/health(or open the overview) —skillEvolver.lastOkAtisnulland the card reads "Not called yet", despiteskillEvolverrows existing inapi_logs.Environment
memos-local-pluginv2.0.16system_model_statusand the newestskillEvolverrow: 0 rows on one install (no rows at all in the retained window), 2,193 on the second, 4,896 on the third — the latter two both read "Not called yet" while holding 144 and 195 historicalskillEvolverrows.api_logsappears capped at 10,000 rows, so the retained history is roughly a week of busy traffic; the window can be exceeded within days of uptime.Additional Context
Found while confirming #2362 on the same installs. #2362 explains why the slot stopped being invoked (dedicated
bgReflectLlmnever passed to the skill subscriber); this issue explains why the card cannot tell "never invoked" apart from "invoked a while ago" in the first place. Fixing the wiring alone will restore fresh rows and hide this for a while, but any install whose evolver goes quiet for a few thousand status rows will misreport again.Relevant precedent in the same function: the surrounding comment notes the deliberate decision to drive card colour from in-memory stats so a stale historical error row cannot mask a freshly-booted process. The persisted lookup exists precisely to give that decision a floor — it just needs to be able to reach the row it is looking for.
Happy to provide raw row dumps, timestamps, or the health payloads on request.