Skip to content

Log tool-surfacing outcome on every tools/list call#21

Merged
swarup-padhi-glean merged 5 commits into
mainfrom
enhance-tools-list-logging
Jun 23, 2026
Merged

Log tool-surfacing outcome on every tools/list call#21
swarup-padhi-glean merged 5 commits into
mainfrom
enhance-tools-list-logging

Conversation

@swarup-padhi-glean

Copy link
Copy Markdown
Contributor

What

Enhance logging on the tools/list path so the most common question — "why don't my tools appear?" — is answerable from ~/.glean/glean-server.log alone.

Before, the handler only logged on the two failure paths (connect.backend-error, tools-list.fetch-failed). The success path and both pre-auth early returns (no server URL, no token) were silent, so you couldn't tell whether an empty dynamic surface was caused by missing config, missing auth, a backend blip, or a backend that genuinely returned nothing.

Change

Emit one structured tools-list.served line on every return path of the handler, via a small serve(reason) helper:

tools-list.served {"static":3,"dynamic":7,"names":["search","read_document","chat","memory","memory_schema","user_activity","employee_search"],"reason":"fetched"}
  • static — count of always-present local tools (find_skills, run_tool, setup)

  • dynamic / names — the remote tools actually surfaced (freshly fetched, or served from cache on a blip)

  • reason — which path produced this result:

    reason meaning
    unconfigured no server URL set → served static + cache, skipped remote
    unauthenticated no token → served static + cache, skipped remote
    connect-error couldn't open remote client → served static + last-known cache
    fetch-failed listTools failed → served static + last-known cache
    fetched fresh remote list succeeded

The existing connect.backend-error / tools-list.fetch-failed error lines (which carry the error msg) are kept — the new line complements them with counts + outcome.

Because the allow-list only ever drops tools outside our fixed 7-tool set, a missing allow-listed name (e.g. chat absent from names) means the backend never returned it — so the name list alone distinguishes "backend didn't serve it" from "auth/connectivity problem."

Privacy: only tool names, counts, and the reason tag are logged — never argument values (consistent with summarizeCall).

No behavior change to which tools are served on any path.

Note / tradeoff

tools-list.served fires on every tools/list, and hosts poll that on idle cycles — so it's chatty on a healthy plugin (this is consistent with the already-always-on error lines). It's exactly what you want when debugging tool-surfacing. If the noise is unwanted in steady state, an easy follow-up is to gate the reason === "fetched" case behind a debug env flag while keeping the diagnostic paths always-on. Happy to do that if preferred.

Test plan

  • npm run typecheck
  • npm test ✓ (154 passed)
  • npm run build
  • bash scripts/check-version-bump.sh origin/main ✓ (0.2.30 → 0.2.31, all three manifests aligned)

Version bumped to 0.2.31 across all host manifests; plugins/glean/dist/index.js rebuilt by the pre-commit hook.

The tools/list handler only logged on failure paths (connect.backend-error,
tools-list.fetch-failed), so the success and pre-auth early-return paths were
silent. That left the most common question — "why don't my tools appear?" —
unanswerable from the log: you couldn't tell whether the dynamic surface was
empty because of a missing server URL, missing token, a backend blip, or a
backend that genuinely returned nothing.

Emit one structured tools-list.served line on every return path with the
static count, the surfaced dynamic tool names, and a reason tag
(unconfigured | unauthenticated | connect-error | fetch-failed | fetched).
Because the allow-list only ever drops tools outside our fixed set, a missing
allow-listed name (e.g. chat) means the backend never returned it. Only tool
names, counts, and the reason are logged — never argument values.

No change to which tools are served.

@eshwar-sundar-glean eshwar-sundar-glean left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! We should make sure all these works are folding into feedback tool

Comment thread src/index.ts Outdated
// outside our fixed set, so a missing allow-listed name (e.g. `chat`) means
// the backend never returned it. Only tool *names*, counts and the reason
// tag are logged — never argument values, which can carry PII/secrets.
const serve = (reason: string): { tools: Tool[] } => {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it fair to call this reason, this is more of "state"

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good call — renamed reasonstate in d1f9bb3.

@eshwar-sundar-glean

eshwar-sundar-glean commented Jun 22, 2026

Copy link
Copy Markdown
Collaborator

@swarup-padhi-glean - Before merging, just check this, you are mutating tools array by adding new tools, previously this was not the case. I am thinking what happens if the state keeps changing, so there will be duplicate copies. I think this might happen please do check before merge. This should probably not happen since tools array is in tools/list call, but just check once

Per review: build the served list as a fresh [...staticTools, ...dynamic]
in the serve() helper instead of pushing into the per-call tools array, so
the handler never mutates a shared array (cachedRemoteTools is only
reassigned or spread, never mutated). Rename the log field reason -> state.

No behavior change to which tools are served on any path.
@swarup-padhi-glean

Copy link
Copy Markdown
Contributor Author

@eshwar-sundar-glean thanks for the careful read on both points 🙏

Mutation / duplicate copies: tools is allocated fresh on every tools/list call (const tools = [...] inside the handler), and I never mutated the module-level cachedRemoteToolspush(...cachedRemoteTools) spreads its elements into the per-call array; the cache itself is only ever reassigned. So there was no cross-call accumulation. That said, your instinct is right that keeping it non-mutating is cleaner — in d1f9bb3 I refactored serve() to return a fresh [...staticTools, ...dynamic] and dropped all .push, so the handler is now provably non-mutating (and output is identical on every path).

Folding into the feedback tool: these lines all land in ~/.glean/glean-server.log, which the feedback tool already captures — so they fold in for free, no extra wiring. Happy to add tools-list.served to whatever the feedback bundle explicitly surfaces if useful.

Three open PRs (#21/#22/#23) were stacked on 0.2.31; move this one to the
next version so it stays strictly above main and mergeable regardless of
order against a single 0.2.31 PR.
main advanced to 0.2.32 (#24 merged), so move above it again.
Resolve version-line conflicts in the three plugin manifests to 0.2.33
(above main's 0.2.32). dist rebuilt from merged source; src auto-merged
(logging change + #24 find_skills hint). typecheck + 154 tests pass.
@swarup-padhi-glean swarup-padhi-glean merged commit 7fb9625 into main Jun 23, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants