Skip to content

fix(blockrun): surface Surf endpoint discovery so the agent stops guessing#67

Merged
1bcMax merged 1 commit into
mainfrom
feat/surf-endpoint-discovery
May 24, 2026
Merged

fix(blockrun): surface Surf endpoint discovery so the agent stops guessing#67
1bcMax merged 1 commit into
mainfrom
feat/surf-endpoint-discovery

Conversation

@KillerQueen-Z
Copy link
Copy Markdown
Collaborator

Problem

A user hit this cascade (real report):

✗ BlockRun GET /v1/surf/market/concept-ranking failed: Not Found (404)
✗ BlockRun GET /v1/surf/market/token-ranking  failed: Not Found (404)
✗ BlockRun GET /openapi.json — path must start with "/v1/" or "/.well-known/"
✗ BlockRun has failed 3 times this session and is now disabled.

Reproduced locally: the agent guessed non-existent Surf paths, tried to self-correct via /openapi.json (which 404s through this tool), and tripped the tool-failure circuit breaker.

Root cause (two issues, neither is "the LLM being dumb")

  1. The prompt invited guessing. Surf was advertised as wildcards (/v1/surf/market/*) + prose ("token rankings"), with no exact paths inline. The model completed the prose into a plausible-but-wrong path (market/token-ranking). The exact list lives in a skill doc that isn't reliably loaded, and /openapi.json only covers 17 of 83 surf endpoints and is unreachable through this tool (it prepends /api, and the openapi rewrite is site-root).

  2. The gateway already returned the answer — the tool hid it. A wrong /v1/surf/... path returns a 404 whose body lists every valid endpoint under available (all 83, straight from the live registry). But the tool put that in fullOutput and only showed the model "Not Found", so the agent never saw it.

Fix (prevention + cure)

  • Inline high-frequency Surf endpoints with exact paths (market/ranking, fear-greed, price, etf, options, exchange/*, liquidations, indicators) + an explicit guard: "there is no market/token-ranking or market/concept-ranking".
  • Discovery note: tell the model a wrong /v1/surf/ path returns available — read it and retry instead of looping.
  • Surface available + message into the model-visible output on any 4xx (blockrun.ts). Comes straight from the live registry, so it's complete and never drifts. Guarded with Array.isArray so non-Surf 4xx bodies are unaffected.

Regression

Same repro prompt, after the fix:

> Use the BlockRun surf API to get the current crypto market token rankings...
→ agent calls /v1/surf/market/ranking on the FIRST try
→ returns top-20 table with prices / 24h change / market cap
→ no 404s, no circuit breaker

(Tested with a forced model since the router's default pick nvidia/deepseek-v4-flash was provider-side 503 at the time — unrelated infra.)

Known limitation / follow-up

The inlined list in context.ts is hand-written, so it can drift from the gateway's surf.ts registry over time. The durable fix is a generated typed Surf tool whose endpoint enum is derived from SURF_ENDPOINTS (or /openapi.json) — tracked as a follow-up. This PR is the immediate, low-risk stop-gap; the 404-surfacing cure already covers all 83 endpoints regardless of what's inlined.

Scope

Two files (src/tools/blockrun.ts, src/agent/context.ts), +18/-7. No behavior change for non-Surf endpoints. npm run build clean.

…ssing

A user hit a cascade: the agent guessed non-existent Surf paths
(market/concept-ranking, market/token-ranking), the /openapi.json
self-correction path 404'd, and after 3 failures the tool-failure
circuit breaker disabled BlockRun for the session.

Root cause was twofold and neither was the LLM "being dumb":
1. The system prompt advertised Surf as wildcards (/v1/surf/market/*) with
   prose ("token rankings"), with no exact paths inline — so the model
   completed the prose into a plausible-but-wrong path. The real path list
   lives in a skill doc that isn't reliably loaded, and openapi discovery
   was unreachable through this tool.
2. The gateway already returns the full valid-endpoint list under
   `available` on a 404, but the tool buried it in fullOutput and only
   showed the model "Not Found" — so the agent never saw the answer.

Fixes (prevention + cure):
- Inline the high-frequency Surf endpoints (market/ranking, fear-greed,
  price, etf, options, exchange/*, liquidations, indicators) with exact
  paths, plus an explicit "there is no market/token-ranking" guard against
  the observed hallucination.
- Add an endpoint-discovery note telling the model that a wrong /v1/surf/
  path returns `available` — read it and retry instead of looping.
- Surface the gateway's `available` list (and `message`) into the
  model-visible output on any 4xx, straight from the live registry so it's
  always complete and never drifts.

Regression: same repro prompt now picks /v1/surf/market/ranking on the
first try and returns data — no 404s, no circuit breaker.
@1bcMax 1bcMax merged commit 5c8f479 into main May 24, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant