fix(nv_build): declare glm-5.2's real limits so calls stop failing - #390
Open
Mark2Mac wants to merge 1 commit into
Open
fix(nv_build): declare glm-5.2's real limits so calls stop failing#390Mark2Mac wants to merge 1 commit into
Mark2Mac wants to merge 1 commit into
Conversation
Mark2Mac
force-pushed
the
fix/nv-build-stale-registry
branch
from
August 18, 2026 08:25
c485261 to
427e2ad
Compare
Mark2Mac
changed the title
The bundled registry gave z-ai/glm-5.2 a 1000000-token context window and no
output cap. model_info derives the output budget as
ctx * (1 - MAX_INPUT_TOKENS_PCT), so every request asked for 250000 output
tokens and the endpoint answered:
400 This model configuration accepts at most 202749 combined input and output
tokens. However, your request has 1249 input tokens and asks for 250000
output tokens (251249 tokens total).
202749 is quoted verbatim by the endpoint in that 400. With the entry corrected
the same scan completes with 4/4 LLM calls and the meta-analyzer applied.
An over-stated context window does not degrade gracefully: it zeroes the LLM
stage, and nothing in the error points at the registry. Under-stating is safe,
over-stating is not.
Limits may vary per account, which is now noted in the YAML.
Scope is deliberately one entry. The registry also names three models the
catalogue no longer serves, but removing them is coupled to DEFAULT_MODEL by an
invariant the suite already asserts ("nv_build's default model is in its
registry"), so that change travels with the default in a separate PR.
Refs NVIDIA#388
Signed-off-by: Mark2Mac <Mark2Mac@users.noreply.github.com>
Mark2Mac
force-pushed
the
fix/nv-build-stale-registry
branch
from
August 18, 2026 09:10
427e2ad to
4ec490c
Compare
Mark2Mac
added a commit
to Mark2Mac/SkillSpector
that referenced
this pull request
Aug 18, 2026
…ed one Depends on NVIDIA#390. Three registry entries name models GET /v1/models does not serve: deepseek-v4-flash (410 Gone since 2026-08-07), deepseek-v4-pro, and glm-5.1. One of them is DEFAULT_MODEL, and another is the meta_analyzer slot override, so with no SKILLSPECTOR_MODEL set the out-of-the-box path failed every call. Removing them and retargeting the default is ONE change, not two. The suite already asserts the invariant that couples them, in test_constants: "nv_build's default model is in its registry — no warnings expected" Dropping the default from the registry while leaving it as the default breaks that test, and it is right to break: a default the registry does not describe gets its token budget from a guess, silently. Splitting these two edits was tried and abandoned for exactly this reason. The replacement is chosen for DETECTION, not latency, and that is the part worth arguing about. On a bait skill carrying credential exfiltration disguised as a synchronisation step, the fast served model (deepseek-v4-flash-0731, ~1.6 s/call) completed every call, reported no degradation, and returned a clean verdict. A confident false negative is the worst failure mode a security scanner has: it is the case where someone signs off. z-ai/glm-5.2 costs ~16 s per call on the same skill and returns CRITICAL. The meta_analyzer slot loses its override rather than gaining a new one. The aggregation pass does benefit from a stronger model, but naming a second model doubles the surface that can go stale — which is how the previous default rotted unnoticed. Happy to restore an override if preferred. Refs NVIDIA#388 Signed-off-by: Mark2Mac <Mark2Mac@users.noreply.github.com>
Mark2Mac
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of #388, and deliberately one entry wide.
The bundled registry gave
z-ai/glm-5.2a 1 000 000-token context window and no output cap.model_infoderives the output budget asctx * (1 - MAX_INPUT_TOKENS_PCT), so every request asked for 250 000 output tokens:202749is quoted verbatim by the endpoint in that 400. With the entry corrected, the same scan completes with 4/4 LLM calls and the meta-analyzer applied.An over-stated context window does not degrade gracefully: it zeroes the LLM stage, and nothing in the error points at the registry. Under-stating is safe, over-stating is not. Limits may vary per account, now noted in the YAML.
On scope. The registry also names three models the catalogue no longer serves. Removing them is coupled to
DEFAULT_MODELby an invariant this suite already asserts — "nv_build's default model is in its registry" (test_constants.py) — so that change travels with the default in #391. I tried splitting it the other way first; the test correctly refused.1044 passed, ruff format and check clean, DCO signed.