Skip to content

crew: glm-5.3-flash as the default worker took 22 minutes for a one-line fix on a fresh install #1522

Description

@AbirAbbas

On a fresh profile, the crew router (#1436) seats z-ai/glm-5.3-flash as the worker for a narrow fix (bugfix · worker glm-5.3-flash (openrouter) · checker glm-5.3-flash · est $0.013). In a fresh-install check on 2026-09-25 (dev-20260925-1194d4b8fe88), a one-line fix in a tiny Python repo (add returned a - b) took 46 steps and 22m37s and cost $0.0248.

The trajectory shows these as the model's own mistakes:

  • Unanchored sed -i 's/return a - b/return a + b/' matched both add and sub, so every fix broke the other function. It flip-flopped six times.
  • It read pytest's exit 5 ("no tests ran" on a module-level-assert file) as a failure.
  • It was green at step 29 and went on for 17 more steps, including a find /.

The harness made it worse in three ways, and all three are fixed since:

Ruling needed

Re-run the same fresh-install /task on current dev. If a narrow fix still takes more than a few minutes on glm-5.3-flash, decide whether the default worker for bugfix, or its λ, should move one rung up.

Replication (field)

  1. Install through curl -fsSL https://agentfield.ai/get/devaf | bash into a throwaway HOME, and paste an OpenRouter key.
  2. In a git repo with calc.py (def add(a, b): return a - b, plus correct sub/mul) and a test_calc.py of module-level asserts, run /task fix the failing test.
  3. Record steps and wall time from the task room head.

It costs about $0.03.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:providerRouting, lanes, refusals, hedging, what a call costsneeds a rulingAn owner decision comes before any code — do not start

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions