Skip to content

agent: Fail-fast preflight verify step before run_agent #563

Description

@krokoko

Component

Agent (Python runtime)

Describe the feature

Support an explicit verify_build or verify_lint step before the first run_agent that fails the task without invoking the LLM when the repo is already broken.

steps:
  - { kind: clone_repo, name: setup }
  - { kind: verify_build, name: preflight, gate: strict, on_failure: fail }
  - { kind: run_agent, name: implement }
  - { kind: verify_build, name: build, gate: regression_only }
  - { kind: ensure_pr, name: open_pr, strategy: create }

Use case

What exists today (not sufficient):

  • clone_repo / setup_repo already runs mise run build and mise run lint deterministically and records build_before / lint_before in RepoSetup.
  • If the initial build fails, the task still proceeds to run_agent — failure is noted in setup notes and prompts tell the agent to run build, but there is no workflow-level fail-fast gate.
  • Post-agent verify_build runs only after the agent consumes tokens.

Gap: Operators need a declarative, deterministic preflight gate — not prompt guidance — to skip LLM execution when the tree is already red.

Orchestrator pre-flight (ARCHITECTURE.md) covers GitHub reachability; this covers repo health after clone.

Proposed solution

  1. Validator — If a verify_* step appears before run_agent, gate must be strict or regression_only (not informational); record baseline in workflow_state.json for post-agent regression_only compare.
  2. Runner — Step order already supported; add test proving preflight failure never invokes run_agent.
  3. Costclone_repo already runs build; consider skipping duplicate full build in clone when a preflight verify_build immediately follows, or document opt-in on workflows only.
  4. Docs — WORKFLOWS.md preflight pattern + cost trade-off.
  5. Opt-in — Enable on coding/new-task-v1 only after cold-start cost measurement.

Other information

Acknowledgements

  • I may be able to implement this feature
  • This might be a breaking change

Metadata

Metadata

Assignees

No one assigned

    Labels

    agent-runtimePython agent container: pipeline, runner, hooks, prompts, tools, DockerfileenhancementNew feature or requestvalidation-loopTasks related to improve the validation loop for ABCA's codebase

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions