Add supervisor pattern: /add-supervisor skill with best-fit selection - #30
Open
alexbaur wants to merge 2 commits into
Open
Add supervisor pattern: /add-supervisor skill with best-fit selection#30alexbaur wants to merge 2 commits into
alexbaur wants to merge 2 commits into
Conversation
Introduces a supervisor agent as a post-scaffold pattern (like eval gates,
governance, monitoring) applied once a project has >1 agent. A supervisor
routes user queries across the project's agents and other managed sub-agents.
The core contribution is a Selection Matrix that picks the best-fit supervisor
pattern from three options based on the user's needs, architecture, and the
complexity required — because the three are NOT interchangeable; they differ in
who owns the routing loop and what artifact lands in the bundle:
custom Hand-written LangGraph supervisor. GA, fully DAB-declarable
(an agent App under src/agents/), gated by the same CI eval
loop as any agent. The default.
supervisor_api Databricks-managed loop (Responses API) wrapped in a
declarable App. Beta (AI Gateway + UC OTel-traces preview).
agent_bricks_mas Agent Bricks Supervisor tile. NOT a DAB resource — scaffolds
a bundle-declared bootstrap job that provisions it imperatively
and a consumed-endpoint reference. UI GA / SDK Beta.
Integration is deliberately lightweight and maps to what already exists:
- Models the supervisor as "just another agent" (custom/api) so CI's
detect_patterns -> eval_gate picks it up with zero workflow changes.
- Reuses the add_agent.py wiring approach; renders graph/tools/deps variants.
- Records the choice in .agentops-stacks/manifest.yml under `supervisor:` — the
same contract CI/tooling already read.
- No new bundle-init input (choice made post-scaffold when it's meaningful).
- Honors the standing "declarative over notebook-created; document why" rule:
the MAS tile can't be a DAB resource, so it's a documented bootstrap job.
Files:
- plugin/skills/add-supervisor/SKILL.md — skill + Selection Matrix + security posture
- plugin/commands/add-supervisor.md — /add-supervisor command
- plugin/skills/agentops-stacks/scripts/add_supervisor.py — engine (3 patterns)
- .../scripts/supervisor_templates/ — graph (custom, api), tools, MAS job+notebook
- template/.../docs/supervisor-patterns.md.tmpl — pattern deep-dive in rendered project
- template/.../.agentops-stacks/manifest.yml.tmpl — supervisor contract placeholder
- workflows/single-account-single-agent.json — machine-readable decision matrix
- plugin/skills/agentops-lifecycle/SKILL.md — Step 3.5 routing note
- plugin/README.md, install_skills.sh — register the new skill/command
- tests/test_add_supervisor.py — 13 unit tests (no CLI needed), all passing
Co-authored-by: Isaac
Review pass found two defects that would break the generated supervisor App: 1. supervisor_api graph: LangChain message .type "ai" was passed to the Responses API as role "ai" (invalid — expects "assistant"). Added an explicit _ROLE_MAP so human/ai/system/tool map to valid roles. 2. agent.py inheritance: the script copied the source agent's agent.py, which — when the base agent had Lakebase memory — imports get_async_checkpointer from graph. The supervisor graph.py doesn't define it, so the App crashed on import. Now overwrite agent.py with a dedicated stateless supervisor handler (agent_supervisor.py.tmpl). Adds tests covering the agent.py overwrite, including the Lakebase-base case. Co-authored-by: Isaac
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Introduces a supervisor agent to AgentOps Stacks as a post-scaffold pattern (applied once a project has >1 agent, alongside eval gates / governance / monitoring). A supervisor routes user queries across the project's agents and other managed sub-agents.
The core contribution is a Selection Matrix that picks the best-fit supervisor pattern based on the user's needs, architecture, and required complexity — because the three options are not interchangeable. They differ in who owns the routing loop and what artifact lands in the bundle:
src/agents/src/agents/Decision matrix (first gate that fires wins)
databricks.yml) → customWhy this fits AgentOps Stacks
detect_patterns → eval_gatepicks it up with zero workflow changes.add_agent.pywiring approach; renders graph/tools/agent/deps variants per pattern..agentops-stacks/manifest.ymlundersupervisor:— the same contract CI/tooling already read.bundle initinput — the choice is made post-scaffold, when it's meaningful.Files
plugin/skills/add-supervisor/SKILL.md— skill + Selection Matrix + per-pattern security postureplugin/commands/add-supervisor.md—/add-supervisorcommandplugin/skills/agentops-stacks/scripts/add_supervisor.py— engine (3 patterns).../scripts/supervisor_templates/— graph (custom, api), agent handler, tools, MAS job + notebooktemplate/.../docs/supervisor-patterns.md.tmpl— pattern deep-dive in the rendered projecttemplate/.../.agentops-stacks/manifest.yml.tmpl— supervisor contract placeholderworkflows/single-account-single-agent.json— machine-readable decision matrix inrouting_notesplugin/skills/agentops-lifecycle/SKILL.md— Step 3.5 routing noteplugin/README.md,install_skills.sh— register the new skill/commandtests/test_add_supervisor.py— 15 unit tests (no CLI needed), all passingTesting
add_supervisor.pypass — build a fake project tree, assert all three patterns wiredatabricks.yml/ manifest / layout correctly; no Databricks CLI needed.databricks.ymlparses (yaml.safe_load), renderedgraph.py/agent.pycompile, no leftover placeholders.supervisor_apisent LangChainaimessages to the Responses API with an invalid role; (2) a supervisor cloned from a Lakebase-enabled agent inherited anagent.pyimportingget_async_checkpointer(undefined in the supervisor graph) — now overwritten with a dedicated stateless handler. Both are covered by tests.test_create_project.pytests require Databricks CLI v1.1.0 (unrelated to this change; unaffected).This pull request and its description were written by Isaac.