Skip to content

Add create-issue skill with Haiku delegation and cross-shell fix#13

Open
oto-macenauer-absa wants to merge 3 commits into
masterfrom
feature/skills-create-issue
Open

Add create-issue skill with Haiku delegation and cross-shell fix#13
oto-macenauer-absa wants to merge 3 commits into
masterfrom
feature/skills-create-issue

Conversation

@oto-macenauer-absa

@oto-macenauer-absa oto-macenauer-absa commented Jun 4, 2026

Copy link
Copy Markdown

Summary

Resolves #12

  • Add create-issue skill — two-phase architecture: Haiku subagent discovers repo templates (cheap), primary model drafts and posts
  • Leaner than initial prototype: 68% fewer instruction lines (~950 fewer input tokens per invocation)
  • Broader trigger matching: "log bug", "report problem", "submit ticket", etc.
  • Eval suite with 3 test cases (bug report, deep context, cross-repo feature request)
  • Fix shell compatibility: write body to temp file instead of stdin heredocs that break on PowerShell/special chars

Release Notes

Two-phase architecture with Haiku delegation

Template discovery (repo scanning, API calls, YAML parsing) is delegated to a Haiku subagent in Phase 1. Only drafting and posting runs on the primary model.

  • Template discovery is mechanical work — perfect for a cheaper, faster model
  • Saves 25-30% cost on template-heavy repos where discovery involves multiple API round-trips

Instruction footprint reduced by 68%

Skill definition went from 347 lines across 4 files down to 109 lines across 3 files (net -238 lines):

Change Lines removed
Deleted --help/SKILL.md sub-skill -29
Deleted references/gh-commands.md -112
Condensed references/templates.md -124
Leaner SKILL.md (74 to 57 lines) -17

Every line of skill definition is injected into context when the skill triggers. 238 fewer lines = ~950 fewer input tokens per invocation.

Broader trigger matching

Description now catches natural language variations: "log a bug", "report a problem", "submit a ticket", "track as an issue" — not just "create an issue".

Behavioral guardrails

  • Fix suggestions only when root cause is clear from context (prevents hallucinated fixes)
  • Template structure respected — no extra sections added beyond what the template defines
  • Body written to temp file instead of stdin heredocs — works across PowerShell and Bash

Eval suite

3 test cases covering distinct scenarios:

  • Login blank screen — bug report with TypeError context
  • Rate limiter units mismatch — deep technical context capture
  • VS Code feature request — cross-repo issue with -R flag and remote template discovery

Measured results

Metric With skill Without skill Delta
Pass rate 100% +/- 0% 93% +/- 9% +7%
Consistent --body-file 100% 0% Eliminates quoting bugs
Template discovery Yes (remote repos) No Critical for repos with issue bots

Token cost analysis:

  • 950 tokens/invocation saved by instruction reduction (v1 to v2)
  • Haiku delegation moves ~50% of tool calls to a model ~10x cheaper
  • Template compliance prevents rejected issues on repos that enforce templates via bots — avoiding a retry that would cost 2x tokens

Architecture

User prompt -> Skill triggers
  |-- Phase 1 (Haiku): discover templates -> structured JSON
  |-- Phase 2 (primary model): draft from JSON + post via gh CLI

Test plan

  • Verified skill triggers on natural language issue requests
  • Confirmed gh issue create --body-file tempfile works across PowerShell and Bash
  • Eval pass rate: 100% with skill vs 93% baseline
  • Cross-repo template discovery tested (microsoft/vscode feature_request.md)

…tter triggering

- Delegate template discovery to Haiku subagent (Phase 1) for cost reduction
  on template-heavy repos (~25-30% effective cost savings)
- Expand trigger description to catch more phrasings (log bug, report problem,
  submit ticket, track as issue)
- Add behavioral guardrails: no extra sections beyond template, fix suggestions
  only when root cause clear
- Delete redundant --help sub-skill and gh-commands.md reference
- Condense templates.md from 146 to 22 lines (rendering rules only)
- Add evals for automated testing (3 test cases with assertions)
- Net: -282 lines across skill files
…sue body

Heredocs and --body-file - break across shells (PowerShell here-strings,
special chars in markdown). Write body to temp file, use --body-file <path>,
delete after creation.

@miroslavpojer miroslavpojer left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice idea.

@lsulak lsulak left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, although, why not just just adjust the existing create-issue skill we have on the other place?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Create-issue skill: Haiku delegation, 68% leaner instructions, eval suite

3 participants