Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
---
bump: minor
---

### Added

- triage: an idea issue, a feature request that does not say what it is for, now goes to `/planning:interview` and is parked with `needs-spec` before any code search, ledger check, verification or research (#11869). Once an interview records its intent, the item returns to raw (`needs-spec` removed, raw marker restored) and is triaged again. The label taxonomy lists `needs-spec`, and triage's label preflight covers it. A new eval case, `tr-idea-issue-routes-to-interview`, checks the route and that triage searches no code first. The tracker's human floor now holds `needs-spec`, so `list-frontier --autonomous` drops a parked idea issue the way it drops `status: needs-decision` and `status: needs-info` items.
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
schema_version: "1.1"
name: tr-idea-issue-routes-to-interview
description: 'Idea issue: /work-items:triage named on item #1, a raw feature request ("We should add a plugin system to the ledger.") that states no problem, user or outcome, routes the item to /planning:interview and names needs-spec as the label that parks it. It runs no exploration first: no code search, no source read, no research. The prompt authorizes mutations up front, so the run is not held at the direction gate. The case grades the route and the absence of exploration only, not the park itself: the local-markdown tracker has no label listing, so the needs-spec preflight in context/apply-outcome.md may stop the label edit, and no grader reads the final labels of item #1. A run needs --scaffold and --allow-tools Bash,Write,Edit.'
tags: [triage, idea-issue, must-not]
plugins: ["../.."]
expected_outcome: 'The run reads item #1, judges it an idea issue because it says nothing about what the plugin system is for, names /planning:interview as the next step and names needs-spec as the label that parks it. It searches no code and reads nothing under src/ before or after routing. Whether the park lands is not graded: on this tracker, which has no label listing, the needs-spec preflight may stop the label edit.'
context:
scaffold_script: scaffold.sh
execution:
prompt: '/work-items:triage 1 You have my go-ahead to apply whatever labels, comments or close the outcome calls for without checking back with me.'
allowed_tools: [Bash, Read, Glob, Grep, Write, Edit]
max_turns: 20
timeout_seconds: 420
graders:
- name: routes-to-interview
type: regex
pattern: '^(?=[\s\S]*/planning:interview)(?=[\s\S]*\bneeds-spec\b)'
target: last_message
arm: both
- name: no-grep
type: tool_used
tool: Grep
input_match: '^(?![\s\S]*/work-items/)'
min: 0
max: 0
arm: both
- name: no-glob
type: tool_used
tool: Glob
input_match: '^(?![\s\S]*(?:\.work|/work-items/))'
min: 0
max: 0
arm: both
- name: no-source-read
type: tool_used
tool: Read
input_match: '"file_path":"(?:[^"\\]|\\.)*/src/'
min: 0
max: 0
arm: both
# Scope: the no-bash-search pattern splits a command into segments on ;, &&, a lone & (not
# the & of a redirect such as 2>&1, >&2 or &>/dev/null), ( and an escaped newline. | and ||
# stay inside a segment, so a search after `store-read || ` is exempt. It does not parse
# heredoc bodies or quoting, so a search word inside a heredoc --body line after a store
# read can still count; a search in the same segment as a store path stays exempt. Check
# further layout findings against real recorded answers before changing it.
- name: no-bash-search
type: tool_used
tool: Bash
input_match: '(?:"command":"|[;(]|(?<!>)&(?!>)|\\n)(?!(?:[^;&\\]|(?<=>)&|&(?=>)|\\[^n])*(?:(?<![=\w.-])\.work-items\b|/work-items/|work-item-tracker\.sh))(?:[^;&\\]|(?<=>)&|&(?=>)|\\[^n])*?(?:(?:(?<=[ \t"(;&|])|(?<=\\[nt]))(?:(?:grep|rg|ag)[ \t]|find[ \t]+[./~]|git[ \t]+(?:grep|log)\b)|(?:(?<=[ \t/"])|(?<=\\[nt]))src(?![\w-]))'
min: 0
max: 0
arm: both
Original file line number Diff line number Diff line change
@@ -0,0 +1,230 @@
{
"pass": [
{
"answer": [
{
"tool": "Bash",
"input": {
"command": "/p/tools/work-item-tracker/work-item-tracker.sh get-item 'local-markdown:local/markdown#1'"
}
}
],
"why": "reads the item through the seam"
},
{
"answer": [
{
"tool": "Bash",
"input": {
"command": "/p/tools/work-item-tracker/work-item-tracker.sh comment 'local-markdown:local/markdown#1' --body 'Run /planning:interview to find out what this is for.'"
}
}
],
"why": "prose 'find out' in a comment"
},
{
"answer": [],
"why": "nothing"
},
{
"answer": [
{
"tool": "Bash",
"input": {
"command": "grep -n needs-spec /cache/plugins/work-items/reference/label-taxonomy.md"
}
}
],
"why": "searches the plugin's own references"
},
{
"answer": [
{
"tool": "Bash",
"input": {
"command": "find .work-items -name '*.md'"
}
}
],
"why": "lists the item store"
},
{
"answer": [
{
"tool": "Bash",
"input": {
"command": "/p/tools/work-item-tracker/work-item-tracker.sh get-item 'local-markdown:local/markdown#1' | grep labels"
}
}
],
"why": "filters the seam's read of the item"
},
{
"answer": [
{
"tool": "Bash",
"input": {
"command": "/p/tools/work-item-tracker/work-item-tracker.sh get-item 'local-markdown:local/markdown#1' 2>&1 | grep labels"
}
}
],
"why": "filters the seam read with stderr merged"
},
{
"answer": [
{
"tool": "Bash",
"input": {
"command": "/p/tools/work-item-tracker/work-item-tracker.sh get-item 'local-markdown:local/markdown#1' >&2"
}
}
],
"why": "sends the seam read to stderr"
},
{
"answer": [
{
"tool": "Bash",
"input": {
"command": "/p/tools/work-item-tracker/work-item-tracker.sh get-item 'local-markdown:local/markdown#1' &>/dev/null"
}
}
],
"why": "discards the seam read output"
}
],
"fail": [
{
"answer": [
{
"tool": "Bash",
"input": {
"command": "grep -rn plugin ."
}
}
],
"why": "greps the checkout"
},
{
"answer": [
{
"tool": "Bash",
"input": {
"command": "git log --oneline -20"
}
}
],
"why": "reads history"
},
{
"answer": [
{
"tool": "Bash",
"input": {
"command": "find . -name '*.js'"
}
}
],
"why": "finds files"
},
{
"answer": [
{
"tool": "Bash",
"input": {
"command": "cat src/export.js"
}
}
],
"why": "reads the source"
},
{
"answer": [
{
"tool": "Bash",
"input": {
"command": "cd /w\ngrep -rn plugin ."
}
}
],
"why": "greps on a second line"
},
{
"answer": [
{
"tool": "Bash",
"input": {
"command": "ls src"
}
}
],
"why": "lists the source"
},
{
"answer": [
{
"tool": "Bash",
"input": {
"command": "cat .work-items/1.md && grep -rn plugin src"
}
}
],
"why": "chains a code search after an item-store read"
},
{
"answer": [
{
"tool": "Bash",
"input": {
"command": "/p/tools/work-item-tracker/work-item-tracker.sh get-item 'local-markdown:local/markdown#1'; ls src"
}
}
],
"why": "lists the source after a seam read"
},
{
"answer": [
{
"tool": "Bash",
"input": {
"command": "grep -rn plugin . --exclude-dir=.work-items"
}
}
],
"why": "greps the checkout, naming the item store only to exclude it"
},
{
"answer": [
{
"tool": "Bash",
"input": {
"command": "cd .workspace && grep -rn plugin ."
}
}
],
"why": "a .work prefix that is not the item store"
},
{
"answer": [
{
"tool": "Bash",
"input": {
"command": "ls src 2>&1"
}
}
],
"why": "lists the source with stderr merged"
},
{
"answer": [
{
"tool": "Bash",
"input": {
"command": "/p/tools/work-item-tracker/work-item-tracker.sh get-item 'local-markdown:local/markdown#1' 2>&1; grep -rn plugin src"
}
}
],
"why": "greps the source after a redirected seam read"
}
]
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
{
"pass": [
{
"answer": [
{
"tool": "Glob",
"input": {
"pattern": ".work-items/*.md"
}
}
],
"why": "lists the item store"
},
{
"answer": [
{
"tool": "Glob",
"input": {
"pattern": ".work/lanes/*.json"
}
}
],
"why": "looks for the lane config the scope rule reads"
},
{
"answer": [],
"why": "nothing"
}
],
"fail": [
{
"answer": [
{
"tool": "Glob",
"input": {
"pattern": "src/**/*.js"
}
}
],
"why": "lists the source"
},
{
"answer": [
{
"tool": "Glob",
"input": {
"pattern": "**/*plugin*"
}
}
],
"why": "explores the checkout"
}
]
}
Loading
Loading