fix: replace shell-based context updates with marker-based upsert#2259
Merged
mnriem merged 11 commits intogithub:mainfrom Apr 17, 2026
Merged
fix: replace shell-based context updates with marker-based upsert#2259mnriem merged 11 commits intogithub:mainfrom
mnriem merged 11 commits intogithub:mainfrom
Conversation
Replace ~3500 lines of bash/PowerShell agent context update scripts
with a Python-based approach using <!-- SPECKIT START/END --> markers.
IntegrationBase now manages the agent context file directly:
- upsert_context_section(): creates or updates the marked section at
init/install/switch time with a directive to read the current plan
- remove_context_section(): removes the section at uninstall, deleting
the file only if it becomes empty
- __CONTEXT_FILE__ placeholder in command templates is resolved per
integration so the plan command references the correct agent file
- context_file is persisted in init-options.json for extension access
The plan command template instructs the LLM to update the plan
reference between the markers in the agent context file.
Removed:
- scripts/bash/update-agent-context.sh (857 lines)
- scripts/powershell/update-agent-context.ps1 (515 lines)
- 56 integration wrapper scripts (update-context.sh/.ps1)
- templates/agent-file-template.md
- agent_scripts frontmatter key and {AGENT_SCRIPT} replacement logic
- update-context reference from integration.json
- tests/test_cursor_frontmatter.py (tested deleted scripts)
Added:
- upsert/remove context section methods on IntegrationBase
- __CONTEXT_FILE__ placeholder support in process_template()
- context_file field in init-options.json (init/switch/uninstall)
- Per-integration tests: context file correctness, plan reference,
init-options persistence (78 new context_file tests)
- End-to-end CLI validation across all 28 integrations
Contributor
There was a problem hiding this comment.
Pull request overview
This PR replaces the previously shell-driven “update agent context” workflow with a Python-managed, marker-based upsert/remove implementation on IntegrationBase, preventing unbounded growth of agent context files while keeping integration command templates referencing the correct context file.
Changes:
- Add marker-based context section management (
<!-- SPECKIT START/END -->) toIntegrationBaseand invoke it during integration setup/teardown. - Remove shared bash/PowerShell context update scripts and per-integration wrapper scripts; update templates and tests accordingly.
- Add
__CONTEXT_FILE__placeholder support in template processing and persistcontext_filein.specify/init-options.json.
Show a summary per file
| File | Description |
|---|---|
| tests/test_presets.py | Updates preset expectations after removing the agent-file template. |
| tests/test_extensions.py | Removes agent_scripts placeholder expectations in skill rendering tests. |
| tests/test_extension_skills.py | Removes agent_scripts placeholder expectations for extension skill rendering. |
| tests/test_cursor_frontmatter.py | Deletes tests tied to removed shell/PowerShell context updater scripts. |
| tests/test_agent_config_consistency.py | Removes consistency checks that referenced the deleted context update scripts. |
| tests/integrations/test_integration_generic.py | Updates generic integration tests to validate context markers and context_file persistence. |
| tests/integrations/test_integration_forge.py | Updates forge integration tests to validate marker upsert and plan command context-file references. |
| tests/integrations/test_integration_copilot.py | Updates copilot integration tests to validate plan command context-file references and file inventory. |
| tests/integrations/test_integration_claude.py | Updates claude integration tests to validate marker upsert content instead of scripts. |
| tests/integrations/test_integration_catalog.py | Adjusts catalog descriptor expectations for scripts becoming empty. |
| tests/integrations/test_integration_base_yaml.py | Updates base YAML integration tests to validate marker upsert/remove and context-file references. |
| tests/integrations/test_integration_base_toml.py | Updates base TOML integration tests to validate marker upsert/remove and context-file references. |
| tests/integrations/test_integration_base_skills.py | Updates base skills integration tests to validate marker upsert/remove and context-file references. |
| tests/integrations/test_integration_base_markdown.py | Updates base Markdown integration tests to validate marker upsert/remove and context-file references. |
| tests/integrations/test_cli.py | Updates CLI init integration test to validate marker upsert and context_file in init-options. |
| templates/commands/plan.md | Replaces “run agent context scripts” instructions with marker-based update instructions using __CONTEXT_FILE__. |
| templates/agent-file-template.md | Removes the old agent context file template. |
| src/specify_cli/integrations/windsurf/scripts/update-context.sh | Removes obsolete wrapper script (delegated to removed shared script). |
| src/specify_cli/integrations/windsurf/scripts/update-context.ps1 | Removes obsolete wrapper script (delegated to removed shared script). |
| src/specify_cli/integrations/vibe/scripts/update-context.sh | Removes obsolete wrapper script (delegated to removed shared script). |
| src/specify_cli/integrations/vibe/scripts/update-context.ps1 | Removes obsolete wrapper script (delegated to removed shared script). |
| src/specify_cli/integrations/trae/scripts/update-context.sh | Removes obsolete wrapper script (delegated to removed shared script). |
| src/specify_cli/integrations/trae/scripts/update-context.ps1 | Removes obsolete wrapper script (delegated to removed shared script). |
| src/specify_cli/integrations/tabnine/scripts/update-context.sh | Removes obsolete wrapper script (delegated to removed shared script). |
| src/specify_cli/integrations/tabnine/scripts/update-context.ps1 | Removes obsolete wrapper script (delegated to removed shared script). |
| src/specify_cli/integrations/shai/scripts/update-context.sh | Removes obsolete wrapper script (delegated to removed shared script). |
| src/specify_cli/integrations/shai/scripts/update-context.ps1 | Removes obsolete wrapper script (delegated to removed shared script). |
| src/specify_cli/integrations/roo/scripts/update-context.sh | Removes obsolete wrapper script (delegated to removed shared script). |
| src/specify_cli/integrations/roo/scripts/update-context.ps1 | Removes obsolete wrapper script (delegated to removed shared script). |
| src/specify_cli/integrations/qwen/scripts/update-context.sh | Removes obsolete wrapper script (delegated to removed shared script). |
| src/specify_cli/integrations/qwen/scripts/update-context.ps1 | Removes obsolete wrapper script (delegated to removed shared script). |
| src/specify_cli/integrations/qodercli/scripts/update-context.sh | Removes obsolete wrapper script (delegated to removed shared script). |
| src/specify_cli/integrations/qodercli/scripts/update-context.ps1 | Removes obsolete wrapper script (delegated to removed shared script). |
| src/specify_cli/integrations/pi/scripts/update-context.sh | Removes obsolete wrapper script (delegated to removed shared script). |
| src/specify_cli/integrations/pi/scripts/update-context.ps1 | Removes obsolete wrapper script (delegated to removed shared script). |
| src/specify_cli/integrations/opencode/scripts/update-context.sh | Removes obsolete wrapper script (delegated to removed shared script). |
| src/specify_cli/integrations/opencode/scripts/update-context.ps1 | Removes obsolete wrapper script (delegated to removed shared script). |
| src/specify_cli/integrations/kiro_cli/scripts/update-context.sh | Removes obsolete wrapper script (delegated to removed shared script). |
| src/specify_cli/integrations/kiro_cli/scripts/update-context.ps1 | Removes obsolete wrapper script (delegated to removed shared script). |
| src/specify_cli/integrations/kimi/scripts/update-context.sh | Removes obsolete wrapper script (delegated to removed shared script). |
| src/specify_cli/integrations/kimi/scripts/update-context.ps1 | Removes obsolete wrapper script (delegated to removed shared script). |
| src/specify_cli/integrations/kilocode/scripts/update-context.sh | Removes obsolete wrapper script (delegated to removed shared script). |
| src/specify_cli/integrations/kilocode/scripts/update-context.ps1 | Removes obsolete wrapper script (delegated to removed shared script). |
| src/specify_cli/integrations/junie/scripts/update-context.sh | Removes obsolete wrapper script (delegated to removed shared script). |
| src/specify_cli/integrations/junie/scripts/update-context.ps1 | Removes obsolete wrapper script (delegated to removed shared script). |
| src/specify_cli/integrations/iflow/scripts/update-context.sh | Removes obsolete wrapper script (delegated to removed shared script). |
| src/specify_cli/integrations/iflow/scripts/update-context.ps1 | Removes obsolete wrapper script (delegated to removed shared script). |
| src/specify_cli/integrations/goose/scripts/update-context.sh | Removes obsolete wrapper script (delegated to removed shared script). |
| src/specify_cli/integrations/goose/scripts/update-context.ps1 | Removes obsolete wrapper script (delegated to removed shared script). |
| src/specify_cli/integrations/generic/scripts/update-context.sh | Removes obsolete wrapper script (delegated to removed shared script). |
| src/specify_cli/integrations/generic/scripts/update-context.ps1 | Removes obsolete wrapper script (delegated to removed shared script). |
| src/specify_cli/integrations/generic/init.py | Sets context_file and switches setup flow to marker upsert + __CONTEXT_FILE__ substitution. |
| src/specify_cli/integrations/gemini/scripts/update-context.sh | Removes obsolete wrapper script (delegated to removed shared script). |
| src/specify_cli/integrations/gemini/scripts/update-context.ps1 | Removes obsolete wrapper script (delegated to removed shared script). |
| src/specify_cli/integrations/forge/scripts/update-context.sh | Removes obsolete wrapper script (delegated to removed shared script). |
| src/specify_cli/integrations/forge/scripts/update-context.ps1 | Removes obsolete wrapper script (delegated to removed shared script). |
| src/specify_cli/integrations/forge/init.py | Switches forge setup flow to marker upsert + __CONTEXT_FILE__ substitution. |
| src/specify_cli/integrations/cursor_agent/scripts/update-context.sh | Removes obsolete wrapper script (delegated to removed shared script). |
| src/specify_cli/integrations/cursor_agent/scripts/update-context.ps1 | Removes obsolete wrapper script (delegated to removed shared script). |
| src/specify_cli/integrations/copilot/scripts/update-context.sh | Removes obsolete wrapper script (delegated to removed shared script). |
| src/specify_cli/integrations/copilot/scripts/update-context.ps1 | Removes obsolete wrapper script (delegated to removed shared script). |
| src/specify_cli/integrations/copilot/init.py | Switches copilot setup flow to marker upsert + __CONTEXT_FILE__ substitution. |
| src/specify_cli/integrations/codex/scripts/update-context.sh | Removes obsolete wrapper script (delegated to removed shared script). |
| src/specify_cli/integrations/codex/scripts/update-context.ps1 | Removes obsolete wrapper script (delegated to removed shared script). |
| src/specify_cli/integrations/codebuddy/scripts/update-context.sh | Removes obsolete wrapper script (delegated to removed shared script). |
| src/specify_cli/integrations/codebuddy/scripts/update-context.ps1 | Removes obsolete wrapper script (delegated to removed shared script). |
| src/specify_cli/integrations/claude/scripts/update-context.sh | Removes obsolete wrapper script (delegated to removed shared script). |
| src/specify_cli/integrations/claude/scripts/update-context.ps1 | Removes obsolete wrapper script (delegated to removed shared script). |
| src/specify_cli/integrations/bob/scripts/update-context.sh | Removes obsolete wrapper script (delegated to removed shared script). |
| src/specify_cli/integrations/bob/scripts/update-context.ps1 | Removes obsolete wrapper script (delegated to removed shared script). |
| src/specify_cli/integrations/auggie/scripts/update-context.sh | Removes obsolete wrapper script (delegated to removed shared script). |
| src/specify_cli/integrations/auggie/scripts/update-context.ps1 | Removes obsolete wrapper script (delegated to removed shared script). |
| src/specify_cli/integrations/amp/scripts/update-context.sh | Removes obsolete wrapper script (delegated to removed shared script). |
| src/specify_cli/integrations/amp/scripts/update-context.ps1 | Removes obsolete wrapper script (delegated to removed shared script). |
| src/specify_cli/integrations/agy/scripts/update-context.sh | Removes obsolete wrapper script (delegated to removed shared script). |
| src/specify_cli/integrations/agy/scripts/update-context.ps1 | Removes obsolete wrapper script (delegated to removed shared script). |
| src/specify_cli/integrations/base.py | Implements marker-based upsert/remove, adds __CONTEXT_FILE__ template replacement, updates setup/teardown flows. |
| src/specify_cli/agents.py | Removes agent_scripts handling in frontmatter path rewriting and placeholder resolution. |
| src/specify_cli/init.py | Removes integration.json update-script wiring and persists context_file in init-options. |
| scripts/powershell/update-agent-context.ps1 | Removes the legacy PowerShell context updater script. |
| scripts/bash/update-agent-context.sh | Removes the legacy bash context updater script. |
| pyproject.toml | Stops bundling the removed agent-file template into the wheel. |
Copilot's findings
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comments suppressed due to low confidence (1)
src/specify_cli/integrations/base.py:485
remove_context_section()also usescontent.find(CONTEXT_MARKER_END)without anchoring it to the start marker position. With multiple marker pairs (or an END marker appearing before START), this can delete an unintended region. Usecontent.find(CONTEXT_MARKER_END, start_idx + len(CONTEXT_MARKER_START))and ensure the end marker occurs after the start marker before removing.
content = ctx_path.read_text(encoding="utf-8")
start_idx = content.find(self.CONTEXT_MARKER_START)
end_idx = content.find(self.CONTEXT_MARKER_END)
if start_idx == -1 or end_idx == -1:
return False
end_of_marker = end_idx + len(self.CONTEXT_MARKER_END)
if end_of_marker < len(content) and content[end_of_marker] == "\n":
end_of_marker += 1
# Also strip a blank line before the section if present
if start_idx > 0 and content[start_idx - 1] == "\n":
if start_idx > 1 and content[start_idx - 2] == "\n":
start_idx -= 1
new_content = content[:start_idx] + content[end_of_marker:]
- Files reviewed: 82/82 changed files
- Comments generated: 1
Address Copilot review: content.find(CONTEXT_MARKER_END) searched from the start of the file rather than after the located start marker. If the file contained a stray end marker before the start marker, the wrong slice could be replaced. Now both upsert_context_section() and remove_context_section() pass start_idx as the second argument to find() and validate end_idx > start_idx before performing the replacement.
Contributor
There was a problem hiding this comment.
Copilot's findings
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Files reviewed: 82/82 changed files
- Comments generated: 4
1. Fix grammar in _build_context_section() directive text — add commas for a complete sentence. 2. Resolve __CONTEXT_FILE__ in resolve_skill_placeholders() — skills generated via extensions/presets for codex/kimi now replace the placeholder using the context_file value from init-options.json. 3. Handle Cursor .mdc frontmatter — when creating a new .mdc context file, prepend alwaysApply: true YAML frontmatter so Cursor auto-loads the rules. 4. Fix empty-file leading newline — when the context file exists but is empty, write the section directly instead of prepending a blank line.
Contributor
There was a problem hiding this comment.
Copilot's findings
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Files reviewed: 82/82 changed files
- Comments generated: 3
1. Ensure .mdc frontmatter on existing files — upsert_context_section() now checks for missing YAML frontmatter on .mdc files during updates (not just creation), so pre-existing Cursor files get alwaysApply. 2. Guard against context_file=None — use 'or ""' instead of a default arg so explicit null values in init-options.json don't cause a TypeError in str.replace(). 3. Clean up .mdc files on removal — remove_context_section() treats files containing only the Speckit-generated frontmatter block as empty, deleting them rather than leaving orphaned frontmatter.
Contributor
There was a problem hiding this comment.
Copilot's findings
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Files reviewed: 82/82 changed files
- Comments generated: 4
1. CRLF-safe .mdc frontmatter check — use lstrip().startswith('---')
instead of startswith('---\n') so CRLF files don't get duplicate
frontmatter.
2. CRLF-safe .mdc removal check — normalize line endings before
comparing against the sentinel frontmatter string.
3. Call remove_context_section() during integration_uninstall() — the
manifest-only uninstall was leaving the managed SPECKIT markers
behind in the agent context file.
4. Fix stale docstring — remove 'agent_scripts' mention from
test_lean_commands_have_no_scripts().
Contributor
There was a problem hiding this comment.
Copilot's findings
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Files reviewed: 82/82 changed files
- Comments generated: 3
1. Remove unused script_type parameter from _write_integration_json() and all 3 call sites — the parameter was no longer referenced after the update-context script removal. 2. Fix _build_context_section() docstring — correct example path from '.specify/plans/plan.md' to 'specs/<feature>/plan.md'. 3. Improve .mdc frontmatter-only detection in remove_context_section() — use regex to match any YAML frontmatter block (not just the exact Speckit-generated one), so .mdc files with additional frontmatter keys are also cleaned up when no body content remains.
Contributor
There was a problem hiding this comment.
Copilot's findings
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comments suppressed due to low confidence (1)
src/specify_cli/integrations/base.py:462
- When creating a new
.mdccontext file, the inserted YAML frontmatter is justalwaysApply: true. Previously Cursor.mdcrules included additional metadata (e.g.,descriptionandglobs) and some Cursor versions/features rely on those fields for scoping/UX. It would be safer to write the full expected frontmatter (description/globs/alwaysApply) to preserve compatibility with existing Cursor behavior.
# Cursor .mdc files require YAML frontmatter to be loaded
if ctx_path.suffix == ".mdc":
new_content = "---\nalwaysApply: true\n---\n\n" + section
else:
- Files reviewed: 82/82 changed files
- Comments generated: 2
1. Handle partial/corrupted markers in upsert_context_section() — if only the START marker exists (no END), replace from START through EOF. If only the END marker exists, replace from BOF through END. This keeps upsert idempotent even when a user accidentally deletes one marker. 2. Parse .mdc YAML frontmatter properly — new _ensure_mdc_frontmatter() helper parses existing frontmatter and ensures alwaysApply: true is set, rather than just checking for the --- delimiter. Handles missing frontmatter, existing frontmatter without alwaysApply, and already-correct frontmatter.
Contributor
There was a problem hiding this comment.
Copilot's findings
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Files reviewed: 82/82 changed files
- Comments generated: 3
1. Rewrite _ensure_mdc_frontmatter() with regex — preserves comments, formatting, and custom keys in existing frontmatter instead of destructively re-serializing via yaml.safe_dump(). Inserts or fixes alwaysApply: true in place. 2. Add 6 focused .mdc frontmatter tests to cursor-agent test file: new file creation, missing frontmatter, preserved custom keys, wrong alwaysApply value, idempotent upserts, removal cleanup. 3. Call remove_context_section() during integration switch Phase 1 — prevents stale SPECKIT markers from being left in the old integration's context file. Also clear context_file from init-options during the metadata reset.
Contributor
There was a problem hiding this comment.
Copilot's findings
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Files reviewed: 83/83 changed files
- Comments generated: 4
…ze bare CR 1. Remove unused MDC_FRONTMATTER class variable — dead code after _ensure_mdc_frontmatter() was rewritten with regex. 2. Preserve inline comments when fixing alwaysApply — the regex substitution now captures trailing '# comment' text and keeps it. 3. Normalize bare CR in upsert_context_section() — match the behavior of remove_context_section() which already normalizes both CRLF and bare CR. 4. Clarify .mdc removal comment — 'treat frontmatter-only as empty' instead of misleading 'strip frontmatter'.
Contributor
There was a problem hiding this comment.
Copilot's findings
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comments suppressed due to low confidence (1)
src/specify_cli/integrations/base.py:557
- Similar to
upsert_context_section(), the newline-skipping logic afterCONTEXT_MARKER_ENDonly checks for\n. On CRLF files this can leave a stray\r(which later normalizes to an extra newline), so removal may leave behind an unintended blank line. Consider advancing over\r\nas well.
end_of_marker = end_idx + len(self.CONTEXT_MARKER_END)
if end_of_marker < len(content) and content[end_of_marker] == "\n":
end_of_marker += 1
- Files reviewed: 83/83 changed files
- Comments generated: 3
…tion 1. Handle corrupted markers in remove_context_section() — mirror upsert's behavior: start-only removes start→EOF, end-only removes BOF→end. Previously bailed out leaving partial markers behind. 2. CRLF-safe end-marker consumption — both upsert and remove now handle \r\n after the end marker, not just \n. Prevents extra blank lines at replacement boundaries in CRLF files. 3. Clarify path rule in plan template — distinguish filesystem operations (absolute paths) from documentation/agent context references (project-relative paths).
Contributor
There was a problem hiding this comment.
Copilot's findings
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Files reviewed: 83/83 changed files
- Comments generated: 1
remove_context_section() previously treated mismatched markers as corruption and aggressively removed from BOF→end-marker or start-marker→EOF, which could delete user-authored content if only one marker remained. Now it only removes when both START and END markers exist and are properly ordered, returning False otherwise.
Contributor
There was a problem hiding this comment.
Copilot's findings
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Files reviewed: 83/83 changed files
- Comments generated: 0 new
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #2246
Summary
Replace ~3,500 lines of bash/PowerShell agent-context-update scripts with a
Python-based approach using
<!-- SPECKIT START/END -->markers. This eliminatesthe unbounded growth of agent context files (e.g.
CLAUDE.md) that occurred when/speckit.planappended duplicate "Active Technologies" and "Recent Changes"entries on every feature plan.
What changed
IntegrationBasenow manages the agent context file directly:upsert_context_section()— creates or replaces the marked section atinit/install/switch time with a directive to read the current plan. Repeated
runs overwrite instead of appending, so the file stays a fixed size.
remove_context_section()— removes the section at uninstall, deleting thefile only when it becomes empty.
__CONTEXT_FILE__placeholder — resolved per integration so the plancommand references the correct agent file.
context_fileininit-options.json— persisted for extension access.Removed
scripts/bash/update-agent-context.sh(857 lines)scripts/powershell/update-agent-context.ps1(515 lines)update-context.sh/.ps1)templates/agent-file-template.mdagent_scriptsfrontmatter key and{AGENT_SCRIPT}replacement logicupdate-contextreference fromintegration.jsontests/test_cursor_frontmatter.py(tested deleted scripts)Added
IntegrationBase__CONTEXT_FILE__placeholder support inprocess_template()context_filefield ininit-options.json(init/switch/uninstall)Net impact
-3,515 lines / +555 lines across 82 files.