PR Review Slop stage emits template/runtime errors (CommentReview/SuggestRemoval)

[taras]

Summary

The Slop section of PR Review can fail with raw template/runtime errors instead of structured feedback.

Repro Context (baseline)

• PR: docs: align runtime specs with consolidated runtime APIs #52
• PR URL: docs: align runtime specs with consolidated runtime APIs #52
• Branch: docs/runtime-spec-alignment
• Head commit: 0eca22e02bd43a14ed3eff26428f68c4ff868db5
• Review comment: docs: align runtime specs with consolidated runtime APIs #52 (comment)
• Workflow run (PR Review): https://github.com/taras/executable-markdown-agents/actions/runs/23250893379

Observed Errors

In the Slop section, the bot outputs:

• <!-- ERROR: Assignment to constant variable. -->
• <!-- ERROR: Failed to evaluate expression prop "dismissedReplies={newDismissReplies}" on <SuggestRemoval />: newDismissReplies is not defined -->

Likely Source Path

• .reviews/policies/SlopPolicy.md includes <CommentReview pr={pr} />
• .reviews/components/CommentReview.md computes newDismissReplies and passes it to <SuggestRemoval ... />
• .reviews/components/SuggestRemoval.md consumes dismissedReplies

The second error likely cascades from an earlier failure that prevents newDismissReplies from being defined.

Expected Behavior

• Slop policy should produce stable output or cleanly skip optional steps.
• Runtime/template errors should not leak into PR comments.

Suggested Fix Directions

• Make CommentReview resilient when earlier eval blocks fail.
• Guard SuggestRemoval invocation with safe defaults/defined checks.
• Add diagnostics in workflow logs for template-eval failures.

Replay / Compare

Re-run the PR Review workflow on commit 0eca22e02bd43a14ed3eff26428f68c4ff868db5 and compare produced comment output against the linked baseline comment.

Note: output may drift if prompts/templates/workflow changed since baseline run.

[taras]

Still live as of 2026-07-16 — four months after the #52 baseline. Fresh recurrence data from unrelated PRs:

PR	Slop Assignment to constant variable.
#77	leaked
#79	leaked
#80	leaked
#81	clean (✅ Slop indicators look low.)

Key new signal: the error is diff-dependent, not universal. #81 (a pure deletion PR) produced a clean Slop section while #80 (a broad rename touching many files) leaked the error. That points at the failure being triggered by some property of the diff/comment set that CommentReview.md processes — consistent with this issue's hypothesis that newDismissReplies is left undefined on certain inputs and the const reassignment throws before it's computed.

Only the first of the two errors from the original report (Assignment to constant variable.) showed up in these recent runs; I did not see the newDismissReplies is not defined cascade in #77/#79/#80, which fits the theory that the second is downstream of the first.

Note: the repro links in the issue body point at github.com/taras/executable-markdown-agents, which is now taras/executable.md (repo renamed). The workflow/actions URLs still resolve via GitHub's redirect.