Skip to content

attribution: run the report-only provenance sweep on current main and post its telemetry #9753

Closed

Description

@kyle-sexton

Refs #3465. Slice 1 of 3 (runs first; nothing blocks it).

Problem

The only repo-wide provenance sweep on record ran on 2026-08-28 under the provenance plugin and copy rubric v2. Since then the plugin became attribution, the copy rubric is at version 5 (attribution 0.10.4, #6194), the sweep ledger machinery shipped (#5353), and the trigger-less-stamp check is live (87 findings on 2026-09-06). The disposition sweep (slice 2) and the convention entry (slice 3) need current telemetry and current adjudication candidates, and the stage-1 numbers no longer describe this tree.

Fix

Run /attribution:audit (the read-only audit action, never fix or sweep) over all tracked markdown on current main. Apply no disposition and edit no tracked file. Post one comment on #3465, headed ## Report-only sweep, <date>, <commit>, that carries:

  • telemetry: files considered and declined, stamp candidates/parsed/declined, expired count, trigger-less-stamp findings, fetches attempted/succeeded/failed, 429 count, fingerprint comparisons, categorical carve-outs, per-instance suppressions accumulated;
  • every finding with location, class and tier, and the rubric version it was graded under;
  • adjudication candidates harvested for the golden set (not converted);
  • an honest-limits section: files model-read versus only enumerated, nomination passes run, whether judge_samples panels were met.

The sweep ledger is checkout-local and gitignored, so the comment is the durable record; slices 2 and 3 read it.

Done when

  • gh issue view 3465 --comments --json comments --jq '[.comments[].body|select(startswith("## Report-only sweep"))]|length' prints at least 1
  • That comment contains the strings per-instance suppressions, rubric, and fetches
  • git diff --stat origin/main...HEAD on the run's branch is empty (no tracked file edited; no PR is opened for this slice)

Activity

  1. added
    needs-triageNot yet classified. Floor until a type and one priority tier are set.
    on Oct 24, 2026
  2. github-actions commented on Oct 24, 2026

    @github-actions
    Contributor

    Proposed work class: work-class: read-only
    Basis: automated intake triage; the lanes App applied this class.

    Size: Small
    Complexity: Medium
    Uncertainty: Known
    Flags: none
    Blocked by: none

    Understood: run the read-only /attribution:audit action over all tracked markdown on current main, with no dispositions and no file edits. Then post one ## Report-only sweep, <date>, <commit> comment on #3465. It should carry the telemetry, every finding with its rubric version, the golden-set adjudication candidates, and an honest-limits section. Nothing is missing. The issue states the scope, the output and the done-when checks. The outcome is exact and needs no owner decision. Link fetches may be limited by the lane's web access, and the limits section should record that. I found no duplicate among the open issues.

    Automated triage, applied by the lanes App.

  3. added
    priority: mediumReal value, no hard deadline; normal backlog flow.
    and removed
    needs-triageNot yet classified. Floor until a type and one priority tier are set.
    on Oct 24, 2026
  4. added
    status: needs-decisionAwaiting a human or maintainer judgment call.
    work-class: read-onlyAudits, research, reports. No repository mutation; tracker and queue writes only.
    and removed
    status: readyTriaged, unblocked, and fully specified; eligible to pick up.
    on Oct 24, 2026
  5. kyle-sexton commented on Oct 25, 2026

    @kyle-sexton
    MemberAuthor

    Released for re-triage under the owner's 2026-10-24 rule: an issue stops for a person only for an intent or vision question, a security call, a host or attended run, or spend outside prepaid credit. The hold label came off and intake triage runs again now; it re-applies a hold, with a numbered question, when one of those applies.

    To hold this issue again, add needs-human (or status: needs-decision) and say why in a comment.

  6. 1 remaining item

  7. melodic-automation-lanes commented on Oct 26, 2026

    @melodic-automation-lanes
    Contributor

    The token broker refused the work-resolve-issue run that agent-ready started because the lanes kill switch was on, so it did no work. work-unblock removed and added agent-ready back to start a fresh run. A kill-switch refusal is no fault in this issue or the lanes, so it never counts against later re-fires; after limits.limit-stops-per-issue of them since a person last added agent-ready or a run last reported, work-unblock waits longer before each one, up to 24 hours.

  8. added
    agent-readyFully specified and briefed; eligible for autonomous pickup from the frontier.
    needs-humanHuman-in-the-loop required; autonomous sessions must not resolve items carrying this.
    and removed
    agent-readyFully specified and briefed; eligible for autonomous pickup from the frontier.
    on Oct 26, 2026
  9. melodic-automation-lanes commented on Oct 26, 2026

    @melodic-automation-lanes
    Contributor

    The work-resolve-issue lane stopped partway through this issue and needs a person.

    Why, in the run's own words:

    This lane cannot finish the issue as written. (1) The issue's only deliverable is a comment posted on #3465 headed '## Report-only sweep, <date>, <commit>'. This lane has no gh or GitHub write access: Bash runs only the named lane wrappers, and the runner either opens a PR or comments on #9753, never on #3465. (2) /attribution:audit cannot run here. The attribution plugin is not loaded in this lane, and its audit flow depends on its own scripts (list-corpus.sh, extract-breadcrumbs.sh, check-stamps.sh, fingerprint.mjs via node), which the lane's Bash allowlist does not permit. Without them there is no corpus telemetry, stamp telemetry or fingerprint comparison, which are the fields the comment must carry. (3) The issue's done-when requires that no tracked file is edited and no PR is opened, but this lane turns its working tree into a PR. The issue is read-only work (work-class: read-only), and it needs an attended session, or a lane with the attribution plugin loaded and permission to comment on issues, that runs '/attribution:audit audit' on current main and posts the comment to #3465. No tracked file was changed.

    What the run says it did not do:

    • Running /attribution:audit (audit action) over tracked markdown on current main: the attribution plugin is not loaded in this lane and its detector scripts cannot run under the lane's Bash allowlist
    • Posting the '## Report-only sweep, <date>, <commit>' comment on #3465: this lane has no GitHub write access to another issue
    • PR prep (review, verify, checks steps): skipped because there is no diff; the issue says no PR is opened for this slice

    Run: https://github.com/melodic-software/claude-code-plugins/actions/runs/40372903274

    To retry, remove needs-human and add agent-ready again.

  10. added
    host-runNeeds an unattended run on a host; the host runner takes it, not a person or the issue lane.
    status: confirmedA maintainer confirmed the item is real and in scope; ready to be worked.
    and removed
    status: readyTriaged, unblocked, and fully specified; eligible to pick up.
    needs-humanHuman-in-the-loop required; autonomous sessions must not resolve items carrying this.
    on Oct 27, 2026
  11. kyle-sexton commented on Oct 28, 2026

    @kyle-sexton
    MemberAuthor

    Host run. I ran /attribution:audit in its read-only audit action over tracked markdown on main at 8dcdac6e8d93171f8642b44d9aaa5635491e6b46, with no fix and no sweep.

    The result is written up as the slice-1 comment for #3465.

    What ran:

    • Claude Code 2.1.312, attribution plugin 0.13.0, copy rubric v5, restated-fact rubric v1. The run used a detached worktree at origin/main.
    • list-corpus.sh: 3832 files considered, 235 declined (214 eval fixtures, 21 vendor), 3597 included.
    • extract-breadcrumbs.sh --dir over all 1741 directories.
    • check-stamps.sh --trigger-less over all 3597 files: 3055 stamp candidates, 3031 parsed, 24 declined, 2 expired, 49 trigger-less (87 on 2026-09-06).
    • Two unioned nomination passes over 118 model-read files, chosen by a fixed rule: all of docs/upstream/, the top 60 by external URLs, and a seeded random 20. They gave 862 nominated passages.
    • fetch-docs.sh through the docs lookup: 200 fetches attempted, 163 succeeded, 37 failed, one 429.
    • fingerprint.mjs with --min-containment 0.3 --min-span-words 15: 609 comparisons, 104 firing pairs, 246 above-rule spans.
    • Blind 3-judge panels on 45 candidates, and refutation on the 3 unanimous restated-fact STANDS.

    Results:

    • Copy: all 29 judged copy candidates cleared, so there are no fingerprint-confirmed findings and nothing is fix-eligible.
    • Restated-fact: 2 relay-eligible findings, in docs/upstream/aihero-shipping-course.md:268 and plugins/machine-health/skills/audit/reference/windows/check-catalog.md:414. 1 more was refuted and 1 came back UNKNOWN; both stay on the human report.
    • The relay findings file was written by emit-findings.sh to the gitignored .work/reviews/main/: 53 rows, 2 withheld.

    Done-when, against #9753:

    Honest limits: 118 of 3597 files were model-read. 217 above-rule spans and 186 strong restated-fact candidates were not judged. The comment on #3465 lists every limit.

    Target labels: add none; remove host-run.

  12. removed
    host-runNeeds an unattended run on a host; the host runner takes it, not a person or the issue lane.
    on Oct 28, 2026
  13. kyle-sexton commented on Oct 29, 2026

    @kyle-sexton
    MemberAuthor

    Re-triage: closing. All three Done-when checks hold.

    1. Running the issue's own command (gh issue view 3465 --comments ... select(startswith("## Report-only sweep")) | length) prints 1. The comment is #3465 issuecomment-6467462271, posted 2026-10-28T11:10:55Z and headed ## Report-only sweep, 2026-10-28, 8dcdac6e8d93171f8642b44d9aaa5635491e6b46.
    2. That comment contains per-instance suppressions, rubric and fetches (checked with a string match on its body).
    3. The 2026-10-28 host-run comment on this issue reports git diff --stat origin/main...HEAD empty in the run's worktree. No tracked file was edited and no PR was opened.

    Slices 2 and 3 read that comment. The disposition-applying slice is #9755.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    priority: mediumReal value, no hard deadline; normal backlog flow.status: confirmedA maintainer confirmed the item is real and in scope; ready to be worked.work-class: read-onlyAudits, research, reports. No repository mutation; tracker and queue writes only.

    Type

    Fields

    No fields configured for Task.

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions