Skip to content

Watcher: quote arXiv phrases and fix hint ids — 27 of 39 recent issues were noise - #95

Merged
emmanuelgjr merged 5 commits into
mainfrom
fix/watch-arxiv-noise
Sep 15, 2026
Merged

emmanuelgjr merged 5 commits into
mainfrom
fix/watch-arxiv-noise

Conversation

@emmanuelgjr

Copy link
Copy Markdown
Contributor

Plan ID: — (follows the watch backlog triage) Ticket: — Wave: —
Constraints honored:

  • C4: the watcher's hints are triage suggestions on an issue, not mappings. Ids were corrected to match the entry titles the rules already named.
  • C2: no docs/ change.
  • C1: none.

Stacked on #37, because the new test file triggers the CI jobs that are red on main (Drift 11). Merge #37, then retarget to main. This PR touches no generated file, so it won't conflict with #87/#88/#90.

Why

The watcher opened 39 arXiv issues between 2026-08-31 and 2026-09-14. Triage closed 27 of them as out of scope (CBDC settlement, 6G NOMA, MIMO lattice hardness, TPM measured-boot tracing…). All carry triage:out-of-scope with a reason.

Cause: unquoted multi-word terms. In the arXiv API ti:LLM+security is ti:LLM plus a bare security term matched in any field, so the query admitted nearly all of cs.CR.

Live check, 2026-09-14, 30 newest results AI-related titles
Current query 3 / 30
This PR's query (quoted, title-scoped) 30 / 30

The issue bodies' Suggested OWASP Mapping was wrong too:

Changes

  • scripts/watch.js
    • arXiv query with quoted phrases: "prompt injection", "large language model", "AI agent", "retrieval-augmented", plus jailbreak, LLM, agentic, RAG, MCP, still inside cat:cs.CR.
    • ARXIV_HINT_RULES: word-bounded patterns, each naming the entry title it targets.
    • main() guarded by require.main === module, and the query/rules exported for tests. Running the file directly behaves as before.
  • scripts/watch.test.mjs (6 tests)
    • Multi-word terms are quoted, plus a test that runs the same check on the old query and expects all four defects.
    • The query stays in cs.CR.
    • No substring matches ("storage", "toolchain").
    • 2026 ids are used.
    • Every hint id exists in data/entries and still carries the title its rule names, so the next renumbering fails CI instead of silently mis-hinting.

Dry run (node scripts/watch.js --dry-run --watcher arxiv): the 20 papers returned are LLM, agent or RAG security topics.

Deliberately NOT changed

Noted while testing

While testing, I loaded the previous watch.js with require(). It had no require.main guard, so it ran all watchers once from this machine. With no GITHUB_TOKEN set it opened nothing; the issue list confirms none were created. It wrote no state file. This PR adds the guard that would have prevented that.

Verify

$ node --test scripts/watch.test.mjs   6/6
$ npm test                              57/57
$ node scripts/validate.js              0 error(s), 84 warning(s), 312 passed  (unchanged)

🤖 Generated with Claude Code

https://claude.ai/code/session_014SfR2YzLxRH54DAVzDk8gR

emmanuelgjr and others added 4 commits August 30, 2026 10:10
Every docs/*.js bundle carried a `// Generated: <today>` header. Nothing
else in the generator output varies between runs, so that one line was
the whole reason a regenerate on a different day showed four dirty
files — and, since #30, the reason the Generator reproducibility job and
the `committed entries match a fresh generation` test could only pass on
the day the bundles were last committed.

- generate.js: drop the run-date lines; the Source header now reads the
  version from package.json instead of a hard-coded, stale `v1.5.2`.
- generate.test.mjs: assert no bundle header names a run or carries a date.
- validate.yml: the reproducibility job now diffs every generated
  artefact (backlinks.json, backlinks.js, frameworks-registry.js too).
- CONTRIBUTING.md: document the build contract — generated files, the
  determinism requirement, and why the bundles are committed (Pages
  serves docs/ from main; no deploy workflow exists).

Determinism only. No structural, route, layout or logo change (C2).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0147wBugcuzLkswKPqgofcke
…s were noise

The weekly arXiv watcher opened 39 issues between 2026-08-31 and
2026-09-14; triage closed 27 as outside GenAI security (CBDC settlement,
6G NOMA, MIMO lattices, TPM tracing). Cause: unquoted multi-word terms.
`ti:LLM+security` parses as `ti:LLM` OR a bare `security` in any field,
so the query admitted almost every cs.CR paper. A live check on
2026-09-14 returned 3 AI-related titles in 30; the quoted, title-scoped
query returned 30 in 30.

The "Suggested OWASP Mapping" hints were also wrong in two ways: no word
boundaries (`rag` matched "storage"/"average") and pre-2026 ids (supply
chain pointed at LLM10, now Improper Output Handling; poisoning at LLM04,
now Supply Chain). Rules now name the entry title they target.

- watch.js: quoted query; ARXIV_HINT_RULES with boundaries and 2026 ids;
  main() guarded by require.main so the module can be tested without
  running the watchers.
- watch.test.mjs (6): query phrases quoted, and the check proven against
  the old query; cs.CR scope; no substring hits; 2026 ids; every hint id
  exists and still carries the title its rule names.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014SfR2YzLxRH54DAVzDk8gR
Conflicts were only in the four generated webapp bundles, and only in
their header lines: #92 regenerated them with a `// Generated:` date
that this branch removes. Resolved by re-running scripts/generate.js on
the merged sources; a second run is byte-identical.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014SfR2YzLxRH54DAVzDk8gR
Generated files re-run on the merged sources; a second generation is byte-identical.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014SfR2YzLxRH54DAVzDk8gR
Base automatically changed from fix/T-ENG03-deterministic-webapp-build to main September 14, 2026 21:08
#37 landed as a squash, so this branch's copies of its commits resolve as identical content. Generated files re-run on the merged sources; a second generation is byte-identical.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014SfR2YzLxRH54DAVzDk8gR
@emmanuelgjr
emmanuelgjr merged commit c040920 into main Sep 15, 2026
6 checks passed
@emmanuelgjr
emmanuelgjr deleted the fix/watch-arxiv-noise branch September 15, 2026 01:25
emmanuelgjr added a commit that referenced this pull request Sep 15, 2026
Generated files re-run on the merged sources; a second generation is byte-identical. text-integrity test passes on this branch's files.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014SfR2YzLxRH54DAVzDk8gR
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant