chore: add ASF source headers and a release header audit gate - #3397
chore: add ASF source headers and a release header audit gate#3397Astro-Han wants to merge 2 commits into
Conversation
66d3014 to
d3e0eef
Compare
jackwener
left a comment
There was a problem hiding this comment.
Automated review of exact head d3e0eefd1f8934d1e1ac94a1e7b3bbafcc634b69.
Blocking correctness finding: the mechanical sweep prepends a second ASF header to .asf.yaml. The parent already begins with the standard ASF header, but its license-URL indentation is four spaces while renderHeader("hash") emits two. Because hasHeader accepts only byte-exact rendered text, applyHeader treats the existing valid header as absent and adds another; the resulting file has two consecutive license blocks. The new audit then passes because it sees its exact block at offset zero, so neither the gate nor the tests catch the duplication.
Please make application recognize/replace an existing standard ASF header (or otherwise prevent duplicate semantic headers), clean the duplicated .asf.yaml header, and add a regression covering this pre-existing whitespace variant. This matters beyond one file: a future valid formatting variant would otherwise be silently doubled again.
Merge readiness: not ready. The PR is draft, required CI is still running, independent licensing/mentor review is required, and the duplicate-header blocker must be resolved on a new exact head.
c6f3418 to
f1d7d83
Compare
|
Thanks — good catch, and the failure mode was worse than the one file: my own mechanical-purity check proved Fixed at
Two checks added because the original one was not enough: a scan of the pre-sweep tree confirming Agreed on merge readiness. The |
f1d7d83 to
0f97abe
Compare
M4n5ter
left a comment
There was a problem hiding this comment.
English
Reviewed exact head 0f97abe69fca7b14fab6e0e057e59ae1ded73f12.
The mechanical sweep is internally consistent, the focused 35 tests pass, and all four CI checks are green. However, I found four policy-level gaps that should be addressed before this becomes the repository's release gate.
1. [Blocking] Keep known mixed-origin files out of the mechanical ASF-header sweep
The extension fallback adds whole-file ASF headers to:
packages/runtime/src/tool-output.tspackages/runtime/src/edit-replace.tspackages/runtime/src/model-protocol.ts
The first two explicitly state that they adapt opencode code; #3325 records the pinned MIT sources and describes them as mixed-origin. #2907 similarly records the Vercel AI SDK material adapted in model-protocol.ts.
The ASF source-header policy defines third-party work to include parts of a file not submitted by their copyright owner, says not to mechanically add the standard ASF header to third-party source, and leaves major modifications to a case-by-case PMC decision.
Please classify these paths as third-party/mixed-source exclusions for now and preserve their upstream attribution. If the PPMC later chooses a combined header for a heavily modified file, that decision can be applied explicitly. Landing #3325 or #2907 later would restore attribution, but would not remove the whole-file header added here.
2. [Blocking] Audit every source-candidate entry instead of skipping build-looking directory names
scripts/asf-license-headers.mjs:254-267,410-427 silently skips every archive directory named dist, release, test-results, and several others. asf-source-release.mjs does not reject most of those names, so an extracted source candidate can contain unaudited source files.
A concrete extracted-tree repro with release/unexpected.kt reports:
Audited 0 covered and 0 excluded file(s)
Every source file carries the ASF header or a reviewed exclusion.
The comment claiming that verifySourceCandidate already rejects these directories is therefore incorrect.
Please remove the build-output skip from archive mode and classify every regular file. Unsupported entry types such as symlinks should be explicitly rejected or classified rather than silently omitted. Since the workflow performs this audit before npm ci, legitimate checkout build output does not need an archive-mode exception. An extracted-candidate fixture containing dist/unexpected.kt should fail as unclassified.
3. [Important] Replace directory-wide exclusions with reviewed inventories
Several exclusions use an entire directory as the policy boundary:
- everything under
apps/desktop/resources/licenses - everything under
apps/desktop/src/renderer/assets/provider-brands - everything under
packages/runtime/resources/bundled-skills
This means a future Maka-authored .ts or .md file placed in any of those directories silently inherits a third-party/verbatim justification. It also already misdescribes git/NOTICE.txt and git/SOURCE_OFFER.txt: those are Maka-authored legal prose, not byte-identical upstream texts verified by the preparation scripts.
This contradicts the gate's fail-closed objective. Please use exact reviewed inventories or structurally provable shapes—for example, exact bundled payload paths and only *.svg provider-brand assets—and add negative tests showing that an unexpected .ts/README.md inside each directory remains unclassified.
4. [Important] Fail closed on existing ASF headers the writer cannot safely canonicalize
The formatting-variant fix still leaves several forms of the original duplicate-header failure open:
- A TypeScript file using a valid leading
//ASF header is parsed only asblock, classified asabsent, and receives a second header. - A common
/** ... */rendering behaves the same way. - The ASF policy explicitly accepts
httpsin the license URL, but that variant is also classified asabsentand duplicated. - Conversely, the exact-header fast path returns
canonicalbefore inspecting the complete leading hash/slash comment, so a directly adjacent third-party copyright line bypasses theentangledsafeguard.
In each duplication case, the next audit passes because the newly inserted canonical header is at offset zero.
The simpler and safer boundary is: keep the audit byte-exact, but if any leading ASF signature is present and is not the isolated canonical rendering, refuse to mutate it and require manual resolution. After this one-time sweep, consider removing the variant-rewriting state machine—or the write mode entirely—instead of continually expanding a semantic license-comment parser.
Simplify audit
The same fixes can materially reduce the policy surface:
- Delete the second archive inclusion authority (
buildOutputDirectories). - Replace broad prefix exclusions with explicit reviewed inventories.
- Retire the one-shot variant-rewriting machinery after the initial sweep, leaving one small exact audit gate.
Verdict: request changes. The current tree appears mechanically sound, but the gate does not yet enforce the fail-closed licensing boundary described by the PR.
简体中文
已审查精确 head 0f97abe69fca7b14fab6e0e057e59ae1ded73f12。
机械 header sweep 内部一致,35 项定向测试通过,四项 CI 也全部通过。不过,我确认了四个 policy 层面的缺口;在它成为仓库 release gate 前应当修复。
1. [Blocking] 不要对已知 mixed-origin 文件机械添加 whole-file ASF header
扩展名 fallback 给以下文件添加了整个文件的 ASF header:
packages/runtime/src/tool-output.tspackages/runtime/src/edit-replace.tspackages/runtime/src/model-protocol.ts
前两个文件明确说明其改编自 opencode;#3325 固定了 MIT 来源,并将其描述为 mixed-origin。#2907 同样记录了 model-protocol.ts 改编的 Vercel AI SDK 材料。
ASF source-header policy 明确将文件内并非由版权所有者提交的部分也视为 third-party work,要求不要机械地在第三方源码顶部添加标准 ASF header,并要求 PMC 对重大修改逐案决定。
请暂时将这些路径分类为 third-party/mixed-source exclusion,并保留上游 attribution。如果 PPMC 后续决定某个重度修改文件使用组合 header,可以再显式实施。之后合入 #3325 或 #2907 只能补回 attribution,无法移除本 PR 已经添加的 whole-file header。
2. [Blocking] 审计 source candidate 中的每个 entry,不要按看似 build output 的目录名跳过
scripts/asf-license-headers.mjs:254-267,410-427 会静默跳过 archive 中所有名为 dist、release、test-results 等的目录。asf-source-release.mjs 并不会拒绝其中大多数目录,因此解压后的 source candidate 可以包含完全未被审计的源码。
在解压目录中放入 release/unexpected.kt 后,实际输出为:
Audited 0 covered and 0 excluded file(s)
Every source file carries the ASF header or a reviewed exclusion.
所以,代码中声称 verifySourceCandidate 已经拒绝这些目录的注释并不成立。
请移除 archive mode 的 build-output skip,并分类每个普通文件。symlink 等不支持的 entry type 也应明确拒绝或分类,而不是静默忽略。workflow 在 npm ci 前运行该审计,因此 archive mode 不需要为 checkout build output 提供例外。建议增加包含 dist/unexpected.kt 的 extracted-candidate fixture,并断言其因 unclassified 而失败。
3. [Important] 用经过审查的文件清单替代整个目录级别的 exclusion
当前多个 exclusion 把整个目录作为 policy 边界:
apps/desktop/resources/licenses下的所有内容apps/desktop/src/renderer/assets/provider-brands下的所有内容packages/runtime/resources/bundled-skills下的所有内容
这会导致未来放入这些目录的 Maka 自有 .ts 或 .md 文件静默继承 third-party/verbatim 理由。当前理由也已经错误描述了 git/NOTICE.txt 和 git/SOURCE_OFFER.txt:它们是 Maka 编写的法律说明,不是由 preparation scripts 校验 digest 的上游逐字节文本。
这与 gate 的 fail-closed 目标相矛盾。请使用精确、经过审查的文件清单,或者能由结构严格证明的规则——例如精确列出 bundled payload,只允许 provider-brand 的 *.svg——并增加反向测试,确保这些目录中新出现的 .ts/README.md 仍然会被判为 unclassified。
4. [Important] 对 writer 无法安全 canonicalize 的已有 ASF header 应当 fail closed
当前 formatting-variant 修复仍留下了几种原始重复 header 问题:
- TypeScript 文件若使用有效的
//ASF header,只会按blockstyle 解析,因此被判为absent并叠加第二份 header。 - 常见的
/** ... */格式有同样的问题。 - ASF policy 明确接受 license URL 使用
https,但该变体仍会被判为absent并重复添加。 - 反过来,exact-header 快速路径会在检查完整的 hash/slash leading comment 前直接返回
canonical,导致紧邻 header 的第三方 copyright 行绕过entangled保护。
在所有重复场景中,下一次 audit 都会通过,因为新添加的 canonical header 位于 offset zero。
更简单且安全的边界是:audit 保持 byte-exact;但只要发现 leading ASF 标志,而它不是独立的 canonical rendering,就拒绝自动修改并要求人工处理。完成这次一次性 sweep 后,建议删除 variant rewrite 状态机——甚至整个 write mode——而不是继续扩展一个语义化的 license-comment parser。
简化审计
上述修复也能明显缩小 policy surface:
- 删除第二套 archive inclusion authority,即
buildOutputDirectories。 - 用明确的 reviewed inventory 替代宽泛的 prefix exclusion。
- 初次 sweep 完成后收回一次性的 variant rewrite 机制,只保留一个小型、精确的 audit gate。
结论:request changes。当前 tree 的机械改动本身没有发现问题,但这个 gate 尚未真正实现 PR 所描述的 fail-closed licensing boundary。
c6f4637 to
c66bfcb
Compare
jackwener
left a comment
There was a problem hiding this comment.
Automated review of exact head c66bfcb5fd836904952fccc214dd17bd7820c20c against current main@d62857a8357e9160926726a2a13096bc2dc2b91d.
P1 — checkoutOnly exclusions still pass in archive mode. exclusionRules correctly marks .claude, .maka-shots, and maka-proposal-zh-review.txt as paths that are legitimate only in a checkout because .gitattributes must keep them out of a source archive. But classifyPath(path) ignores audit mode, and both checkout and archive enumeration call it unchanged. Therefore an extracted candidate that accidentally contains any of those paths is classified as a reviewed exclusion and passes, rather than proving export-ignore worked. That contradicts the rule's own contract and the stated “enumeration never subtracts / unexpected file fails” release guarantee.
Smallest fix: make classification mode-aware and do not allow a checkoutOnly rule to match in archive mode; add an extracted-tree fixture showing one of these paths is unclassified/failing. This is not theoretical: the whole reason field is “not in source release,” so accepting it in the archive defeats the decision being recorded.
The exact head also is not green: required test deterministically fails because the header-only sweep touches every protocol file while compatibility epoch remains unchanged. Do not bump the runtime epoch for comments; either teach the epoch guard to ignore provably header-only changes or otherwise reconcile the gate without inventing a wire change. The Windows package failure is a separate process-cleanup failure and may be transient, but it is still not a terminal-green gate.
Required conclusions:
- Optimal for the actual problem: not yet; the revised loose-detection/strict-acceptance design is materially better, but archive-only exclusion semantics are still fail-open.
- Production code to delete: none identified beyond avoiding any mode-blind duplicate classification path.
- Tests to delete/replace: none; add the missing archive-mode
checkoutOnlynegative case. - Deeper refactor: no; thread
modethrough the existing classifier/audit seam. - Ready to merge: no.
- Residual risks/gaps: exclusion/provenance judgments and release-policy equivalence require independent human/PPMC/mentor review; required CI must be green on the repaired exact head.
c66bfcb to
db3e9fe
Compare
|
@M4n5ter Thanks — all four reproduced locally, and all four were real. Fixed at They also turned out to be one root cause: the check consulted a proxy it could compute from a path instead of the authority that owns the fact. Patching instances is exactly what let the next round find more, so each fix below re-points a question at its authority. 1. Mixed-origin files. You're right, and this was the one with licensing consequences. All three are now excluded as third-party source, with the §3party reasoning and the #3325 / #2907 references recorded in the rule. The proxy was 2. Build-output skip. Reproduced exactly as you reported. Worse, the code comment claiming 3. Directory prefixes. Correct, including the specific error — 4. Duplicate headers. Correct on all four cases. Took your suggested boundary rather than extending the parser: detection is loose, acceptance stays byte-exact, and One more thing your review changed: the old mechanical-purity proof compared each file against |
|
@jackwener Both correct. Fixed at P1 — The checkout enumeration now asks Git: The epoch guard. Agreed on both halves — don't bump for comments, and don't invent a wire change. The guard's question is whether the protocol changed, and "a file under Against the actual sweep: The Windows Agreed on your residual risks: the exclusion and provenance judgements still need independent human review, and the RAT-equivalence argument needs a mentor. |
M4n5ter
left a comment
There was a problem hiding this comment.
English
Re-reviewed exact head db3e9fee8b4e5ede7d6e3007dabff304beba5f26.
The four findings from my previous review are resolved: mixed-origin files are excluded from the mechanical sweep, archive enumeration no longer skips build-looking directories or irregular entries, broad directory exclusions have been replaced with reviewed inventories or structural rules, and header detection is now loose while acceptance and mutation remain fail-closed.
I found one new issue in the protocol-epoch exemption.
[Important] Treat added and deleted protocol files as real changes instead of crashing
In scripts/protocol-epoch-check.mjs:114-123, isHeaderOnlyChange reads both revisions before entering its try block:
const before = git(['show', `${base}:${file}`], exec);
const after = git(['show', `${head}:${file}`], exec);
try {
return applyHeader(before, style) === after;
}changedProtocolFilesBetween uses --no-renames --name-only, so an added file has no ${base}:${file}, a deleted file has no ${head}:${file}, and a rename is represented as one of each. The corresponding git show therefore throws before the conservative fallback can run.
I reproduced both cases with a real Git fixture:
packages/runtime-host/src/protocol/added.ts THREW:
Command failed: git show HEAD^:packages/runtime-host/src/protocol/added.ts
packages/runtime-host/src/protocol/removed.ts THREW:
Command failed: git show HEAD:packages/runtime-host/src/protocol/removed.ts
This makes a legitimate protocol addition, deletion, or rename fail with an internal guard error even when the compatibility epoch was correctly bumped.
Please treat a file missing at either revision as not header-only—for example, include both reads in the conservative try/catch and return false on failure—and extend the Git fixture with added and deleted paths. Renames are already covered by the --no-renames behavior.
Everything else in the updated scope looks sound. The focused 52 tests pass, the checkout audit reports 2,625 covered and 117 reviewed exclusions, the sweep remains mechanically clean, and all four exact-head CI checks are green.
Non-blocking documentation correction: the PR description says all 2,623 files in the second commit have zero removed lines. DISCLAIMER-WIP intentionally removes one line and .git-blame-ignore-revs is also a non-header edit, so the mechanical-proof wording should exclude those two files.
Verdict: request changes for the protocol add/delete boundary.
简体中文
已复审 exact head db3e9fee8b4e5ede7d6e3007dabff304beba5f26。
我上次 review 提出的四个问题均已解决:mixed-origin 文件已排除在机械 sweep 之外;archive enumeration 不再跳过看似 build output 的目录或 irregular entry;宽泛的目录 exclusion 已替换为经过审查的清单或结构规则;header detection 保持宽松,而 acceptance 与 mutation 现在会 fail closed。
不过,protocol epoch 例外中出现了一个新问题。
[Important] 新增和删除的 protocol 文件应当视为真实变更,而不是让 guard 崩溃
在 scripts/protocol-epoch-check.mjs:114-123 中,isHeaderOnlyChange 在进入 try 之前读取两个 revision:
const before = git(['show', `${base}:${file}`], exec);
const after = git(['show', `${head}:${file}`], exec);
try {
return applyHeader(before, style) === after;
}changedProtocolFilesBetween 使用 --no-renames --name-only,所以新增文件没有 ${base}:${file},删除文件没有 ${head}:${file},rename 则会被表示为一次新增和一次删除。对应的 git show 会在保守 fallback 生效前直接抛错。
我使用真实 Git fixture 复现了两种情况:
packages/runtime-host/src/protocol/added.ts THREW:
Command failed: git show HEAD^:packages/runtime-host/src/protocol/added.ts
packages/runtime-host/src/protocol/removed.ts THREW:
Command failed: git show HEAD:packages/runtime-host/src/protocol/removed.ts
这会导致正常的 protocol 新增、删除或 rename 即使已经正确 bump compatibility epoch,也仍然以 guard 内部错误失败。
请将任一 revision 中不存在的文件视为“不是 header-only”——例如把两次读取一并放入保守的 try/catch,失败时返回 false——并在 Git fixture 中覆盖新增和删除路径。由于使用了 --no-renames,rename 会自然得到覆盖。
更新范围内的其他部分没有发现问题。52 项定向测试通过;checkout audit 报告 2,625 个 covered 文件和 117 个 reviewed exclusion;sweep 保持机械一致;exact-head 的四项 CI 全部通过。
另有一项非阻塞的文档修正:PR 描述称第二个 commit 的全部 2,623 个文件都没有删除行,但 DISCLAIMER-WIP 有一行有意删除,.git-blame-ignore-revs 也不是 header edit。建议在 mechanical proof 的表述中排除这两个文件。
结论:针对 protocol 新增/删除边界 request changes。
db3e9fe to
382cc7c
Compare
|
@M4n5ter Correct, and it's a real defect — thanks. Fixed at Both Added a git-fixture test covering an added file, a deleted file, and both halves of a rename — with Thanks also for re-checking the earlier four. |
M4n5ter
left a comment
There was a problem hiding this comment.
English
Re-reviewed exact head 382cc7ca398379040bdc9bc0f904f656f307839b.
The protocol add/delete/rename finding is resolved. Both revision reads now sit inside the conservative guard, so a path that exists on only one side is treated as a real protocol change rather than escaping with an internal error. The Git fixture covers added, deleted, and both rename paths, and I additionally verified the combined path: those files remain in the changed set and a correct epoch bump passes normally.
The rebase added six Maka-authored desktop source files; all are covered by the header audit and do not reopen the mixed-origin boundary.
No new blocking issue found. The focused 53 tests pass, the checkout audit reports 2,631 covered and 117 reviewed exclusions, and the sweep-level epoch check passes. The exact-head dependency audit and Windows W0 protocol check are green; the core test and Windows package checks are still running.
One remaining non-blocking documentation correction: the mechanical-proof paragraph still says all 2,629 files in the second commit have zero removed lines. DISCLAIMER-WIP intentionally removes one line, while .git-blame-ignore-revs is also separate from the 2,627 header insertions. Please narrow that claim accordingly.
LGTM.
简体中文
已复审 exact head 382cc7ca398379040bdc9bc0f904f656f307839b。
protocol 新增、删除和 rename 的问题已经解决。两个 revision 的读取现在都位于保守 guard 内,因此只存在于一侧的路径会被视为真实 protocol change,而不会以内部错误逃出。Git fixture 覆盖了新增、删除以及 rename 的两侧;我还额外验证了组合路径:这些文件会保留在 changed set 中,并且正确 bump epoch 后能够正常通过。
本次 rebase 新增了 6 个 Maka 自有的 desktop 源文件;它们均已进入 header audit,没有重新打开 mixed-origin 边界。
没有发现新的阻塞问题。53 项定向测试通过;checkout audit 报告 2,631 个 covered 文件和 117 个 reviewed exclusion;sweep 层面的 epoch check 通过。exact-head 的依赖审计和 Windows W0 protocol check 已通过;core test 和 Windows package check 仍在运行。
仍有一项非阻塞的文档修正:mechanical proof 段落仍称第二个 commit 的全部 2,629 个文件都没有删除行。DISCLAIMER-WIP 有一行有意删除,而 .git-blame-ignore-revs 也不属于 2,627 个 header insertion。请相应收窄该表述。
LGTM。
|
@jackwener Following up — both points from your review are resolved, and the head has moved twice since: P1 — the CI — green at this head: Since then @M4n5ter re-reviewed and found one more defect in that epoch exemption — added, deleted, and renamed protocol files threw out of Your |
The first Apache Maka (Incubating) source release has to apply the ASF header policy and enforce it, and nothing in the repository decided which files the policy covers or checked that they comply. `scripts/asf-license-headers.mjs` is that decision. It asserts two properties of the tree it is pointed at, and nothing else. Every entry is classified exactly once: covered by a comment syntax and required to carry the ASF header, or matched by one reviewed exclusion rule that records why the header does not belong there. And in every covered file the license text occurs exactly once, as the canonical rendering at the top. A file that is neither covered nor excluded fails the audit, so a new file type or an unexpected path cannot reach a release candidate until someone writes down which of the two it is. Both properties are stated about the artifact rather than about what the writer emits. An audit that asks "does this file start with the bytes I would write?" can only confirm its own writer: it treats a header rendering it does not recognize as absent, writes a second license block above it, and then accepts the result, because the block it just wrote is at the top of the file. So detection is deliberately loose and acceptance deliberately strict. Detection flattens comment punctuation and line wrapping away and finds the license text written with `//`, in a JSDoc block, with `#`, inside an HTML comment, at any indentation, with an `https` license URL, or rewrapped into a paragraph. Acceptance takes only the exact canonical rendering. `write` is correspondingly dumb: it inserts a header where there is none and refuses every other state, because reconciling someone else's rendering is a decision, not a rewrite. Nothing is dropped from the tree under audit. An extracted candidate is walked whole, with no directory skipped by name, because a file inside a `dist/` or `release/` directory in a source release is not a tree to audit around — it is unclassified, and the gate says so. An entry that is neither a directory nor a regular file is reported rather than passed over. What the enumeration does decide is which tree a checkout stands in for: the files a source release would carry, which means subtracting what `export-ignore` prunes. `.gitattributes` owns that fact, so the audit asks Git for it rather than restating it in a rule. A copy would answer for the original in the one case that matters — an extracted candidate containing such a path would be excused by the very reason that says it cannot be there. Asking Git also has to walk the ancestors, since `export-ignore` on a directory prunes the subtree while `git check-attr` reports one path at a time and does not inherit. The two modes now audit the same set of files. An exclusion rule names the files it excludes or states a structural property that holds for anything it matches. A bare directory prefix does neither: it lends its justification to whatever is added to that directory next. The rules cover the ASF release documents themselves, verbatim upstream license texts, the notices Maka publishes about a bundled third-party runtime, third-party work kept under its own license, generated files that a `check:` script byte-compares against a fresh run, payloads republished verbatim as prompt text or as a pull request body, byte-significant fixtures, formats with no comment syntax, binaries, and version-control metadata. ASF policy treats the part of a file that the copyright owner did not contribute as third-party work, so the files Maka adapted from opencode and from the Vercel AI SDK are excluded rather than given a whole-file header. Whether a heavily modified file should carry a combined header is a PPMC decision. Their attribution belongs to LICENSE and the NOTICE audit. The audit cannot determine provenance — that is a fact about people, not a property of a path — so it refuses to let an unreviewed claim of one pass. A covered file carrying an SPDX identifier, an explicit copyright line, or an adaptation notice fails unless it is recorded in `reviewedProvenance` with the decision that put an ASF header on it anyway. The markers were chosen by measuring them against the whole tree: those three hit three files between them, while `derived from` hits seventy and `opencode` hits forty-five, because Maka supports it as a provider. A net that noisy gets answered with a list of paths added to quiet it. This is a net and not a proof: it catches a file that states its origin, not prose that alludes to one. `scripts/protocol-epoch-check.mjs` gains one exemption. It requires an epoch bump whenever a file under the protocol directory changes, a conservative proxy for "the protocol changed" that is right to be conservative: a needless bump costs a number, a missed one ships two incompatible protocols under one. But inserting a license header provably does not change the protocol, and bumping for it would tell every peer the wire is incompatible over a comment. The guard now exempts a change that is exactly `applyHeader` applied to the previous content — the same authority that writes the headers, so the exemption covers the canonical insertion and nothing that merely resembles it. A protocol file that gained a header and a real edit fails that comparison and still needs an epoch. `Prepare ASF source candidate` runs the audit inside the extracted archive before anything installs or builds into that tree, so the release gate reads the exact bytes a voter downloads instead of a checkout that resembles them. CI runs it unconditionally beside the other install-free checks, because a header gate that only sees the affected surfaces is not a gate. Apache RAT was the alternative. It would put a JVM and a downloaded jar into the audit path that reviewers are expected to reproduce, and its glob exclusions fail open: a new unexpected file inside an excluded directory is approved silently. This audit needs no toolchain beyond the Node the archive already requires, and an unexpected file fails it. The headers themselves are not in this commit. Until the sweep lands, the audit reports every existing source file as missing one. Refs #3271 Generated-by: Claude Code
`npm run write:asf-headers` produced this commit, and `npm run check:asf-headers` now passes on the checkout and on the extracted source archive. Nothing here was hand-edited: every changed file is exactly its previous content with the standard ASF header inserted at the top, below an interpreter shebang, an HTML doctype, or Markdown front matter where one of those has to open the file. `.asf.yaml`, which already carried the header, is untouched. `DISCLAIMER-WIP` no longer discloses that source files lack Apache license headers, because they no longer do. The remaining disclosures about `NOTICE` and about the software grant and ICLAs are unchanged and still open. `.git-blame-ignore-revs` gets an entry for this sweep. Following the convention the file documents, the placeholder records the landed hash in a follow-up right after the squash-merge, since a hash that never reaches main makes `git blame` fail outright. Merging this will require open pull requests to rebase. That is the cost of doing the sweep once; splitting it per directory would multiply the rebases instead of avoiding them. Refs #3271 Generated-by: Claude Code
382cc7c to
ad0c431
Compare
Summary
Applies the ASF source-header policy and enforces it with an automated audit of the source release candidate.
scripts/asf-license-headers.mjsasserts two properties of the tree it is pointed at, and nothing else:Unclassified is the design. A new file extension or an unexpected path cannot reach a release candidate until someone writes down which of the two it is, so the exclusion list stays a list of reasons rather than a list of paths that happened to be noisy.
.github/ASF_SOURCE_HEADERS.mddocuments the policy;npm run check:asf-headers -- --reportprints the resolved file list behind every rule.One rule: ask the authority, don't restate it
Every question this gate answers has somewhere that already owns the answer. Where the code substituted a proxy it could compute from a path, it was wrong — and each round of review found another instance of the same substitution. The design now points each question at its own authority:
.gitattributesgit check-attr export-ignore, walking ancestorsLICENSE, #3325, #2907protocol/touched, minus provable header-only insertionsNothing in the audit is defined in terms of what the writer emits. An audit that asks "does this file start with the bytes I would write?" can only confirm its own writer: it treats a header rendering it does not recognize as absent, writes a second license block above it, then accepts the result, because the block it just wrote is now at the top of the file.
Detection is loose, acceptance is strict
//, in a JSDoc block, with#, inside an HTML comment, at any indentation, with anhttpslicense URL, or rewrapped into one paragraph.A covered file is therefore
absent,canonical,unrecognized, orduplicated, and onlyabsentis a state a program may act on.npm run write:asf-headersinserts a header where there is none and refuses every other state, because reconciling someone else's rendering is a decision, not a rewrite.Nothing is dropped from the tree under audit
An extracted candidate is walked whole, with no directory skipped by name. A file inside a
dist/orrelease/directory in a source release is not a tree to audit around — it is unclassified, and the gate says so. Nor is a pathexport-ignoreshould have pruned: finding one means it did not, so the audit reports it instead of excusing it with the reason that says it cannot be there. An entry that is neither a directory nor a regular file is reported rather than passed over.What the enumeration does decide is which tree a checkout stands in for: the files a source release would carry. Both modes now audit the same 2,642 covered and 117 excluded files.
Why not Apache RAT
The issue allows "Apache RAT or an equivalent mentor-approved audit". This PR implements the equivalent, for three reasons:
apache-maka-<version>-incubating-src.tar.gz. RAT would put a JVM and a downloaded jar into the path a voter is expected to reproduce; this audit needs nothing beyond the Node the archive already requires.scripts/*.mjsregenerate-and-diff checks wired intocheck:npm scripts and run from the extracted candidate. This extends that seam.Mentors who prefer literal RAT output should say so on the issue — the policy data would carry over to a
rat-excludesfile, but the fail-closed property would not.Reviewed exclusions
A rule names the files it excludes, or states a structural property that holds for anything it matches. A bare directory prefix does neither: it lends its justification to whatever is added to that directory next.
asf-release-documentsLICENSE,NOTICE,DISCLAIMER-WIPare the license and notice themselvesthird-party-license-textsmaka-authored-legal-noticesthird-party-sourcegenerated-filesverbatim-runtime-payloadsverbatim-github-templatesbyte-significant-fixturesno-comment-syntaxbinary-filesno-creative-contentThere is deliberately no rule for paths
export-ignoreprunes. Third-party license classification stays with #3270 / G3; this gate only decides where an ASF header may be asserted.Mixed-origin source
ASF policy treats the portion of a file the copyright owner did not contribute as third-party work.
tool-output.tsandedit-replace.tsare adapted from opencode (#3325 pins the MIT provenance and states the notice is scoped to the adapted portion — no whole-file identifier), andmodel-protocol.tsfrom Vercel AI SDK material recorded by #2907. All three are excluded rather than given a whole-file ASF header. Whether a heavily modified file should instead carry a combined header is a PPMC decision, taken case by case.Unreviewed provenance
The gate cannot determine provenance — that is a fact about people, not a property of a path — so it refuses to let an unreviewed claim of one through. A covered file carrying an SPDX identifier, an explicit copyright line, or an adaptation notice fails unless it is recorded in
reviewedProvenancewith the decision that put an ASF header on it anyway.The markers were chosen by measuring them against the whole tree: those three hit five files, while
derived fromhits seventy andopencodehits forty-five, because Maka supports it as a provider. A net that noisy gets answered with a list of paths added to quiet it. This is a net, not a proof — it catches a file that states its origin, not prose that alludes to one.It has already fired twice on files it had never seen. Rebasing onto current
mainbrought indocs/code-origin-audit.mdfrom #2907 andscripts/sync-model-metadata.mjs, and each time the audit stopped and asked for a decision — the first quotes upstream copyright lines as its findings, the second writes the models.dev attribution into the catalog it generates. Both are recorded with the reason they may carry an ASF header anyway.The protocol compatibility epoch
scripts/protocol-epoch-check.mjs(#3339) requires an epoch bump whenever a file under the Runtime Host protocol directory changes. That proxy is right to be conservative — a needless bump costs a number, a missed one ships two incompatible protocols under one — but inserting a license header provably does not change the protocol, and bumping for it would tell every peer the wire is incompatible over a comment.The guard now exempts a change that is exactly
applyHeaderapplied to the previous content, using the same authority that writes the headers. Everything else is a protocol change: a file that gained a header and a real edit fails the comparison, and a file that exists on only one side — added, deleted, or one half of a rename, since the diff is taken with--no-renames— has no pair to compare and is one by definition. Every revision read sits inside the guard for that reason, so the exemption fails toward requiring an epoch rather than toward crashing the check that would have demanded one. Git-fixture tests cover both.Two commits, on purpose
13260da— policy, audit, CI wiring, release gate, and the epoch-guard exemption. On its own the audit reports every existing source file as missing a header, rather than passing silently.ad0c431— the sweep: 2,638 files, purely mechanical..git-blame-ignore-revsrecords the sweep. Following the convention that file documents, this PR adds the placeholder entry and the landed squash hash goes in a follow-up immediately after merge — a hash that never reachesmainmakesgit blamefail outright.Merging this will require open pull requests to rebase. With ~60 open PRs that is a real cost, but it is paid once; splitting the sweep by directory would multiply the rebases rather than avoid them.
DISCLAIMER-WIPno longer discloses that source files lack Apache license headers. TheNOTICEand software-grant/ICLA disclosures are unchanged and still open.Refs #3271
Exit conditions satisfied
scripts/asf-license-headers.mjsand.github/ASF_SOURCE_HEADERS.md..asf.yamlalready carried the header and is untouched.Prepare ASF source candidateruns it insidecandidate-sourcebeforenpm ci.DISCLAIMER-WIPwhen its missing-header disclosure is no longer true..git-blame-ignore-revs— placeholder added; the hash follows the squash-merge per the file's own convention.Verification
Run in a clean worktree at
ad0c431, rebased onto7994a4d. All four required checks are green at that exact head.Against the real extracted archive, which is what the gate actually does — built
apache-maka-0.1.11-incubating-src.tar.gz, extracted it, and ran the audit inside it:release/unexpected.ktadded.claude/skills/local/SKILL.mdandmaka-proposal-zh-review.txtaddedsettings.tsEvery one of those negative cases is a failure a reviewer found in an earlier revision of this PR.
The epoch guard, run against the actual sweep:
The sweep is mechanical, proved without calling the writer. An earlier proof compared each file against
applyHeader(parent)— same assumptions as the code it was checking, and blind to the bug that got through. This one is a text-level property: for each of the 2,640 changed files,git diff --unified=0contains exactly one hunk and zero removed lines, so the change is a single contiguous insertion. Zero files fail that. One file (foreign-session.test.ts) contains a NUL byte and Git treats it as binary, so it was checked bytewise instead:child === header + parent.Property 2 asserted over the whole tree by text search, not by the audit: exactly one file contains the license text more than once —
scripts/asf-license-headers.mjs, which defineslicenseLines, and which is the sole entry inlicenseTextAsDatawith that reason recorded.Other checks:
npm run check:asf-headers— 2,642 covered, 117 excluded, nothing unclassified, unrecognized, duplicated, or unreviewed. Checkout and archive report identical counts.npm run check:asf-source— 45 pass, including tests over an actual extracted-tree fixture (a file indist/, anexport-ignored path, a symlink, a stacked header, an unreviewed copyright line).node --test scripts/protocol-epoch-check.test.mjs— 10 pass, including git-fixture cases for the header-only exemption and for added, deleted, and renamed protocol files.npm run format:check,npm run build,npm run typecheck— pass.npm run check:stale,npm run astryx:theme -- --check,npm run astryx:surface-inventory,npm run windows:inventory,npm run check:third-party-notices,npm run check:cli-third-party-notices,npm run check:windows-cargo-notices— pass.Review focus
The exclusion list and the provenance decisions are the parts that need human judgement, not the header insertion.
npm run check:asf-headers -- --reportprints every rule with its justification and the exact files it claims — one screen, and the part of this PR where being wrong has release consequences.The sweep does not need to be read file by file. Its claim is that every changed file is its parent plus one inserted block, and that claim is checkable from the diff alone: one hunk, no deletions.
Addressed from review
Two automated reviews found seven issues between them. All seven reproduced locally and all seven were real. Six were instances of one root cause: the check consulted a proxy it could compute instead of the authority that owns the fact. Fixing instances is what let the next round find more, so each fix below re-points a question at its authority rather than patching the symptom. The seventh was a plain coding defect in the fix for the fifth.
@M4n5ter at
0f97abe:classifyPath(path)itself — provenance is not a function of a path — soreviewedProvenancenow makes an in-band origin claim mechanical rather than remembered.dist,release,test-resultsby name. Correct, reproduced exactly as reported, and the code comment claimingverifySourceCandidaterejected those directories was wrong —forbiddenSegmentscovers only.agents,.claude,.git,.maka-shots,node_modules. The skip list is deleted; nothing replaces it. It was also dead in checkout mode, since all six directories are gitignored.git/NOTICE.txtandgit/SOURCE_OFFER.txtare Maka-authored legal notices, not digest-verified upstream texts, and they now have their own rule. The three prefix rules are replaced by explicit inventories and one structural rule (provider-brands/*.svg), with a test asserting that a new.tsorREADME.mdin any of those directories stays unclassified.//header in a.tsfile, a JSDoc block, anhttpslicense URL, and the exact-match fast path bypassing the entangled check. Rather than extend the variant parser, the fix follows the suggestion in the review: detection is loose, acceptance stays byte-exact,writerefuses anything notabsent.classifyExistingHeader,normalizeCommentText,leadingComment, andEntangledHeaderErrorare deleted.@jackwener at
c66bfcb:checkoutOnlyexclusions still passed in archive mode. Correct, and the sharpest statement of the root cause in this PR: the rule's own justification was ".gitattributeskeeps this out of the archive", so accepting it in the archive used the reason as cover for the one thing it says cannot happen. Rather than make classification mode-aware, the rule is deleted. The checkout enumeration asks Git directly —git check-attr export-ignore, walking ancestors, since a directory attribute prunes the subtree butcheck-attrdoes not inherit. That removes the duplicated fact, thecheckoutOnlyflag, and the concept of a rule that holds in one mode and not the other. An extracted-tree fixture asserts such a path now fails.testwas red. Correct — and bumping the epoch was the wrong answer, as the review said. Fixed in the guard instead; see The protocol compatibility epoch above. The Windowspackagejob was a pre-existingmainbreakage fixed by fix(desktop): drain untracked Runtime Host before update #3382, which has since landed; all four checks are green atad0c431.@M4n5ter's re-review at
db3e9feconfirmed those four resolved and found a seventh issue, in the epoch exemption itself:isHeaderOnlyChangeread both revisions before entering its guard, so an added, deleted, or renamed protocol file threw out ofmain()instead of falling back conservatively. Correct and mine. Both reads are now inside the guard — a file with no pair to compare is a protocol change by definition — with git-fixture coverage confirmed to fail against the previous code.@jackwener's first automated review at
d3e0eeffound the original instance of #4:.asf.yamlalready carried an ASF header whose license-URL indentation differed from the rendering, so the byte-exact test called it absent and the sweep stacked a second block on it. The fix then was a semantic recognizer for that one variant, which is why the four cases above survived it. The canonical rendering now uses the indentation.asf.yamlalready used, so that file is untouched.AI use
Select exactly one:
Tool(s) and scope: Claude Code (Opus) drafted the policy module, its tests, the CI and release-workflow wiring, the epoch-guard exemption, the policy document, and ran the sweep and the verification above. Both commits carry
Generated-by: Claude Code. The exclusion rules, the provenance decisions, and their justifications are AI-drafted and need a human contributor of record to confirm them; licensing conclusions remain with the PPMC, mentors, and IPMC.Checklist
Does this PR entail a change in behavior?