Allow to use short SPDX licence identifier for selected files

[potiuk]

While we have some discussion on-going whether we should use some shorter, machine-readable friendly versions of licence specification in our source code headers here [1], the notion is that:

a) PMC can make judgment calls when to include different versions of
the licence

b) This expectation only applies to the code we actually release
in our official releases.

This change makes some judgment call on using much shorter, SPDX driven licence headers in some specific files:

• markdown files that are intended to be consumed by agents (AGENTS.md, SKILLS.md, CLAUDE.md and so on)

• all the markdown .github/* files that are clearly meta-data for GitHub and which we exclude from released sources

We also make sure all those files are excluded from the official source releases and distribution packages we prepare.

[1] https://lists.apache.org/thread/j1tn63r2lf13v3d1tnnqff8fkcl4nx53

---

Was generative AI tooling used to co-author this PR?

• Yes (please specify the tool below)

---

• Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
• For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
• When adding dependency, check compliance with the ASF 3rd Party License Policy.
• For significant user-facing changes create newsfragment: {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

[Dev-iL]

I thank you in the name of all agents and their users!

[github-actions]

Backport successfully created: v3-1-test

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

Status	Branch	Result
✅	v3-1-test

[zvr]

@potiuk @jscheffl Please, please, don't use an incompatible format!

The line

<!-- SPDX-License-Identifier: Apache-2.0 https://www.apache.org/licenses/LICENSE-2.0 -->

will break every license parser in existence.

After the SPDX-License-Identifier:, an SPDX license expression is expected. See the guidelines, which in SPDXv2 used to be part of the specification that is an ISO standard here.

If you need to put the license URL, please put it on a separate line.

Personally, I don't believe it's needed, since SPDX-License-Identifier: Apache-2.0 automatically means https://spdx.org/licenses/Apache-2.0.

But in any case, don't add it to the same line!

[bugraoz93]

@potiuk @jscheffl Please, please, don't use an incompatible format!

The line

<!-- SPDX-License-Identifier: Apache-2.0 https://www.apache.org/licenses/LICENSE-2.0 -->

will break every license parser in existence.

After the SPDX-License-Identifier:, an SPDX license expression is expected. See the guidelines, which in SPDXv2 used to be part of the specification that is an ISO standard here.

If you need to put the license URL, please put it on a separate line.

Personally, I don't believe it's needed, since SPDX-License-Identifier: Apache-2.0 automatically means https://spdx.org/licenses/Apache-2.0.

But in any case, don't add it to the same line!

We should be able to use Representing multiple licenses and it looks like fit to format 🤔

[zvr]

I am not sure what you mean by "it looks like fit to format".

Whatever comes after a SPDX-License-Identifier: is an SPDX License Expression, which has a very well-defined format. The grammar is defined in an annex of the specification, which defines the operators (AND, OR, WITH) and the terminals (licenses from the SPDX License List or custom licenses).

The string Apache-2.0 https://www.apache.org/licenses/LICENSE-2.0 cannot be recognized as an expression: it has a license identifier (Apache-2.0), a space, and a URL. This does not follow the grammar and cannot be parsed.

The URL must be placed on a separate line.

[potiuk]

I am not sure what you mean by "it looks like fit to format".

Whatever comes after a SPDX-License-Identifier: is an SPDX License Expression, which has a very well-defined format. The grammar is defined in an annex of the specification, which defines the operators (AND, OR, WITH) and the terminals (licenses from the SPDX License List or custom licenses).

The string Apache-2.0 https://www.apache.org/licenses/LICENSE-2.0 cannot be recognized as an expression: it has a license identifier (Apache-2.0), a space, and a URL. This does not follow the grammar and cannot be parsed.

The URL must be placed on a separate line.

Good point. Moving it to two lines is better. Thanks @zvr -> that's a great feedback.

[potiuk]

PR here #62145

[potiuk]

Let us know @zvr if the proposed change in #62145 is good. I want it to be as concise as possible, so the line with licence starts with the comment <!-- but I believe it should be fine. The annex you mentioned only describes what happens after SPDX specification and refers to BNF used by parsers, which I think should be ok, i can't imagine an expectation that the SPDX specification token starts at the beginning of the line - I think having it started after the opening comment token without EOL after it is perfectly fine for parsers?

[zvr]

No worries, almost license scanning tools know to ignore initial comment markers like /*, #, //, ...