add(agent-readiness): TDM reservation (TDMRep) + ship tdm-reservation: 0 - #132
Merged
Conversation
New page documenting the W3C TDM Reservation Protocol — the HTTP header, the <meta> form, /.well-known/tdmrep.json, and their precedence. Framed around the misconception it invites: this is not a crawler blocker. robots.txt governs fetching; tdm-reservation governs mining after the fetch, and its force is legal (Article 4 CDSM) rather than technical. Ships the implementation in the same PR: tdm-reservation: 0 on every response, the machine-readable counterpart to CC BY 4.0. Status is optional — technical uptake is thin and the page says so. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Deploying specification-website with
|
| Latest commit: |
e207b86
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://9c7c6574.specification-website.pages.dev |
| Branch Preview URL: | https://add-tdmrep-2026-07-28.specification-website.pages.dev |
# Conflicts: # public/.well-known/agent-skills/index.json # src/content/spec/agent-readiness/content-signals.md
# Conflicts: # public/.well-known/agent-skills/index.json
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
agent-readiness/tdmrep, statusoptional, order 43 (betweenrobots-for-ai-crawlersandweb-bot-auth).tdm-reservation: 0added to the/*block inpublic/_headers, with a comment explaining why it stays at0.relatedSlugswired oncontent-signals,robots-for-ai-crawlers,agent-readiness-overview.Who actually consumes this — the question that shaped the page
Worth stating up front, because it decides the status:
Technical uptake is thin. Spawning AI reads TDMRep in the API it runs for model trainers; Common Crawl measures its prevalence when surveying opt-out signals. No major crawler operator has published a commitment to honour it the way they publish commitments about
robots.txt.The real consumer is a legal system. Article 4 of the EU CDSM Directive permits text and data mining unless the rightsholder reserved the right in a machine-readable way — so an unexpressed reservation is no reservation. On the other side, GPAI providers must have a policy for identifying and respecting those reservations, and in December 2025 the Commission opened a consultation to agree which protocols qualify, with TDMRep among the named candidates.
The page says both of these plainly rather than implying enforcement that does not exist. Hence
optional, notrecommended.The misconception it names
TDMRep looks like another crawler blocker and is not one.
robots.txtanswers may you fetch this?;tdm-reservationanswers having fetched it, may you mine it? A crawler honouringDisallow:never sees the content; a crawler honouringtdm-reservation: 1reads the page happily and is then on notice. One is a gate, the other is a notice — and shipping TDMRep blocks nobody.Why we send
0The content is CC BY 4.0 and the entire point of this site is being read by machines.
tdm-reservation: 0states that in the same machine-readable form a refusal would take, rather than leaving it inferred from a licence file. It also makes the worked example demonstrate the grant direction, which is the less-published half of this convention.We ship the header only, not
/.well-known/tdmrep.json— the header covers non-HTML assets too, and the file is the lowest-precedence mechanism of the three. No api-catalog orLinkheader change: nothing new is discoverable at a URL, andtdm-reservationhas no registered IANA link relation.Verification
npm run buildpasses. The header is present indist/_headers; it cannot be exercised locally because_headersis interpreted by Cloudflare Pages, not the preview server — worth acurl -sIagainst the deploy preview before this leaves draft.🤖 Generated with Claude Code