Repository navigation
Conversation
Release documentation published under /versions/<version>/ currently declares every page as its own canonical URL and ships its own sitemap.xml, so search engines see each versioned copy as a competing page. Only the development docs at the site root should be indexed. Clear html_baseurl in the release overlay so Sphinx emits no <link rel="canonical"> tag, and drop sphinx-sitemap from release builds. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
github-actions
Bot
added
documentation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
Rationale for this change
Release documentation published under
/versions/<version>/is built with the overlay indocs/scripts/release/conf.py, which setshtml_baseurlto the versioned prefix. Sphinx uses that setting to emit a<link rel="canonical">tag on every page, so today each versioned copy of a page declares itself canonical and competes in search results with the development docs at the site root (for example, compare https://datafusion.apache.org/user-guide/configs.html and https://datafusion.apache.org/versions/55.1.0/user-guide/configs.html). Each versioned tree also ships its ownsitemap.xml.Only the development documentation at the site root should be indexed by search engines.
What changes are included in this PR?
html_baseurlin the release overlay so release builds emit no<link rel="canonical">tagsphinx_sitemapfrom release builds so no per-versionsitemap.xmlis written (the extension requireshtml_baseurl, and the rootrobots.txtonly references the root sitemap anyway)--with sphinx-sitemapnote from the release instructionsThe already published trees for 55.0.0 and 55.1.0 keep their current tags until they are rebuilt with the overlay.
Are these changes tested?
Built the
55.1.0tag with the modified overlay (sphinx-build -c docs/scripts/release) and confirmed the output has norel="canonical"link on any page, nositemap.xml, and still has the version switcher configured withversion_matchof55.1.0.Are there any user-facing changes?
No. The rendered documentation looks the same; only the
<head>metadata and the generated sitemap of versioned builds change.