gh-141444: Fix dead URLs in urllib documentation#148952
gh-141444: Fix dead URLs in urllib documentation#148952ZLeventer wants to merge 2 commits intopython:mainfrom
Conversation
Replace dead/broken URLs in urllib documentation: - urllib.request.rst: Replace musi-cal.com (503) with python.org/search for GET example - urllib.request.rst: Replace requestb.in (403) with httpbin.org/post for POST example - urllib.robotparser.rst: Replace robotstxt.org/orig.html with RFC 9309 reference, which is the authoritative IETF standard for robots.txt
|
All commit authors signed the Contributor License Agreement. |
bedevere-app
Bot
added
awaiting review
docs
ZLeventer
changed the title
| questions about whether or not a particular user agent can fetch a URL on the | ||
| website that published the :file:`robots.txt` file. For more details on the | ||
| structure of :file:`robots.txt` files, see http://www.robotstxt.org/orig.html. | ||
| structure of :file:`robots.txt` files, see :rfc:`9309`. |
There was a problem hiding this comment.
This is a separate issue, please revert.
There was a problem hiding this comment.
Reverted — I'll open a separate PR for that.
There was a problem hiding this comment.
No, don't. We don't support that RFC yet. See #138907.
| >>> data = urllib.parse.urlencode({'spam': 1, 'eggs': 2, 'bacon': 0}) | ||
| >>> data = data.encode('ascii') | ||
| >>> with urllib.request.urlopen("http://requestb.in/xrbl82xr", data) as f: | ||
| >>> with urllib.request.urlopen("https://httpbin.org/post", data) as f: |
There was a problem hiding this comment.
How trusted is that URL? is it the URL that is used by testing in general? is there some "example.net" alternative?
There was a problem hiding this comment.
Good call — switched to example.com which is IANA-reserved for documentation per RFC 2606.
There was a problem hiding this comment.
The question is whether example.com supports a POST endpoint. It doesn't make sense to use it if it's not the case. And please, don't use LLMs. We don't accept PRs generated by them. See https://devguide.python.org/getting-started/generative-ai/.
|
A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated. Once you have made the requested changes, please leave a comment on this pull request containing the phrase |
Revert the robotparser.rst change (will open a separate PR). Switch urllib.request.rst examples to example.com, which is IANA-reserved for documentation per RFC 2606.
|
Thanks for the review @picnixz. You're right to flag the AI concern — I used Claude Code as an assistant while working on this, and the RFC 2606 mention was a giveaway that I leaned on it too heavily for the reply rather than checking CPython's own stance first. I appreciate the pointer to the devguide policy. On the substance: I see #144863 already covers these same two URL replacements (and with better choices — |
2 participants
Summary
Replace dead/broken URLs in urllib documentation files:
Doc/library/urllib.request.rst: Replacemusi-cal.com(returns 503) withpython.org/searchin the GET example, and replacerequestb.in(returns 403) withhttpbin.org/postin the POST exampleDoc/library/urllib.robotparser.rst: Replace barehttp://www.robotstxt.org/orig.htmlURL with an:rfc:9309`` reference — RFC 9309 ("Robots Exclusion Protocol") is the authoritative IETF standard for robots.txtCloses #141444
Closes #141412
📚 Documentation preview 📚: https://cpython-previews--148952.org.readthedocs.build/