gh-150626: Update Format String Syntax documentation - #153143
Conversation
Co-authored-by: Robsdedude <dev@rouvenbauer.de>
Documentation build overview
|
|
@robsdedude @skirpichev @encukou @ZeroIntensity Could you please make a review of this PR? 🙏 If I can or should do something more here — please tell me and I will. (And I will be 200% careful this time) ℹ️ FYI: Same changes for the target file with original PR branch. |
robsdedude
left a comment
There was a problem hiding this comment.
✅ (I'm not an org member; I can't formally approve the PR)
There was a problem hiding this comment.
LGTM, based on previos discussions I am going to tag @encukou and @StanFromIreland
If they don't have any feedback, I will merge this in a couple of days :)
|
@zooba was somewhat against this here: https://discuss.python.org/t/format-string-syntax-specification-differs-from-actual-behaviour/46716/11 |
|
Thanks @encukou I remain against it, and even moreso looking at the proposed alternate text and re-reading the original discussion. What is this change meant to achieve? It's taken a simple grammar defined in terms of the language and indicating the intended use of the feature, and is now codifying edge cases that were probably unintentional and almost certainly not how we want people using it. The change I'd support would be to stop trying to tell the values apart in the grammar (i.e. make it >>> "{None}{is}{*}{1/0}{ 5}".format_map({"None":1, "is":2, "*": 3, "1/0": 4, " 5": 5})
'12345'Contorting the specification into something that is bound to have new edge cases just means we have to have this argument again. |
|
Thanks everyone for the feedback. After reading through the discussion again, I think I initially interpreted Steve's comment more broadly than it was intended, and that led me to mix together two different topics. I completely agree with the general documentation philosophy that API documentation shouldn't enumerate every concrete type or implementation detail. For example, documenting an argument as accepting an iterable is much better than listing list, tuple, dict, etc. Likewise, where there are notable caveats, documenting those is often sufficient. What I see as different here is that this section is not documenting an API contract — it is describing the grammar of the format string language. I think there is an important distinction between describing an interface and describing a language grammar. For interfaces, it makes perfect sense to specify the minimum that must be accepted without trying to describe every possible implementation detail. A grammar serves a different purpose: its job is to describe the language that is actually accepted. If the documented grammar rejects constructs that the implementation accepts, then it is no longer an accurate description of that language. From my perspective, that's what this PR is about. The current documentation isn't simply broader or narrower than the implementation — it describes a grammar that doesn't match what the parser actually accepts. The goal of this PR is only to bring those two back into alignment. At the same time, I completely agree that documentation shouldn't become overly legalistic or unnecessarily complicated. I've tried to keep the proposed wording as concise as possible while making it accurate. If there's a simpler or clearer way to express the same grammar, I'd be very happy to revise the proposal. One thing I'm still unsure about is what exactly "I remain against it" refers to. Is the objection to making this grammar description match the implementation at all, or is it specifically about the wording or approach proposed in this PR? If it's the latter, I'd be very happy to iterate on the wording. Any suggestions for how this could be expressed more clearly or more concisely would be greatly appreciated. |
You're right, and the thing to be aware of here is whether the document is documentation or specification. I would argue that it's documentation, and therefore the use of a grammar syntax is an analogy for describing the behaviour in a way that makes it easy for people who already understand that syntax. It seems that you're arguing that it is specification, which does have an obligation to be precise, but also then has to reflect the intent of the design rather than the reality of the design. What that means is that my proposed fix (remove the grammar, or make it obviously not specification) isn't changing the definition of the feature, but just the way it's described. Your proposal is a specification change, which has second and third order impacts that need to be considered (such as the headings given in the PEP template, which are designed to force/encourage authors to consider things like education, security, maintainability, transfer to other implementations, back-compat, etc.). So far, I haven't seen any consideration or discussion of these, which to me says we should reject an unmotivated proposal to change the language specification. By contrast, making the description less specific (and therefore more obviously allowing the behaviour that you've noticed) isn't a specification change, so it only requires checks for "is this easier to read and apply to the existing implementation". That's a much lower bar to meet. So I'm letting you choose whether you want to treat this as specification or documentation, and if you choose specification, then I'm letting you know that it's more work than you think and I don't think you've done it yet. |
Here a little problem. The Python has no specification other than it's documentation. But I'm with you in this case and strongly -1 on the PR. It's not obvious, that present behavior is intentional. From PEP 3101:
Note, that we don't support integer literals with unicode digits. This is a feature (IMO, unfortunate) of the int's constructor. Though, de-facto we have unicode pitfails, like mentioned by PEP 672: Python 3.16.0a0 (heads/main:6e983938834, Jul 25 2026, 12:03:39) [GCC 14.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> "{৪} {୨}".format(*range(10))
'4 2'Note, that this doesn't "work" for PyPy, as they follows to specification (well, maybe): Python 3.11.15 (194f9f44b505, May 25 2026, 19:34:11)
[PyPy 7.3.23 with GCC 10.2.1 20210130 (Red Hat 10.2.1-11)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>> "{৪} {୨}".format(*range(10))
Traceback (most recent call last):
File "<python-input-0>", line 1, in <module>
"{৪} {୨}".format(*range(10))
KeyError: '৪'But it does work in GraalPy, huh :( Python 3.12.8 (Tue Jun 23 15:26:01 UTC 2026)
[Graal, Oracle GraalVM, Java 25.0.3 (amd64)] on 'linux'
Type "help", "copyright", "credits" or "license" for more information.
>>> "{৪} {୨}".format(*range(10))
'4 2'So, I see following scenarios to address issue:
On this way we introduce more inconsistencies between different Python implementations. For no good reasons. BTW, another case like this: #135025 |
This PR is a replacement for the original PR #132736 that I have accidentally ruined. 😬
Related Issue: #150626
Original PR description.
Hello, let me try to help a bit with docs.
Here is an string formatting example:
This behaviour differs from Format String Syntax:
So I updated
arg_nameandattribute_namesyntax in that section to follow the actual behaviour:identifierwithattribute_nameinarg_nameattribute_nameasanything that doesn’t contain "." or "]"You can find original discussion thread here