Bug report

Bug description:

The incremental decoder of the idna codec counts the dot before a label as consumed only if the labels before it were non-empty. After leading empty labels it returns the dot as text but leaves it in its buffer, so the next call decodes it again.

import codecs, io
d = codecs.getincrementaldecoder("idna")()
print(repr(d.decode(b".example.com") + d.decode(b"", True)))  # '.example..com'
print(len(io.TextIOWrapper(io.BytesIO(b"." * 1000), encoding="idna").readline()))  # 500500

Expected '.example.com' and 1000, as bytes.decode("idna") gives. Bytes also remain in the decoder after final=True, and the UnicodeDecodeError position for an invalid label after the first is one less than the one-shot decoder reports.

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

Linked PRs

• gh-161102: Fix idna incremental decoder after leading empty labels #161103
• [3.14] gh-161102: Fix idna incremental decoder after leading empty labels (GH-161103) #161129
• [3.15] gh-161102: Fix idna incremental decoder after leading empty labels (GH-161103) #161130