fix(i18n): fall back to bare language for script-only codes and catch ValueError (#1896) - #1900
fix(i18n): fall back to bare language for script-only codes and catch ValueError (#1896)#1900Olegt0rr wants to merge 1 commit into
Conversation
… ValueError (aiogram#1896) Regression from aiogram#1881: SimpleI18nMiddleware appended the bare language candidate only when a territory was present, so script-qualified Telegram codes (zh-hans, zh-hant, sr-latn) skipped the plain `zh`/`sr` catalogue and fell to the default locale. - Candidates are now tried as: full form, language_TERRITORY, language_Script, bare language. Territory precedes script because Babel infers a script for codes such as `zh-cn` (parsed as zh_Hans_CN). - Locale.parse ValueError (underscore form like `en_US`, empty string) is caught and resolves to the default locale instead of dropping the update. - Membership is checked on I18n.locales directly, hoisted out of the loop. - Document the resolution order in the SimpleI18nMiddleware docstring. - Regression tests for script-only, script+territory, inferred-script and malformed codes, plus gettext checks for the script and territory tiers. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
❌ Changelog is required!You need to add a brief description of the changes to the Changes file should be named like The content of the file should be a brief description of the changes in Possible categories are: |
There was a problem hiding this comment.
🟢 Approval recommended
The locale resolution logic aligns with the stated negotiation order, adds appropriate error handling, and is backed by targeted regression and end-to-end gettext tests.
Pull request overview
This PR fixes locale resolution in SimpleI18nMiddleware for Telegram language_code values that include a script but no territory (e.g. zh-hans, sr-latn), restoring the expected fallback to the bare language when only zh/ or sr/ catalogues exist. It also hardens locale parsing by treating ValueError from babel.Locale.parse() as an invalid locale and falling back to default_locale, preventing updates from being dropped on malformed/underscore-form inputs.
Changes:
- Extend locale candidate negotiation to try, in order: full parsed form,
language_TERRITORY,language_Script, then barelanguage. - Catch
(UnknownLocaleError, ValueError)from Babel parsing and fall back to the default locale. - Add regression tests and a towncrier changelog fragment documenting the behavior and operational note about FSM-cached locales.
File summaries
| File | Description |
|---|---|
aiogram/utils/i18n/middleware.py |
Adjusts locale candidate generation and parsing error handling; documents the resolution order in the middleware docstring. |
tests/test_utils/test_i18n.py |
Adds regression coverage for script-only, script+territory, malformed/underscore-form, and empty language_code inputs, including gettext end-to-end checks. |
CHANGES/1896.bugfix.rst |
Adds a changelog fragment describing the regression fix, the new resolution order, and the FSM cache note. |
Review details
- Files reviewed: 3/3 changed files
- Comments generated: 0
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Fixes #1896.
Problem
#1881 (shipped in 3.31.0) changed
SimpleI18nMiddleware.get_locale()to append the bare-language candidate only whenlocale.territoryis set. Telegram sends script-qualified codes with no territory (zh-hans,zh-hant,sr-latn,uz-cyrl); Babel parses them to e.g.zh_Hanswithterritory=None, so the only candidate waszh_Hansand a bot shipping a plainzh/catalogue silently fell todefault_locale. On 3.30.0 the same users gotzh.Separately,
Locale.parse("en_US", sep="-")andLocale.parse("", sep="-")raise a plainValueError(UnknownLocaleErroris not aValueErrorsubclass), which propagated out of the middleware and dropped every update from such a user.Fix
str(locale)),language_TERRITORY,language_Script, barelanguage. Territory goes before script because Babel infers a script the user never sent (zh-cn→zh_Hans_CN), while the territory is usually explicit; withzh_Hans/andzh_CN/both present,zh-cnnow resolves tozh_CN.except (UnknownLocaleError, ValueError)→ default locale.I18n.locales(the dictgettextitself keys on), hoisted out of the loop.SimpleI18nMiddlewaredocstring (rendered via autoclass).Resolution examples with catalogues
{en, zh, zh_Hans, zh_CN}:language_codezh-hanszhenzh_Hanszh-hantzhenzhsr-latn(withsr/)srensrzh-cnzhzhzh_CNen_US,""en(default)Not changed (deliberately)
en_US) are not normalized toen-US; they resolve to the default locale as before-the-crash behaviour would suggest. Telegram never sends underscores.zh-hant-cnparses with scriptHans). Fixing that would require bypassingLocale.parse; out of scope.FSMI18nMiddlewaredoes not re-resolve locales already cached in FSM storage; the changelog fragment tells operators of 3.31.0 to clear the key or callset_locale.Validation
New regression rows in
tests/test_utils/test_i18n.py:zh-hans,zh-hant,sr-latn,uz-cyrl,zh-hans-cn,zh-cn,zh-tw,zh-sg,en_US,pt_BR, empty string; gettext checks prove thezh/(script fallback) andzh_CN/(territory tier) catalogues are the ones actually used.🤖 Generated with Claude Code