Skip to content

fix(i18n): fall back to bare language for script-only codes and catch ValueError (#1896) - #1900

Open
Olegt0rr wants to merge 1 commit into
aiogram:dev-3.xfrom
Olegt0rr:claude/aiogram-issue-1896-2d4ef5
Open

fix(i18n): fall back to bare language for script-only codes and catch ValueError (#1896)#1900
Olegt0rr wants to merge 1 commit into
aiogram:dev-3.xfrom
Olegt0rr:claude/aiogram-issue-1896-2d4ef5

Conversation

@Olegt0rr

@Olegt0rr Olegt0rr commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

Fixes #1896.

Problem

#1881 (shipped in 3.31.0) changed SimpleI18nMiddleware.get_locale() to append the bare-language candidate only when locale.territory is set. Telegram sends script-qualified codes with no territory (zh-hans, zh-hant, sr-latn, uz-cyrl); Babel parses them to e.g. zh_Hans with territory=None, so the only candidate was zh_Hans and a bot shipping a plain zh/ catalogue silently fell to default_locale. On 3.30.0 the same users got zh.

Separately, Locale.parse("en_US", sep="-") and Locale.parse("", sep="-") raise a plain ValueError (UnknownLocaleError is not a ValueError subclass), which propagated out of the middleware and dropped every update from such a user.

Fix

  • Candidates are tried in order: full form (str(locale)), language_TERRITORY, language_Script, bare language. Territory goes before script because Babel infers a script the user never sent (zh-cnzh_Hans_CN), while the territory is usually explicit; with zh_Hans/ and zh_CN/ both present, zh-cn now resolves to zh_CN.
  • except (UnknownLocaleError, ValueError) → default locale.
  • Membership is checked on I18n.locales (the dict gettext itself keys on), hoisted out of the loop.
  • The resolution order is documented in the SimpleI18nMiddleware docstring (rendered via autoclass).

Resolution examples with catalogues {en, zh, zh_Hans, zh_CN}:

language_code 3.30.0 3.31.0 this PR
zh-hans zh en zh_Hans
zh-hant zh en zh
sr-latn (with sr/) sr en sr
zh-cn zh zh zh_CN
en_US, "" raises raises en (default)

Not changed (deliberately)

  • Underscore-form codes (en_US) are not normalized to en-US; they resolve to the default locale as before-the-crash behaviour would suggest. Telegram never sends underscores.
  • Babel's likely-subtag resolution can still override an explicit script for exotic codes (zh-hant-cn parses with script Hans). Fixing that would require bypassing Locale.parse; out of scope.
  • FSMI18nMiddleware does not re-resolve locales already cached in FSM storage; the changelog fragment tells operators of 3.31.0 to clear the key or call set_locale.

Validation

uv run ruff check --show-fixes --preview aiogram examples
uv run ruff format --check --diff aiogram tests scripts examples
uv run mypy aiogram                       # Success: no issues found in 763 source files
uv run pytest tests                       # 2326 passed, 51 skipped
uv run pytest tests/test_utils/test_i18n.py --cov=aiogram/utils/i18n   # middleware.py 100%
uv run towncrier build --draft --version 3.99.0

New regression rows in tests/test_utils/test_i18n.py: zh-hans, zh-hant, sr-latn, uz-cyrl, zh-hans-cn, zh-cn, zh-tw, zh-sg, en_US, pt_BR, empty string; gettext checks prove the zh/ (script fallback) and zh_CN/ (territory tier) catalogues are the ones actually used.

🤖 Generated with Claude Code

… ValueError (aiogram#1896)

Regression from aiogram#1881: SimpleI18nMiddleware appended the bare language
candidate only when a territory was present, so script-qualified Telegram
codes (zh-hans, zh-hant, sr-latn) skipped the plain `zh`/`sr` catalogue and
fell to the default locale.

- Candidates are now tried as: full form, language_TERRITORY,
  language_Script, bare language. Territory precedes script because Babel
  infers a script for codes such as `zh-cn` (parsed as zh_Hans_CN).
- Locale.parse ValueError (underscore form like `en_US`, empty string) is
  caught and resolves to the default locale instead of dropping the update.
- Membership is checked on I18n.locales directly, hoisted out of the loop.
- Document the resolution order in the SimpleI18nMiddleware docstring.
- Regression tests for script-only, script+territory, inferred-script and
  malformed codes, plus gettext checks for the script and territory tiers.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Copilot AI lite review requested due to automatic review settings September 3, 2026 10:07
@github-actions

github-actions Bot commented Sep 3, 2026

Copy link
Copy Markdown

❌ Changelog is required!

You need to add a brief description of the changes to the CHANGES directory.

Changes file should be named like <issue or PR number>.<category>.rst,
example 1234.bugfix.rst where 1234 is the PR or issue number and bugfix is the category.

The content of the file should be a brief description of the changes in
the PR in the format of a description of what has been done.

Possible categories are: feature, bugfix, doc, removal and misc.

@github-actions github-actions Bot added the 3.x Issue or PR for stable 3.x version label Sep 3, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The locale resolution logic aligns with the stated negotiation order, adds appropriate error handling, and is backed by targeted regression and end-to-end gettext tests.

Pull request overview

This PR fixes locale resolution in SimpleI18nMiddleware for Telegram language_code values that include a script but no territory (e.g. zh-hans, sr-latn), restoring the expected fallback to the bare language when only zh/ or sr/ catalogues exist. It also hardens locale parsing by treating ValueError from babel.Locale.parse() as an invalid locale and falling back to default_locale, preventing updates from being dropped on malformed/underscore-form inputs.

Changes:

  • Extend locale candidate negotiation to try, in order: full parsed form, language_TERRITORY, language_Script, then bare language.
  • Catch (UnknownLocaleError, ValueError) from Babel parsing and fall back to the default locale.
  • Add regression tests and a towncrier changelog fragment documenting the behavior and operational note about FSM-cached locales.
File summaries
File Description
aiogram/utils/i18n/middleware.py Adjusts locale candidate generation and parsing error handling; documents the resolution order in the middleware docstring.
tests/test_utils/test_i18n.py Adds regression coverage for script-only, script+territory, malformed/underscore-form, and empty language_code inputs, including gettext end-to-end checks.
CHANGES/1896.bugfix.rst Adds a changelog fragment describing the regression fix, the new resolution order, and the FSM cache note.
Review details
  • Files reviewed: 3/3 changed files
  • Comments generated: 0
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

3.x Issue or PR for stable 3.x version

Projects

None yet

Development

Successfully merging this pull request may close these issues.

SimpleI18nMiddleware no longer falls back to the bare language for script-only codes (zh-hans, zh-hant, sr-latn)

2 participants