Skip to content

[Content Understanding] Fix --schema-dir natural version sort (v10 > v9)#47957

Open
chienyuanchang wants to merge 1 commit into
mainfrom
cu-sdk/fix-discover-inner-version-sort
Open

[Content Understanding] Fix --schema-dir natural version sort (v10 > v9)#47957
chienyuanchang wants to merge 1 commit into
mainfrom
cu-sdk/fix-discover-inner-version-sort

Conversation

@chienyuanchang

Copy link
Copy Markdown
Member

[Content Understanding] Fix --schema-dir natural version sort (source-of-truth for JS/Java/.NET ports)

Summary

Fixes the version-sort bug in _discover_inner_from_dir (the classify-and-route --schema-dir shortcut) that Copilot flagged during review of the sibling .NET PR (Azure/azure-sdk-for-net#60394).

Bug (still on main): when --schema-dir finds multiple JSON files matching a category alias, the previous code did sorted(schema_dir.glob("*.json")) and took matches[-1] as the "newest version". That breaks as soon as any alias reaches double-digit versions:

invoice_v9.json  vs  invoice_v10.json
  → sorted lexicographically, '1' < '9' at position 9, so
    invoice_v10 sorts BEFORE invoice_v9,
    "alphabetical last" picks invoice_v9,
    and the tool silently loads the older schema.

Fix

Sort by a "version key" tuple, highest wins:

Group Matches Sort key
0 bare <alias>.json (no suffix; oldest baseline) (0, 0, "")
1 <alias>_v<N>.json or <alias>_<N>.json (1, N, "")
2 any other <alias>_<suffix>.json (2, 0, suffix)

So invoice_v10 beats invoice_v9 beats invoice, and invoice_draft still beats bare invoice.json but loses to any versioned file.

New _version_sort_key helper is module-level (single-underscore = "implementation detail", not test-invisible) so tests can pin the key extractor directly. Same shape as the .NET / JS / Java ports for cross-language parity.

Cross-language coordination

This bug came from Python and was faithfully ported to Java, JS, and .NET. To keep the four in lockstep:

Language Status
.NET ✅ Fixed on the open PR: Azure/azure-sdk-for-net#60394 commit 61bda1abd16
JS ✅ Fixed on the open PR: Azure/azure-sdk-for-js#39137 commit d2a463cde18
Java ✅ Fixed on the open PR: Azure/azure-sdk-for-java#49672 commit 7b07a7564d3
Python this PR — the source-of-truth is on main, so it needs its own PR

Test coverage (+11 pytest cases)

Test Purpose
test_version_sort_key_bare_alias_returns_group_zero (0, 0, "") for bare <alias>
test_version_sort_key_v_prefixed_numeric_returns_group_one v10 > v9 — pins the fix
test_version_sort_key_bare_numeric_returns_group_one <alias>_<N> (no v prefix) also recognised
test_version_sort_key_non_numeric_suffix_returns_group_two Fallback lex tiebreaker
test_discover_inner_from_dir_resolves_exact_match_stem Baseline resolution
test_discover_inner_from_dir_picks_natural_version_max_not_alphabetical The exact regression Copilot flagged
test_discover_inner_from_dir_prefers_versioned_over_bare_alias Versioned always beats baseline
test_discover_inner_from_dir_skips_prebuilt_aliases prebuilt-* needs no local file
test_discover_inner_from_dir_skips_categories_without_analyzer_id Classification-only buckets
test_discover_inner_from_dir_missing_aliases_raises Every missing alias named in error
test_discover_inner_from_dir_unrelated_json_files_ignored Extra files in dir don't confuse resolution
test_discover_inner_from_dir_non_existent_dir_raises Sanity check

Test results

$ python -m pytest tests/test_skills_classify_route_router.py -v
============================= 19 passed in 11.57s ==============================

Was 8 before this PR (7 pre-existing + 1 pre-existing versionSort ancestor); +11 new = 19 total.

Backward compatibility

  • Public API unchanged.
  • User-visible behaviour: only affects users of --schema-dir who have multiple matching files per alias with _v<N> suffixes AND at least one two-digit N. Before, they silently got the wrong file; now they get the right one.
  • No new dependencies (uses stdlib re).

CHANGELOG

Added a "Bugs Fixed" entry under the existing 1.2.0b3 (Unreleased) section — no version bump needed since b3 has not shipped yet.

Checklist

  • Test coverage (19/19 passing, +11 new)
  • CHANGELOG entry under 1.2.0b3 (Unreleased)
  • No public API changes
  • Cross-language fixes queued on the sibling PRs

Addresses the Copilot review comment from the sibling .NET PR
(azure-sdk-for-net#60394, second-round review). The bug came from
Python and was faithfully ported to Java (#49672) / JS (#39137) /
.NET (#60394) — this PR fixes the source-of-truth so the four
languages stay in lockstep. .NET is already fixed
(commit 61bda1abd16), JS is queued (commit d2a463cde18 on the
open PR), Java is queued (commit 7b07a7564d3 on the open PR).

Before: _discover_inner_from_dir did 'sorted(schema_dir.glob("*.json"))'
and took the last element as the 'newest version'. That breaks the
moment any alias reaches double-digit versions:

  invoice_v9.json  vs  invoice_v10.json
  → sorted lexicographically, '1' < '9' at position 9, so
    invoice_v10 sorts BEFORE invoice_v9,
    'alphabetical last' picks invoice_v9,
    and the tool silently loads the wrong (older) schema.

After: sort by a 'version key' tuple, highest wins:

  - group 0 → bare '<alias>.json'                (no suffix; oldest)
  - group 1 → '<alias>_v<N>.json' or             (numeric suffix;
              '<alias>_<N>.json'                  sorted by N)
  - group 2 → any other '<alias>_<suffix>.json'  (lex tiebreaker)

So invoice_v10 beats invoice_v9 beats invoice, and 'invoice_draft'
still beats bare 'invoice.json' but loses to any versioned file.
New _version_sort_key helper is module-level (leading-underscore is
just 'implementation detail', not test-invisible) so tests can pin
the key extractor directly.

Test coverage (+11 pytest cases in test_skills_classify_route_router.py):
  - test_version_sort_key_bare_alias_returns_group_zero
  - test_version_sort_key_v_prefixed_numeric_returns_group_one   (v10 > v9)
  - test_version_sort_key_bare_numeric_returns_group_one         (_42)
  - test_version_sort_key_non_numeric_suffix_returns_group_two
  - test_discover_inner_from_dir_resolves_exact_match_stem
  - test_discover_inner_from_dir_picks_natural_version_max_not_alphabetical
    (the exact regression the .NET Copilot review flagged)
  - test_discover_inner_from_dir_prefers_versioned_over_bare_alias
  - test_discover_inner_from_dir_skips_prebuilt_aliases
  - test_discover_inner_from_dir_skips_categories_without_analyzer_id
  - test_discover_inner_from_dir_missing_aliases_raises
  - test_discover_inner_from_dir_unrelated_json_files_ignored
  - test_discover_inner_from_dir_non_existent_dir_raises

CHANGELOG.md: added a 'Bugs Fixed' entry to the existing
1.2.0b3 (Unreleased) section (no version bump needed — b3 has
not shipped yet).

Verified: pytest tests/test_skills_classify_route_router.py → 19/19
passing (was 8; +11 new cases).

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes a natural-version-sort bug in the --schema-dir shortcut of the Content Understanding "classify-and-route" skill helper script (create_and_test_router.py). Previously, _discover_inner_from_dir relied on lexicographic sorting (sorted(...)[-1]), which incorrectly ordered invoice_v10.json before invoice_v9.json (because '1' < '9' character-by-character), silently selecting the older schema once versions reached double digits. The fix introduces a module-level _version_sort_key helper that sorts by a (group, numeric_version, suffix) tuple so the highest numeric version wins. It is the source-of-truth counterpart to sibling fixes already opened for .NET, JS, and Java.

Changes:

  • Added _version_sort_key (with a _VERSION_SUFFIX regex) and switched _discover_inner_from_dir to sort matches by that key instead of alphabetical order.
  • Added 11 pytest cases covering the key extractor and directory-resolution behavior.
  • Added a "Bugs Fixed" CHANGELOG entry under the unreleased 1.2.0b3 section.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

File Description
.github/skills/cu-sdk-author-analyzer-classify-route/scripts/create_and_test_router.py Adds _version_sort_key/_VERSION_SUFFIX and replaces alphabetical sort with numeric-version-aware sorting in _discover_inner_from_dir.
tests/test_skills_classify_route_router.py Adds 11 tests for the key extractor and directory resolution, including the v10 > v9 regression.
CHANGELOG.md Documents the --schema-dir version-sort bug fix under 1.2.0b3 (Unreleased).

Comment on lines +143 to +149
m = _VERSION_SUFFIX.fullmatch(suffix)
if m is not None:
try:
return (1, int(m.group(1)), "")
except ValueError:
pass # fall through to group 2
return (2, 0, suffix)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants