Conversation
* Hi ITN: implement Roman semiotic class Add an inverse-text-normalization Roman numeral class for Hindi that converts spoken numbers following a fixed set of context key words (अध्याय / खंड / खण्ड / कक्षा) into Roman numerals. अध्याय तीन -> अध्याय III कक्षा बारह -> कक्षा XII Conversion is restricted to these predictable contexts; regnal/papal and product names are a documented limitation. Mirrors the TN Roman class structure (data/__init__.py, license headers, pytest + sparrowhawk tests). Signed-off-by: Mayuri S <mayuris@nvidia.com> * Hi ITN Roman: address review feedback Signed-off-by: Mayuri S <mayuris@nvidia.com> * Hi ITN Roman: drop strip in test, add boundary cases, 4000 Arabic fallback, bump cache Signed-off-by: Mayuri S <mayuris@nvidia.com> --------- Signed-off-by: Mayuri S <mayuris@nvidia.com>
* feat(hi): add Hindi ITN electronic class Signed-off-by: Mayuri S <mayuris@nvidia.com> * chore: remove laptop setup guide and temp diag scripts Signed-off-by: Mayuri S <mayuris@nvidia.com> * chore: remove scratch helper script Signed-off-by: Mayuri S <mayuris@nvidia.com> * chore(hi): remove percentage class (out of scope for electronic PR) Signed-off-by: Mayuri S <mayuris@nvidia.com> * chore(hi): remove leftover percentage tagger and verbalizer Signed-off-by: Mayuri S <mayuris@nvidia.com> * style: apply black and isort formatting Signed-off-by: Mayuri S <mayuris@nvidia.com> * hi ITN: TSV-driven electronic refactor + serial class + review fixes Signed-off-by: Mayuri S <mayuris@nvidia.com> * hi ITN: move alnum to serial, reduce weights, punctuation postprocessor Signed-off-by: Mayuri S <mayuris@nvidia.com> * updating fixes Signed-off-by: Mayuri S <mayuris@nvidia.com> * hi ITN: updated the changes Signed-off-by: Mayuri S <mayuris@nvidia.com> * hi ITN: move telephone changes to a separate PR Signed-off-by: Mayuri S <mayuris@nvidia.com> * hi ITN: use exact match in serial and electronic tests Signed-off-by: Mayuri S <mayuris@nvidia.com> --------- Signed-off-by: Mayuri S <mayuris@nvidia.com> Signed-off-by: mayuris-00 <mayuris@nvidia.com>
* hi ITN: add IP address support to telephone, dedupe TSV loads Signed-off-by: Mayuri S <mayuris@nvidia.com> * hi ITN: add shared grammar utils, symbols TSV, and CI cache bump Signed-off-by: Mayuri S <mayuris@nvidia.com> * chore: retrigger DCO check Signed-off-by: Mayuri S <mayuris@nvidia.com> --------- Signed-off-by: Mayuri S <mayuris@nvidia.com>
Signed-off-by: Mayuri S <mayuris@nvidia.com>
* hi ITN: add math operations and telephone extensions Signed-off-by: Mayuri S <mayuris@nvidia.com> * Updated the test cases Signed-off-by: Mayuri S <mayuris@nvidia.com> * hi ITN: require single digit words for contextless short numbers Signed-off-by: Mayuri S <mayuris@nvidia.com> --------- Signed-off-by: Mayuri S <mayuris@nvidia.com>
Signed-off-by: Mayuri S <mayuris@nvidia.com>
Signed-off-by: Mai Anh <palasek182@gmail.com>
folivoramanh
left a comment
There was a problem hiding this comment.
Reviewed this as an integration PR rather than re-litigating the eight feature PRs it carries. Full Hindi suite passes 1230/1230 on the merge result, which covers the unchecked unit-test box in the description.
Integration checks are clean:
- Nothing from
mainis lost —git log main ^HEADis empty, so the branch is fully current after18d10e83. Onlyinverse_normalize.pychanges outsidehi/(+15/-0), and the Jenkinsfile keeps main'sJA_/KO_cache bumps while bumpingHI_TN_CACHE. - Sparrowhawk wiring is complete — 15
testITN*entries with 15 matchingtest_cases_*.txt, including the three new classes (electronic, roman, serial). - The new
post_processhook is genuinely applied, not dead code —hiis already in the hardcoded["en", "hi", "ja", "vi"]list innormalize.py, and the flag toggles as intended:मुझे एक सौ रुपये चाहिए । post_process=True -> मुझे ₹१०० चाहिए। post_process=False -> मुझे ₹१०० चाहिए ।
should-fix · verified — nemo_text_processing/inverse_text_normalization/hi/data/electronic/__init__.py and nemo_text_processing/inverse_text_normalization/hi/data/serial/__init__.py are both 0 bytes, while every sibling carries the licence header (hi/data/numbers/__init__.py is 610 bytes, starting # Copyright (c) 2024, NVIDIA CORPORATION.). CONTRIBUTING requires the header on every new Python file, and these are the only two added .py files missing it. (Commenting here rather than inline because an empty added file has no diff hunk to anchor to.)
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com> Signed-off-by: Mai Anh <palasek182@gmail.com>
431db06 to
bfd8204
Compare
What does this PR do ?
Add a one line overview of what this PR aims to accomplish.
Before your PR is "Ready for review"
Pre checks:
git commit -sto sign.pytestor (if your machine does not have GPU)pytest --cpufrom the root folder (given you marked your test cases accordingly@pytest.mark.run_only_on('CPU')).bash tools/text_processing_deployment/export_grammars.sh --MODE=test ...pytestand Sparrowhawk here.__init__.pyfor every folder and subfolder, includingdatafolder which has .TSV files?Copyright (c) 2023, NVIDIA CORPORATION & AFFILIATES. All rights reserved.to all newly added Python files?Copyright 2015 and onwards Google, Inc.. See an example here.try import: ... except: ...) if not already done.PR Type:
If you haven't finished some of the above items you can still open "Draft" PR.