Skip to content

Allow a list of IDSs and occurrences in imas convert --ids, and deprecate --occurrence - #152

Merged
olivhoenen merged 5 commits into
iterorganization:developfrom
Shakhtar-Sankur:convert-ids-list
Nov 17, 2026
Merged

olivhoenen merged 5 commits into
iterorganization:developfrom
Shakhtar-Sankur:convert-ids-list

Conversation

@Shakhtar-Sankur

@Shakhtar-Sankur Shakhtar-Sankur commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

Closes #144

As discussed in #144, imas convert --ids now takes a comma-separated list of IDS, IDS:OCCURRENCE or IDS:* (all occurrences):

imas convert <uri_in> 4.1.0 <uri_out> --ids summary,plasma_profiles:1,plasma_profiles:2
  • As in the IDS path syntax, an IDS without an occurrence means occurrence 0; IDS:* selects all its occurrences. Leaving --ids out still converts every IDS and occurrence.
  • --occurrence is deprecated, as @maarten-ic suggested: it still applies to IDSs given without an occurrence, but logs a warning pointing to --ids IDS:OCCURRENCE, so it can be removed at the next major release.
  • Spaces and repeated entries are ignored.
  • Everything is checked before the output Data Entry is created, and the command stops with exit status 2:
    • a name that is not an IDS in the target DD: unknown IDS name(s) for DD version 4.1.0: core_profile;
    • a malformed occurrence such as core_profiles:x or core_profiles:²;
    • an IDS or occurrence that is not in the input Data Entry: not found in the input Data Entry: core_profiles:7, wall:0.

On develop, a name that is not an IDS (a typo such as --ids core_profile, or a list, which develop reads as one name) converts nothing and still exits with status 0.

Behaviour change: on develop, --ids core_profiles converts all occurrences of core_profiles; with this PR it converts occurrence 0 only, and core_profiles:* gives the old behaviour.

cli.py uses from __future__ import annotations, so the new type hints are not evaluated at import time on Python 3.9.

Tests in imas/test/test_cli.py:

  • test_imas_convert_ids_list: only the listed IDSs are converted;
  • test_imas_convert_unknown_ids: a misspelled name exits with status 2 and creates no output;
  • test_imas_convert_ids_with_occurrences: a bare name converts occurrence 0, IDS:N occurrence N, and IDS:* all of them;
  • test_imas_convert_occurrence_option_is_deprecated: --occurrence logs the warning and still applies to IDSs without an occurrence;
  • test_imas_convert_invalid_occurrence: :x, :-1, :² and an empty occurrence exit with status 2 and create no output;
  • test_imas_convert_ids_missing_from_input: a missing occurrence, a missing IDS and IDS:* with nothing in the input exit with status 2 and create no output.

All of test_cli.py: 15 passed. Full test suite (pytest -n=auto, as in the CI): 1808 passed, 297 skipped. black (24.x, as in linting.yml) and flake8 are clean. I could not run the tests on Python 3.9: imas-core 5.7.2 publishes wheels for cp310 to cp314 only.

The CLI docs and --help describe the new form. I have not touched the changelog, since it is updated at release time; I'm happy to add an entry if you prefer.

Run log: https://www.kaggle.com/code/sankurkundu/notebook75525243ea

imas convert could convert either every IDS in a Data Entry or a single
one. --ids now also takes a comma-separated list, for example
--ids core_profiles,equilibrium,summary; --occurrence applies to each IDS
in the list. The names are checked against the target DD before the
output Data Entry is created, so a misspelled name fails at the start
instead of partway through a conversion.

Closes iterorganization#144
--ids now takes IDS or IDS:OCCURRENCE entries, for example
--ids summary,plasma_profiles:1,plasma_profiles:2, so several
occurrences of an IDS can be converted together. An IDS given without an
occurrence still converts all of its occurrences.

--occurrence only made sense with a single IDS. It still applies to IDSs
given without an occurrence, but logs a deprecation warning pointing to
the new syntax, so it can be removed at the next major release. A
malformed occurrence (core_profiles:x) is rejected before the output
Data Entry is created.
@Shakhtar-Sankur Shakhtar-Sankur changed the title Allow a comma-separated list of IDSs in imas convert --ids Allow a list of IDSs and occurrences in imas convert --ids, and deprecate --occurrence Nov 13, 2026

@maarten-ic maarten-ic left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some comments and suggestions to discuss:

Comment thread imas/command/cli.py Outdated
If not provided, all IDSs in the data entry are converted.",
help="Specify which IDSs to convert, as a comma-separated list of IDS or "
"IDS:OCCURRENCE, for example summary,plasma_profiles:1,plasma_profiles:2. "
"An IDS without an occurrence converts all its occurrences. "

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the IDS Path syntax, omitting the occurrence means you want the default occurrence (0), not all occurrences.

Maybe we'd need an explicit syntax for all occurrences, for example <ids_name>:*?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, done: as in the IDS path syntax, an IDS without an occurrence now means occurrence 0, and IDS:* selects all occurrences, for example --ids summary,plasma_profiles:*. Leaving --ids out still converts every IDS and occurrence, as before.

One behaviour change to be aware of: on develop, --ids core_profiles converts all occurrences of core_profiles; with this PR it converts occurrence 0 only, and core_profiles:* gives the old behaviour. If you'd rather keep the old meaning for a bare name, it's a one-line change.

Comment thread imas/command/cli.py Resolved
Comment thread imas/command/cli.py Resolved
Comment thread docs/source/cli.rst Resolved
As in the IDS path syntax, an IDS given in --ids without an occurrence now
means occurrence 0; IDS:* selects all occurrences. Leaving --ids out still
converts every IDS and occurrence.

An IDS or occurrence that is not in the input Data Entry is now reported
before the output Data Entry is created, instead of failing partway
through or converting nothing.

The CLI docs no longer mention the deprecated --occurrence option.
Comment thread imas/command/cli.py Outdated

def _parse_ids_option(
ids: str, known_ids: list[str], dd_version: str
) -> list[tuple[str, int | None]]:

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this will fail on Python 3.9 and older unless we add from __future__ import annotations.
In pyproject.toml we indicate versions >=3.7 although we only test from 3.10 as part of our CI. Personally, I think we should still support 3.9 which is the standard version on RHEL 9 systems (although even 3.10 is now EOL).

@Shakhtar-Sankur Shakhtar-Sankur Nov 13, 2026 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, done in ec31eb1: cli.py now has from __future__ import annotations, so the new hints are not evaluated at import time, and there is no other 3.10-only syntax in cli.py. I could not run the tests on 3.9 itself: imas-core 5.7.2 publishes wheels for cp310 to cp314 only, so pip cannot install it there. On 3.12, test_cli.py (15) and the full suite (1808 passed, 297 skipped) pass: https://www.kaggle.com/code/sankurkundu/notebook75525243ea

Comment thread imas/command/cli.py Resolved
Comment thread imas/command/cli.py Resolved
Add from __future__ import annotations so that the new type hints
(int | str | None) are not evaluated at import time, which fails on
Python 3.9 and older.

Use str.isdecimal() rather than str.isdigit() for the occurrence, so
that superscript digits such as "2" in "core_profiles:²" are rejected
with a usage error instead of crashing int().

Describe convert as converting selected IDSs/occurrences.

@olivhoenen olivhoenen left a comment •

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me now with imas convert. Thanks!

@olivhoenen
olivhoenen merged commit dca2470 into iterorganization:develop Nov 17, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

imas convert for a selected list of IDSs

4 participants