Skip to content

Add per-problem metadata files with descriptions - #331

Merged
dweindl merged 2 commits into
PEtab-dev:masterfrom
dweindl:problem-metadata
Oct 13, 2026
Merged

dweindl merged 2 commits into
PEtab-dev:masterfrom
dweindl:problem-metadata

Conversation

@dweindl

@dweindl dweindl commented Oct 12, 2026 •

Copy link
Copy Markdown
Member

Adds a machine-readable, PEtab-version-independent metadata file problems/<ProblemID>/metadata.yaml for each problem. For now, it holds a single required field, short_description: one plain-ASCII sentence (20–160 characters) stating the model type (reaction network, compartmental epidemiological model, cell population model, ...), the biological system and, where informative, the data. It's kept short so it can later go into the overview table. The name leaves room for a later free-form description field for longer text (hints, experiences, additional references, ...). Further fields (e.g., a structured model type, DOI) are planned and will be added separately.

Example (Boehm_JProteomeRes2014):

short_description: Reaction network model of Epo-induced STAT5A/STAT5B phosphorylation and homo- and heterodimerization, fitted to quantitative mass spectrometry data.

The metadata is kept out of problem.yaml so it isn't duplicated across v1/ and the upcoming v2/, and doesn't interfere with PEtab YAML validation.

Changes:

  • metadata.yaml with a short description for all 35 problems. Descriptions were written based on the publications and model files; please check the ones for problems you know well.
  • Pydantic model ProblemMetadata defining the metadata format. Unknown fields are rejected, so each new field requires a model change.
  • JSON schema benchmark_models_petab/schemas/metadata.schema.json, generated from ProblemMetadata via write_metadata_schema() and included in the package via MANIFEST.in; a test checks that it is up to date.
  • get_metadata(id_), returning a ProblemMetadata, and a bmp-check-metadata CLI that validates all metadata files; added as a step to the "Tests" workflow.
  • pydantic and pyyaml added as explicit dependencies (both are already transitive dependencies of recent petab versions).
  • CONTRIBUTING.md and the PR template updated.

🤖 Drafted with AI assistance

Each problem now has `problems/<ProblemID>/metadata.yaml` for metadata
that applies to both PEtab versions. For now, it holds a single
required field, `description`: one plain-ASCII sentence (20-160
characters) that states the model type, the biological system and,
where informative, the data. It is short enough to fit a future
overview table column. Descriptions are added for all 35 problems.
Further fields are planned and will be added separately.

The metadata is kept out of `problem.yaml` so it isn't duplicated
across `v1/` and `v2/` and doesn't interfere with PEtab YAML
validation.

* Add JSON schema `schemas/metadata.schema.json`, shipped in the
  package via `MANIFEST.in`. Unknown fields are rejected, so new
  fields require a schema change.
* Add `get_metadata()` and the `bmp-check-metadata` CLI, which
  validates all metadata files; run it in the "Tests" workflow.
* Add `jsonschema` and `pyyaml` as explicit dependencies.
* Document the file in CONTRIBUTING.md and the PR template.

🤖 Drafted with AI assistance

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@dweindl
dweindl marked this pull request as ready for review October 12, 2026 14:10

@dilpath dilpath left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • short_description instead of description? In future, I think all information that is normally stored in the github issues will be inside the metadata file so there is a single-source of information, and then we will need some generic description field where a lot of hints/tips/experiences/additional references/longer descriptions etc. can go
  • implement an object model for metadata, e.g. with pydantic and possibly mkstd? Will simplify things a bit I think, e.g. adding new fields and validating files.

Comment thread src/python/benchmark_models_petab/C.py Outdated
Comment on lines +17 to +18
# version-independent metadata (`<MODELS_DIR>/<ProblemID>/metadata.yaml`)
METADATA_FILENAME: str = "metadata.yaml"

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# version-independent metadata (`<MODELS_DIR>/<ProblemID>/metadata.yaml`)
METADATA_FILENAME: str = "metadata.yaml"
METADATA_FILENAME: str = "metadata.yaml"

Address review on PEtab-dev#331:

* Model `metadata.yaml` by the pydantic model `ProblemMetadata` and
  validate files against it. `get_metadata()` now returns a
  `ProblemMetadata` instance. The packaged JSON schema is generated from
  the model via `write_metadata_schema()`; a test checks that it is up to
  date. Replaces the `jsonschema` dependency by `pydantic`.
* Rename `description` to `short_description`, leaving `description` for
  a future longer free-form description.
* Drop a redundant comment in `C.py`.

🤖 Drafted with AI assistance

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@dweindl

dweindl commented Oct 13, 2026

Copy link
Copy Markdown
Member Author
  • short_description instead of description? In future, I think all information that is normally stored in the github issues will be inside the metadata file so there is a single-source of information, and then we will need some generic description field where a lot of hints/tips/experiences/additional references/longer descriptions etc. can go

    • implement an object model for metadata, e.g. with pydantic and possibly mkstd? Will simplify things a bit I think, e.g. adding new fields and validating files.

Changed to short_description, using pydantic now but keeping the pydantic-derived jsonschema for external validation.

@dweindl
dweindl merged commit 7014f2a into PEtab-dev:master Oct 13, 2026
3 checks passed
@dweindl
dweindl deleted the problem-metadata branch October 13, 2026 19:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants