diff --git a/specs/015-graph-exports/contracts/checks.md b/specs/015-graph-exports/contracts/checks.md index af75cd0b..941e1fd5 100644 --- a/specs/015-graph-exports/contracts/checks.md +++ b/specs/015-graph-exports/contracts/checks.md @@ -4,13 +4,13 @@ Each check is shown failing on wrong output before it is trusted (constitution I ## SBML comparison (FR-002, FR-003) -A tool (`tools/exports-compare/sbml-compare.ts`) that compares an exporter's SBML with a reference file, wherever the file came from: the exporter's own run, or the website's answer. +`sbml-exporter`'s `ci/compare.py` (its spec 001, T016), which compares two folders of SBML, wherever the files came from: the exporter's own run, or the website's answer. It replaces the tool this spec first planned here. -- **The model:** byte for byte, after removing the generation note's timestamp. What the layout changes in the model (the drawn species' SBO terms, the root's `layout:required` and layout namespace; FR-002) is compared with the layout, not here. The reference is the corrected graph-only converter's file (research R1). +- **The model:** element by element with today's relational file of the same release: compartments, species, reactions, reactants, products and modifiers (role, stoichiometry), names, SBO terms, notes, annotations and identifier URLs. Set aside: the generation note's timestamp, element order (today's is HashSet order; elements are matched by id) and the layout, compared below. - **The layout:** the glyph sets, ids, roles, order and targets exactly, compared with today's file. Coordinates are compared by meaning: today's equal the diagram's plus one offset, except at the nodes Java resized, which are listed. - **Inputs:** the 30-event sample (pathways of several sizes, reactions, disease and inferred events), drawn once from a private graph of the same release (research R8). - **Output:** each difference, by element path. Personal values (vCard names) appear as hashes. -- **Shown failing:** on a file with a species reference removed, a sboTerm changed, and a glyph moved. +- **Shown failing:** on a file with a species reference removed, a sboTerm changed, and a glyph moved (the glyph once the layout is compared, its T021). ## BioPAX comparison (FR-005, FR-009) diff --git a/specs/015-graph-exports/contracts/differences.md b/specs/015-graph-exports/contracts/differences.md index b401ae70..012f2960 100644 --- a/specs/015-graph-exports/contracts/differences.md +++ b/specs/015-graph-exports/contracts/differences.md @@ -2,12 +2,7 @@ Each way the new exports differ from their reference, with the reason. An export switches only when every difference its sample shows is here. Additions found during the work are recorded with the pull request that finds them. -## SBML, from the corrected graph-only reference - -| Difference | Reason | -| ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------- | -| The four defects of the graph-only converter are corrected, in the harness ([reference-harness.md](../reference-harness.md)) (one reactant and one product per reaction; catalysts and negative regulators filled from the positive regulators; positive regulators never written) | The reference is the corrected converter; the uncorrected one would lose participants that today's file has (research R1) | -| The generation note's timestamp | It differs on every answer, today too | +SBML's model is compared with today's relational file of the same release (decided 24 Oct 2026; there is no separate corrected-converter reference any more). The one list for SBML is `sbml-exporter`'s [`specs/001-graph-only-sbml/differences.md`](https://github.com/reactome/sbml-exporter/blob/graph-only/specs/001-graph-only-sbml/differences.md), which its T016 creates; the rows below move there then. The generation note's timestamp is set aside, as it differs on every answer, today's too. ## SBML, from today's relational file diff --git a/specs/015-graph-exports/data-model.md b/specs/015-graph-exports/data-model.md index cf03c845..cf5ac5b8 100644 --- a/specs/015-graph-exports/data-model.md +++ b/specs/015-graph-exports/data-model.md @@ -17,7 +17,7 @@ What BioPAX3Exporter must write (spec FR-005), and where each value is in the gr - **compartments.** - **BioPAX document**: one per event. RDF/XML. Individuals carry `rdf:ID="{Class}{n}"` and are written in a deterministic order. `xml:base` is `http://www.reactome.org/biopax/{release}/{topDbId}#`. - **Reference answer**: what each export is compared with: - - SBML's model: the corrected graph-only converter's file, exactly; + - SBML's model: today's relational file of the same release, element by element, order set aside; - SBML's layout: today's file (structure exactly, coordinates by meaning); - BioPAX: today's file, canonicalised. - **Accepted difference**: a way the new output differs from its reference, with the reason, recorded in [contracts/differences.md](contracts/differences.md). An export switches only when every difference found is in that record. @@ -53,5 +53,5 @@ The first two are recorded differences. ## Validation rules (from the requirements) -- **SBML:** byte-identical to the reference apart from the layout, the SBO terms the layout sets, the root's `layout:required` and layout namespace, and the generation note (FR-002). The layout has the same glyphs, ids, roles and order as today's (FR-003). libSBML reports 0 errors (FR-008). +- **SBML:** the model matches today's relational file element by element, apart from element order, the generation note's timestamp and recorded differences (FR-002). The layout has the same glyphs, ids, roles and order as today's (FR-003). libSBML reports 0 errors (FR-008). - **BioPAX:** the canonical triples equal today's, apart from recorded differences (FR-005). The BioPAX validator reports 0 errors (FR-008). diff --git a/specs/015-graph-exports/plan.md b/specs/015-graph-exports/plan.md index 6a78fd54..565ae9cb 100644 --- a/specs/015-graph-exports/plan.md +++ b/specs/015-graph-exports/plan.md @@ -21,7 +21,6 @@ Each format gets one implementation, reading the graph, used by both the release - **The website** gets both from spec 010's Java library service, fronted by node at the URLs R6 decided, with no fallback (R10). That service is spec 010's to build; this spec changes its SBML operation and adds BioPAX. - **This repository keeps the instruments:** - the sample; - - the corrected SBML reference; - today's answers, and the release's published bulk files; - the comparison tools; - the validator runs. @@ -83,7 +82,7 @@ The changes to the two exporters go to their maintainers: an issue first, then p **Constraints**: -- byte parity for SBML's model; +- SBML's model matching today's relational file element by element; - comparison by meaning for SBML's layout and for BioPAX; - the deep shares; - no personal data in this repository's fixtures, logs or output; @@ -124,7 +123,7 @@ _GATE: Must pass before Phase 0 research. Re-check after Phase 1 design._ - **V. Comments carry the failure.** Every recorded difference states why ([contracts/differences.md](contracts/differences.md)). **Pass.** - **The repository's working rules** (CLAUDE.md, and keeping `RELEASE-TESTING.md` and `CURATOR-REPORT.md` current): - **An explicit exception to "byte-identical to Java's answer".** - - The SBML body is compared with the corrected reference and today's layout, by T030. + - The SBML body is compared with today's relational file, model and layout, by `sbml-exporter`'s `ci/compare.py` (T030). - The BioPAX body has no Java counterpart at its URL, and is compared by T031. - Status, headers and errors are still compared live with the parity harness. - **T054 updates the two documents** when the site's links move. @@ -177,7 +176,7 @@ projects/…/download-tab/, …/detail-download-bar/ # the BioPAX links ## Order of work 1. **Done:** - - the sample, the corrected SBML reference made from a private graph copy (since removed; T032 recreates it), and today's answers (T001–T004); + - the sample, the SBML reference harness that found the converter's defects, and today's answers (T001–T004); - the interim BioPAX link (T009, #634); - the issue drafts (T027); - the amendments to spec 010 (T043). diff --git a/specs/015-graph-exports/quickstart.md b/specs/015-graph-exports/quickstart.md index 82e95339..39d2aa7c 100644 --- a/specs/015-graph-exports/quickstart.md +++ b/specs/015-graph-exports/quickstart.md @@ -20,11 +20,11 @@ Each exporter's own CI runs its validator over its output for a small fixture gr ## 3. SBML against the reference -1. Generate the reference files once, for the 30-event sample, with the corrected graph-only converter (research R1). Run it against a private graph database of the same release where possible (research R8). The references stay outside the repository: they hold personal names. +1. Get today's relational files for the 30-event sample from a ContentService of the same release that is not production, one event at a time (research R1). They stay outside the repository: they hold personal names. 2. Run the fixed `sbml-exporter` over each event, from the private graph copy (or, once switched, ask the website). -3. Run the SBML comparison tool over the pairs. +3. Run `sbml-exporter`'s `ci/compare.py` over the two folders. -Expected: the model is identical apart from the generation note and what the layout changes in the model (FR-002). The layout has the same glyphs, ids, roles and order as today's file. The coordinate differences are only those [differences.md](contracts/differences.md) lists. +Expected: the model matches element by element apart from element order, the generation note's timestamp and what the layout changes in the model (FR-002). The layout has the same glyphs, ids, roles and order as today's file. The coordinate differences are only those [differences.md](contracts/differences.md) lists. ## 4. BioPAX against today's file diff --git a/specs/015-graph-exports/research.md b/specs/015-graph-exports/research.md index 33eec63f..ff390513 100644 --- a/specs/015-graph-exports/research.md +++ b/specs/015-graph-exports/research.md @@ -38,7 +38,7 @@ Personal data: SBML's History (creators) and BioPAX's citations and curation com **Decision.** -- **The reference** for FR-002 is the graph-only converter with the four defects corrected (a new `Participant` per row; catalysts read from their own column; positive regulators set; negative regulators set from their own column) and `setUseIdentifierURL(true)`. +- **The reference** for FR-002 was first the graph-only converter with the defects corrected and `setUseIdentifierURL(true)` (the harness, [reference-harness.md](reference-harness.md)). **Changed 24 Oct 2026:** it is today's relational file of the same release, compared element by element, since the files people use are today's and their order (HashSet order) cannot be reproduced. The harness stays as the record of how the defects were found. - **It is run by a harness** of about 40 lines of Java (a throwaway module, depending on sbml-exporter and graph-core): `findById`, then `SbmlConverter(event, version, ads).convert().toString()`, written as UTF-8. It runs on JDK 11 with UTC, en_US and UTF-8. - **The fixes are offered upstream**, and recorded as accepted deviations. diff --git a/specs/015-graph-exports/spec.md b/specs/015-graph-exports/spec.md index 2a8127f8..e905444b 100644 --- a/specs/015-graph-exports/spec.md +++ b/specs/015-graph-exports/spec.md @@ -22,7 +22,7 @@ Each exporter reads the graph and lives in its own repository: `reactome/sbml-ex What the first version produced is kept and used as the instruments for proving the exporters: - the 30-event sample ([sample.md](sample.md)); -- the corrected SBML reference and its harness ([reference-harness.md](reference-harness.md)); +- the SBML reference harness, which found the graph-only converter's defects ([reference-harness.md](reference-harness.md)); - today's answers from the beta host ([today.md](today.md)); - the BioPAX mapping ([data-model.md](data-model.md)); - the comparison and validator decisions ([research.md](research.md) R1, R3–R5, R8). @@ -72,11 +72,11 @@ The people served are: A researcher downloads SBML for one pathway or reaction from the website, or the whole set from the download page. Either way the file comes from the same exporter, reading the graph. It has the same model as today, its species, reactions, compartments, identifiers and annotations, and a layout drawn from the diagram the website shows. Making it needs no relational database. -**Why this priority**: the model has an exact reference (the corrected graph-only converter, already built), and the exporter already has a graph-only half. It is the smaller of the two proofs, and it retires the relational database from both the release's SBML and the website's. +**Why this priority**: the model has a reference to match (today's relational file of the same release), and the exporter already has a graph-only half. It is the smaller of the two proofs, and it retires the relational database from both the release's SBML and the website's. **Independent Test**: -- **The model.** For the 30-event sample, the exporter's model is byte-identical to the corrected reference, generation note aside. +- **The model.** For the 30-event sample, the exporter's model matches today's relational file element by element, element order and the generation note's timestamp aside. - **The layout.** It matches today's by meaning (R3). - **The bulk run.** With no relational database configured, it produces the same set of files under the same names as the release's published archives in its download directory (`download.reactome.org/{release}/`), and a sample of its files compares with theirs. - **Validity.** Every file passes the SBML consistency check. @@ -84,7 +84,7 @@ A researcher downloads SBML for one pathway or reaction from the website, or the **Acceptance Scenarios**: -1. **Given** a pathway or a reaction, **When** the exporter writes its SBML, **Then** everything but the layout and the generation note is byte-identical to the corrected reference. +1. **Given** a pathway or a reaction, **When** the exporter writes its SBML, **Then** its model matches today's relational file element by element, apart from element order, the generation note's timestamp and recorded differences. 2. **Given** a pathway with its own diagram drawing a reaction, **When** its SBML is written, **Then** the layout places and connects every element today's does, with the same identifiers, roles and order, at the diagram's own coordinates. 3. **Given** a release, **When** the bulk SBML is made with no relational database, **Then** both archives hold the same file names as that release's published archives, and every file is valid. 4. **Given** the website's SBML link, **When** it is followed, **Then** the status, content type, file name and errors are today's, and the file is the exporter's. @@ -152,7 +152,7 @@ The people who run a release can make the SBML and BioPAX bulk files without the **SBML (`reactome/sbml-exporter`)** - **FR-001**: The graph-only converter MUST be corrected: the five defects (R1, R9) fixed, with tests that fail without each fix. With those fixes it is the SBML model for both the bulk files and the website. -- **FR-002**: Its model MUST be byte-identical to the corrected reference (R1) for every event in the sample, apart from the generation note and what the layout changes in the model: the drawn species' SBO terms, and the root's `layout:required` and layout namespace (R3). +- **FR-002**: Its model MUST match today's relational file of the same release (R1) for every event in the sample, element by element (compartments, species, reactions, participants with role and stoichiometry, names, SBO terms, notes, annotations and identifier URLs), apart from element order, the generation note's timestamp and what the layout changes in the model (the drawn species' SBO terms, and the root's `layout:required` and layout namespace; R3), or each difference recorded with its reason. - **FR-003**: It MUST write the layout from the diagram files the site draws (R3), for the events that have one today and only those. It MUST place and connect the same elements, with the same identifiers, roles and order, as today's layout. Coordinate differences are recorded and accepted (R3). - **FR-004**: Its command line MUST make the bulk files with no relational database. The relational options become unnecessary, and the file names and archives stay today's. @@ -189,7 +189,7 @@ The people who run a release can make the SBML and BioPAX bulk files without the - **Event**: a pathway or a reaction. It is what one on-demand file describes. - **Species file**: one species' whole BioPAX, or the set of per-event SBML files in an archive. - **Reference**: what an exporter is compared with: - - for SBML's model, the corrected graph-only converter (exact); + - for SBML's model, today's relational file (element by element); - for SBML's layout, today's layout (by meaning); - for BioPAX, today's Pathway-Exchange files (by meaning): the release's published `biopax.zip` for the species files, and the legacy service's answer for one event. - **Accepted difference**: a recorded way the new output may differ from its reference, with the reason. Nothing switches until every difference found is recorded. diff --git a/specs/015-graph-exports/tasks.md b/specs/015-graph-exports/tasks.md index d108bb32..52276205 100644 --- a/specs/015-graph-exports/tasks.md +++ b/specs/015-graph-exports/tasks.md @@ -86,12 +86,7 @@ Both exporters' adoption pull requests found that their own checks do not run ye - [x] T028 Post the two issues once the user has read and approved them, and record their links here: [BioPAX3Exporter#4](https://github.com/reactome/BioPAX3Exporter/issues/4) and [sbml-exporter#14](https://github.com/reactome/sbml-exporter/issues/14), posted on 24 Oct 2026. Spec Kit adoption pull requests: [BioPAX3Exporter#5](https://github.com/reactome/BioPAX3Exporter/pull/5) and [sbml-exporter#15](https://github.com/reactome/sbml-exporter/pull/15). — #648 - [ ] T029 [ext] Agree graph-core and the JDK with both exporters' maintainers: each works on the graph-core version spec 010's library service pins, on JDK 17 (R10). Record the version here. -- [ ] T030 [P] Write the SBML comparison tool, `tools/exports-compare/sbml-compare.ts`, with its spec (`contracts/checks.md`): - - **the model:** byte for byte, the generation note's timestamp set aside, and the layout's changes to the model (FR-002) compared with the layout; - - **the layout:** glyph sets, ids, roles, order and targets exactly; coordinates by meaning, against the diagram plus one offset, with the resized nodes listed; - - **output:** element paths, with vCard names as hashes. - - Show it failing on a removed species reference, a changed sboTerm and a moved glyph. +- [ ] T030 [P] [ext] The SBML comparison tool is `sbml-exporter`'s `ci/compare.py` (its spec 001, T016 for the model, T021 for the layout), not a tool here (decided 24 Oct 2026). It compares element by element with today's relational file of the same release, order and the timestamp set aside; it is shown failing on a removed species reference, a changed sboTerm and, for the layout, a moved glyph. Tick this when its T021 has merged. - [ ] T031 [P] Write the BioPAX comparison tool, `tools/exports-compare/biopax-compare.ts`, with its spec: - each file canonicalised to (class, property, target-key) triples, with `rdf:ID`s replaced by content keys (R4); @@ -100,7 +95,7 @@ Both exporters' adoption pull requests found that their own checks do not run ye It takes a species file as well as an event's. Show it failing on a changed stoichiometry and a removed xref. -- [ ] T032 The private graph copy (R8), for the release being compared, recreated whenever a comparison needs it and removed afterwards. If beta's graph has moved to a newer release by then, regenerate the SBML references (T003) and today's per-event answers (T004) for that release, and compare with that release's download directory. +- [ ] T032 The private graph copy (R8), for the release being compared, recreated whenever a comparison needs it and removed afterwards. If beta's graph has moved to a newer release by then, regenerate today's per-event answers (T004) for that release, and compare with that release's download directory. - [ ] T033 (`biopax.zip` done for release 97, recorded in `today.md`; the SBML archives when T041 needs them.) Fetch the release's bulk references from its download directory, `download.reactome.org/{release}/`: `biopax.zip`, `all_species.3.1.sbml.tgz` and `homo_sapiens.3.1.sbml.tgz`. One at a time, kept outside the repository (they hold people's names). Record each file's size, digest and list of entries in `today.md`, never their content. - [ ] T034 Run the validators on today's files and record the baselines in research.md R5: - libSBML 5.21.2's `checkConsistency()` over T004's SBML and a sample of T033's archive, with the units and modelling-practice categories off; @@ -132,13 +127,13 @@ Both exporters' adoption pull requests found that their own checks do not run ye **Independent Test**: -- the sample's models are byte-identical to the corrected reference; +- the sample's models match today's relational files element by element; - the layouts match today's by meaning; - a graph-only bulk run matches the release's published archives in file names and in a sample of files; - the website's SBML comes from the exporter. -- [ ] T036 [US1] [ext] In `sbml-exporter`: fix the five defects (R9), each with a test that fails without its fix. Add tests for the two queries' row mapping. -- [ ] T037 [US1] Compare the fixed exporter's models with the references over the whole sample (T030), from T032's copy. Byte-identical apart from the timestamp. Record any difference in `contracts/differences.md`, or send it back to T036. +- [x] T036 [US1] [ext] In `sbml-exporter`: fix the five defects (R9), each with a test that fails without its fix. Add tests for the two queries' row mapping. — sbml-exporter #22, #23, #24 (into its `graph-only` branch) +- [ ] T037 [US1] Compare the fixed exporter's models with today's relational files over the whole sample (T030), from T032's copy: element by element, order and the timestamp set aside. Done as `sbml-exporter`'s T016; record the result here. - [ ] T038 [US1] [ext] In `sbml-exporter`: the layout from the diagram files. `LayoutConverter`'s algorithm, ported over `diagram/{stId}.json` of the same release (R3), keeping: - the same ids, order and glyph sets; - the same roles and backbone rules; @@ -146,8 +141,8 @@ Both exporters' adoption pull requests found that their own checks do not run ye It is written only for an event whose own diagram draws a reaction. -- [ ] T039 [US1] Compare the layouts with today's files (T004) over the sample's events with diagrams: glyph sets, ids, roles, order and targets exactly; coordinates against the recorded offset and the resized nodes. Compare every event's model again, outside the layout, byte for byte (FR-002). Record each difference in `contracts/differences.md`. -- [ ] T040 [US1] [ext] In `sbml-exporter`: a graph-only command line. The relational options are dropped. The graph converter with its layout is used for listed ids, species and `all`, writing every pathway of a species as today. File names stay `{stId}.sbml`, and the archives stay as today. +- [ ] T039 [US1] Compare the layouts with today's files (T004) over the sample's events with diagrams: glyph sets, ids, roles, order and targets exactly; coordinates against the recorded offset and the resized nodes. Compare every event's model again, element by element (FR-002). Record each difference in `contracts/differences.md`. +- [ ] T040 [US1] [ext] In `sbml-exporter`: a graph-only command line. The relational options are dropped (done in sbml-exporter #30, with no switch: its branch reads nothing relational; the diagram folder is left). The graph converter with its layout is used for listed ids, species and `all`, writing every pathway of a species as today. File names stay `{stId}.sbml`, and the archives stay as today. - [ ] T041 [US1] A graph-only bulk dry run for one species, from T032's copy, compared with the release's published archive (T033): the same file names, every file valid (libSBML), and a sample of files compared as in T037 and T039. Record the run's time. - [ ] T042 [US1] [ext] In `sbml-exporter`'s CI: libSBML's consistency check over the converter's output for a small fixture graph, failing on any error. - [x] T043 [US1] Spec 010 amended for this spec (R10):