Skip to content

Decouple orf_class from ORF length across the custom/orf* modules - #12498

Open
FelixKrueger wants to merge 3 commits into
nf-core:masterfrom
FelixKrueger:feat/orf-class-positional-vocabulary
Open

Decouple orf_class from ORF length across the custom/orf* modules#12498
FelixKrueger wants to merge 3 commits into
nf-core:masterfrom
FelixKrueger:feat/orf-class-positional-vocabulary

Conversation

@FelixKrueger

Copy link
Copy Markdown
Contributor

custom/orfnormalise overwrote orf_class with smORF for every ORF at or under 100 aa, so a short uORF, a short downstream ORF, a short truncated CDS variant and a short novel intergenic ORF were indistinguishable in exactly the size range microprotein work cares about — and because custom/orfmerge keys its clustering strategy on orf_class and custom/orfcollapse scoped its peptide dedup to that value, the length override also decided how an ORF merged across callers. orf_class is now purely positional and length moves to a separate is_smorf flag driven by a new --smorf-max-aa (default 100, which reproduces the previous membership exactly), and the vocabulary gains uoORF, doORF and intORF — classes the callers were already reporting and the substring matching was folding away. This is a breaking change to orf_class values; it needs the refreshed fixture from nf-core/test-datasets#2193, which is merged.

AI-assisted detail — keep or bin as you see fit

Why the classes were being lost

Tokens were matched by substring, so overlap_uORF matched the uorf rule. Verified against each caller's source:

caller native label was now
RiboCode Overlap_uORF / Overlap_dORF uORF / dORF uoORF / doORF
ribotricer overlap_uORF / overlap_dORF uORF / dORF uoORF / doORF
RiboCode, Ribo-TISH internal / Internal other intORF
PRICE uoORF, iORF, orphan uORF, other, other uoORF, intORF, novel_u

Matching is now exact per caller — every vocabulary is a closed enum — and the count of tokens matching no entry is reported as unmapped_orf_type on the # parser_columns: provenance line, so an upstream label change fails loudly instead of landing in the wrong class. It is 0 for all five callers on the chr20 fixtures.

ribotricer's internal deliberately stays other: it is check_orf_type()'s terminal fall-through rather than a frame-tested call, so intORF would assert a relationship the tool never tested.

Why orf_class had to leave the merge key

Callers disagree on class for the same ORF, and Ribo-TISH cannot agree even in principle — tisType() tests only the start position, so 5'UTR covers both uORF and uoORF. Keying on class emits one row per disagreeing caller. orfmerge now partitions by clustering strategy with class absent from every grouping key, and representative() resolves the disagreement via CLASS_SPECIFICITY (most specific first). There is a new test where four callers report the same ORF as Overlap_uORF, overlap_uORF, uoORF and 5'UTR and collapse to one uoORF row.

Two further clustering fixes fell out of that:

  • Overlap is measured on summed exon-block intersection, not the outer genomic span. For a spliced ORF the span is mostly intron, so span-based overlap is insensitive to real differences — two ORFs differing by a quarter of their codons could score >0.99.
  • Linkage is complete, not single. Single linkage chained distinct ORFs through an intermediate overlapping both; on a synthetic fixture one row absorbed a uORF, a uoORF, a doORF, an internal ORF and an extended CDS.

canonical_cds is grouped by transcript and then overlap-clustered within it, so a short truncated variant is no longer folded into the full-length CDS and emitted as the longest member only. Transcript-anchored classes keep the exact-span key: overlap clustering merges nested ORFs, and a 100 aa uORF covering 84% of its CDS folded into the CDS when I tried it. Separating "same ORF, fuzzy bounds" from "different ORF, mostly nested" needs a shared-stop-codon test, which I have left for a follow-up. The consequence — two callers whose short-ORF bounds differ by a few nt give one row each — is pinned by a test.

Unknown orf_class values and unassigned rows now abort rather than being dropped from the catalogue silently, which the previous closed enumeration did.

orfcollapse

Eligibility is derived from aa_length plus --smorf-max-aa rather than a propagated flag, so the collapse scope cannot drift from the published is_smorf, and the module aborts if the columns it depends on are missing. The survivor of a cross-class peptide fold now prefers the more specific class: previously it was chosen by length alone, so a 40 aa annotated CDS was deleted by a 60 aa novel ORF sharing its peptide and the MultiQC table reported zero canonical_cds.

New column

orf_type_native carries each caller's own ORF-type label through to the catalogue (unioned across a cluster), so every mapping decision above is auditable without re-running callers.

Testing

Nine tests on orfnormalise, seven on orfmerge, two on orfcollapse. New: the is_smorf boundary at --smorf-max-aa and at +1 with orf_class unchanged; a non-default --smorf-max-aa override; the four-caller class-disagreement merge; canonical_cds variant separation; and the pinned short-ORF de-merge. The two synthetic tests from #12495 gain the two new columns — without them the new presence guard aborts.

Snapshots are re-recorded. Content md5s come from a local run, which is byte-identical to CI for deterministic outputs; versions.yml md5s are kept from the committed snapshots, since those carry the container's interpreter rather than my host's. I could not run the module containers locally (no DNS in my Docker VM), so every test was executed container-free and CI is the authority on that last piece.

custom/orfnormalise overwrote orf_class with "smORF" for every ORF at or under
100 aa, so a short uORF, a short downstream ORF, a short truncated CDS variant
and a short novel intergenic ORF were indistinguishable in exactly the size range
microprotein work cares about. Because custom/orfmerge keys its clustering
strategy on orf_class and custom/orfcollapse scoped its peptide dedup to that
value, the length override also decided how an ORF merged across callers.

orf_class is now purely positional, and length moves to a separate is_smorf flag
driven by a new --smorf-max-aa (default 100, reproducing the previous membership
exactly). The vocabulary gains uoORF, doORF and intORF, which the callers were
already reporting: RiboCode and ribotricer emit Overlap_uORF/Overlap_dORF,
RiboCode and Ribo-TISH emit internal, PRICE emits uoORF and iORF. ORF-type tokens
are now matched exactly rather than by substring, which is what previously folded
overlap_uORF into uORF.

orfmerge partitions by clustering strategy with orf_class absent from every
grouping key, because callers disagree on class for the same ORF -- Ribo-TISH
reports 5'UTR for both uORFs and CDS-overlapping uORFs -- and keying on class
emits one row per disagreeing caller. Reciprocal overlap is measured on summed
exon-block intersection rather than the outer span, which for a spliced ORF is
mostly intron, and linkage is complete so a chain of partial overlaps cannot fold
distinct ORFs together. Unknown classes and unassigned rows now abort.

orfcollapse derives its eligible set from aa_length plus --smorf-max-aa rather
than a propagated flag, and prefers the more specific class when a peptide
cluster spans several, so an annotated CDS is not deleted by a longer novel ORF
sharing its peptide.

Adds orf_type_native, carrying each caller's own label through to the catalogue
so every harmonisation decision stays auditable.

Requires the refreshed orfcollapse fixture from nf-core/test-datasets#2193.
@FelixKrueger
FelixKrueger force-pushed the feat/orf-class-positional-vocabulary branch from 5da87e1 to baab83c Compare August 3, 2026 10:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant