Decouple orf_class from ORF length across the custom/orf* modules - #12498
Open
FelixKrueger wants to merge 3 commits into
Open
Decouple orf_class from ORF length across the custom/orf* modules#12498FelixKrueger wants to merge 3 commits into
FelixKrueger wants to merge 3 commits into
Conversation
custom/orfnormalise overwrote orf_class with "smORF" for every ORF at or under 100 aa, so a short uORF, a short downstream ORF, a short truncated CDS variant and a short novel intergenic ORF were indistinguishable in exactly the size range microprotein work cares about. Because custom/orfmerge keys its clustering strategy on orf_class and custom/orfcollapse scoped its peptide dedup to that value, the length override also decided how an ORF merged across callers. orf_class is now purely positional, and length moves to a separate is_smorf flag driven by a new --smorf-max-aa (default 100, reproducing the previous membership exactly). The vocabulary gains uoORF, doORF and intORF, which the callers were already reporting: RiboCode and ribotricer emit Overlap_uORF/Overlap_dORF, RiboCode and Ribo-TISH emit internal, PRICE emits uoORF and iORF. ORF-type tokens are now matched exactly rather than by substring, which is what previously folded overlap_uORF into uORF. orfmerge partitions by clustering strategy with orf_class absent from every grouping key, because callers disagree on class for the same ORF -- Ribo-TISH reports 5'UTR for both uORFs and CDS-overlapping uORFs -- and keying on class emits one row per disagreeing caller. Reciprocal overlap is measured on summed exon-block intersection rather than the outer span, which for a spliced ORF is mostly intron, and linkage is complete so a chain of partial overlaps cannot fold distinct ORFs together. Unknown classes and unassigned rows now abort. orfcollapse derives its eligible set from aa_length plus --smorf-max-aa rather than a propagated flag, and prefers the more specific class when a peptide cluster spans several, so an annotated CDS is not deleted by a longer novel ORF sharing its peptide. Adds orf_type_native, carrying each caller's own label through to the catalogue so every harmonisation decision stays auditable. Requires the refreshed orfcollapse fixture from nf-core/test-datasets#2193.
FelixKrueger
force-pushed
the
feat/orf-class-positional-vocabulary
branch
from
August 3, 2026 10:20
5da87e1 to
baab83c
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
custom/orfnormaliseoverwroteorf_classwithsmORFfor every ORF at or under 100 aa, so a short uORF, a short downstream ORF, a short truncated CDS variant and a short novel intergenic ORF were indistinguishable in exactly the size range microprotein work cares about — and becausecustom/orfmergekeys its clustering strategy onorf_classandcustom/orfcollapsescoped its peptide dedup to that value, the length override also decided how an ORF merged across callers.orf_classis now purely positional and length moves to a separateis_smorfflag driven by a new--smorf-max-aa(default 100, which reproduces the previous membership exactly), and the vocabulary gainsuoORF,doORFandintORF— classes the callers were already reporting and the substring matching was folding away. This is a breaking change toorf_classvalues; it needs the refreshed fixture from nf-core/test-datasets#2193, which is merged.AI-assisted detail — keep or bin as you see fit
Why the classes were being lost
Tokens were matched by substring, so
overlap_uORFmatched theuorfrule. Verified against each caller's source:Overlap_uORF/Overlap_dORFuORF/dORFuoORF/doORFoverlap_uORF/overlap_dORFuORF/dORFuoORF/doORFinternal/InternalotherintORFuoORF,iORF,orphanuORF,other,otheruoORF,intORF,novel_uMatching is now exact per caller — every vocabulary is a closed enum — and the count of tokens matching no entry is reported as
unmapped_orf_typeon the# parser_columns:provenance line, so an upstream label change fails loudly instead of landing in the wrong class. It is 0 for all five callers on the chr20 fixtures.ribotricer's
internaldeliberately staysother: it ischeck_orf_type()'s terminal fall-through rather than a frame-tested call, sointORFwould assert a relationship the tool never tested.Why
orf_classhad to leave the merge keyCallers disagree on class for the same ORF, and Ribo-TISH cannot agree even in principle —
tisType()tests only the start position, so5'UTRcovers both uORF and uoORF. Keying on class emits one row per disagreeing caller.orfmergenow partitions by clustering strategy with class absent from every grouping key, andrepresentative()resolves the disagreement viaCLASS_SPECIFICITY(most specific first). There is a new test where four callers report the same ORF asOverlap_uORF,overlap_uORF,uoORFand5'UTRand collapse to oneuoORFrow.Two further clustering fixes fell out of that:
canonical_cdsis grouped by transcript and then overlap-clustered within it, so a short truncated variant is no longer folded into the full-length CDS and emitted as the longest member only. Transcript-anchored classes keep the exact-span key: overlap clustering merges nested ORFs, and a 100 aa uORF covering 84% of its CDS folded into the CDS when I tried it. Separating "same ORF, fuzzy bounds" from "different ORF, mostly nested" needs a shared-stop-codon test, which I have left for a follow-up. The consequence — two callers whose short-ORF bounds differ by a few nt give one row each — is pinned by a test.Unknown
orf_classvalues and unassigned rows now abort rather than being dropped from the catalogue silently, which the previous closed enumeration did.orfcollapse
Eligibility is derived from
aa_lengthplus--smorf-max-aarather than a propagated flag, so the collapse scope cannot drift from the publishedis_smorf, and the module aborts if the columns it depends on are missing. The survivor of a cross-class peptide fold now prefers the more specific class: previously it was chosen by length alone, so a 40 aa annotated CDS was deleted by a 60 aa novel ORF sharing its peptide and the MultiQC table reported zerocanonical_cds.New column
orf_type_nativecarries each caller's own ORF-type label through to the catalogue (unioned across a cluster), so every mapping decision above is auditable without re-running callers.Testing
Nine tests on orfnormalise, seven on orfmerge, two on orfcollapse. New: the
is_smorfboundary at--smorf-max-aaand at +1 withorf_classunchanged; a non-default--smorf-max-aaoverride; the four-caller class-disagreement merge;canonical_cdsvariant separation; and the pinned short-ORF de-merge. The two synthetic tests from #12495 gain the two new columns — without them the new presence guard aborts.Snapshots are re-recorded. Content md5s come from a local run, which is byte-identical to CI for deterministic outputs;
versions.ymlmd5s are kept from the committed snapshots, since those carry the container's interpreter rather than my host's. I could not run the module containers locally (no DNS in my Docker VM), so every test was executed container-free and CI is the authority on that last piece.