feat: add the fidelity construct descriptor kinds and the layers/attachments/destinations tables - #25
Merged
Merged
Conversation
Six harmonised kinds -- contentControl, field, anchor, link, provenance, division -- each a node payload a package-tree group may carry in place of a container descriptor or an anchor paragraph. The vocabulary is format-agnostic by construction: every kind is confirmed by at least two of the four codec inventories, and a construct with no cross-format analogue degrades to the nearest kind rather than minting a bespoke one. Extents are block-scoped. A construct wraps the block flow of a section, heading group, shape, or list item; it cannot wrap a sub-sequence of one paragraph's runs, because a run-level extent is not expressible without changing ContentParagraph. Run-level constructs keep their existing homes -- an external hyperlink stays on ContentRun.hyperlink, and the link kind is for block-scoped and annotated extents a flat run field cannot express, never a replacement for it. Two harmonisations are deliberately absent. There is no fieldType enum: instruction is required and verbatim, so nothing is lost without one, and a member set frozen from spec recollection would pre-empt the corpus gate all four inventories still flag. There is no residue field on any descriptor: the quarantined source channel is a per-node and package-level facility spanning the whole content model, and a descriptor-only escape hatch would mint exactly the parallel shape it exists to avoid. Refs #22, #24
…ge root Three optional root fields typed as the same generic definitions table the definitions field already uses -- kind-tagged loose entries, so no new entry shape is minted anywhere. layers holds optional-content group definitions (PDF /OCProperties, ODF Draw's layer model); attachments holds package attachments (PDF /Names /EmbeddedFiles, /FileAttachment, /EF, /AF, and their docx and ODF counterparts); destinations holds named destinations and the navigation tree resolving against them (PDF /Dests, the /Names name tree, /Outlines), which is the other end of a link construct's internal target. Separate root fields rather than three more tenants of definitions, for the reason styles is its own field despite being the facility's first tenant: each table is its own key namespace, so a layer and a destination may share a name without colliding. The kind discriminator still earns its keep inside each, because each holds more than one tenant -- a layers table carries group definitions alongside their configuration, and a destinations table carries named destinations alongside outline entries. Layer membership is deliberately not modelled on tree nodes: the inventory that proposed the table places the item-side reference on the producing codec's own item model, and no inventory asks for a layer ref on a content node. Refs #24, ExaDev/pdf-codec#66
…age tree
A construct group is the same { node, children } wrapper every other group uses, with a
construct descriptor as its node and the extent it spans as its children.
Two variants, because a construct is transparent to the flow it wraps and the two block
flows admit different children: the section-scoped one sits in a section's or heading
group's flow and admits heading children, the shape-scoped one sits in a shape's or list
item's and does not -- the SectionChild/ShapeChild split that already existed.
Constructs nest in and around every other group, so a provenance wrapper inside a content
control inside a division validates, and they are not legal as direct children of a slide,
sheet, drawing page, or the package root, which hold containers and leaves rather than
block flow.
Every group guard now tests its own node payload before walking its children.
The order is load-bearing rather than stylistic: a child predicate tries each group arm in
turn, so a guard that walked the subtree first and only then rejected on the node made
each arm pay for the whole subtree before failing, costing arms-per-flow raised to the
power of the tree's depth.
Node payloads discriminate in constant time and no two group kinds share one, so checking
the node first means at most one arm ever descends.
The conjunctions are otherwise unchanged -- both halves are pure predicates, so the order
affects only how much work a rejection does.
A thirty-deep construct chain whose verdict turns on its innermost leaf pins this: it
validates in milliseconds under the current order and does not terminate under the old one.
The published JSON Schema follows the guards key for key: the four block-flow group
fragments gain the construct refs, the two construct wrappers and the whole descriptor
vocabulary are transcribed alongside them, and every descriptor fragment is held to a live
z.toJSONSchema() comparison rather than to hand re-verification, since a descriptor is a
plain strict object reaching no opaque node.
The kinds are additive over 4.0.0: a tree carrying none of them parses unchanged, field for
field.
Refs #24
…decision A Fidelity constructs section covering the six kinds, what each carries, and which producer constructs land on it, plus a worked package showing a tracked insertion inside a content control and a footnote marker whose body lives in the definitions table. The four bounds are recorded as decisions rather than omissions: extents are block-scoped and run-level hyperlinks stay on ContentRun.hyperlink; there are two group variants, one per block flow; division is first-class rather than degraded to a content control; and the quarantined residue channel is deliberately not part of this vocabulary. The division decision is spelled out with its reasoning. ContentSection cannot host an ODF text:section -- that is page geometry, one pageSize and margins pair, and it does not nest, while a division nests arbitrarily and usually changes no page geometry at all -- and burying a structural container in the form-control vocabulary would make contentControl mean two unrelated things. It clears the no-format-specific-kinds bar on a real analogue, tagged PDF's /Sect and /Div, and is spelled division because section already names the page-geometry container descriptor. The definitions-tables section gains the three construct tables and why each is its own root field rather than another tenant of definitions. Refs #22, #24
|
🎉 This PR is included in version 4.1.0 🎉 The release is available on: Your semantic-release bot 📦🚀 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Lands the construct vocabulary the four codec inventories asked for, additively on top of the 4.0.0 tree: new descriptor kinds and package tables, no change to any existing node shape, so a 4.0.0 tree parses unchanged field for field. This is the schema half only — the reading work is gated behind it in ExaDev/ooxml.js#68, ExaDev/odf.js#62, ExaDev/markdown-codec#66, and ExaDev/pdf-codec#69.
Closes #24. Refs #22 — which stays open, because the two-channel model is only half done here: the vocabulary exists, but no codec emits a construct yet, and channel 2 (residue) is not in this PR at all (see below).
What landed
Six construct kinds in
src/construct.ts, each a node payload a tree group may carry in place of a container descriptor or an anchor paragraph. A construct group is the same{ node, children }wrapper as every other group, with the extent it spans as its children — which is what the tree's construct-capable-from-day-one design was for, and why this is a minor rather than a second structural break.contentControlcontrolType,tag,alias,lock,value,checked,optionsw:ffDataform fields, ODFoffice:formscontrols and TOC/index wrappers, PDF AcroForm widgets and field treefieldinstruction,cachedResultw:fldChar/w:instrText/w:fldSimple, ODF field masters and simple fields, ODF cross-reference displays, pptxa:fldanchoranchorType,name,definitiontext:bookmark/text:reference-mark*/text:note/office:annotation, PDF sticky notes and markup annotations, markdown footnote markerslinktarget(external URI or internal anchor name),title@w:anchor, pptx slide jumps, PDFGoTo//Destand link annotations, markdown link/image titlesprovenancechange,author,dateIsow:ins/w:deland move tracking, ODFtext:tracked-changes/text:changed-regiondivisionname,columnCount,protected,sourcetext:sectionandtext:section-source, tagged PDF/Sectand/DivPlus three optional root tables —
layers,attachments,destinations— and the guards, published JSON Schema fragments, and per-feature tests for all of it.The
text:sectionverdict: first-classdivisionExaDev/odf.js#59 proposed exactly one new generic kind and #24 posed it as a decision: a first-class nesting container, or degrade to
contentControlwithtext:name/style:name/protected in residue. First-class, on the inventory's own recommendation and reasoning:ContentSectioncannot host it. That is page geometry — onepageSize/marginspair — and it does not nest, whiletext:sectionnests arbitrarily and usually changes no page geometry at all (columns, protection, naming, linked external chapters).contentControlmean two unrelated things.controlTypeis a form/input vocabulary; a structural container is not an input, and burying one in the other loses the structural meaning the odf inventory calls "the strongest candidate for a first-class structural node"./Sectand/Divstructure elements are the same construct, and the odm reader's chapter model is already section-shaped.Spelled
division, notsection, because{ kind: 'section' }is already the page-geometry container descriptor and one word cannot mean both.divisionis tagged PDF's own name for the shape.The other design calls, and why
Extents are block-scoped. A construct wraps the block flow of a section, heading group, shape, or list item. It cannot wrap a sub-sequence of one paragraph's runs, because a run-level extent is not expressible without changing
ContentParagraph, and this release changes no content shape. Consequences, stated plainly: run-level external hyperlinks stay onContentRun.hyperlink(the standing reconciliation on #22 —linkgroups are for block-scoped and annotated extents a flat run field cannot express, never a replacement), and the inline cases the inventories name — a docxPAGEfield mid-paragraph, a mid-paragraph bookmark, an inline tracked change, pptxa:fldinside a run — wait on a run-level extent mechanism rather than being forced into a block wrapper that would split their paragraph. The block-scoped half (docx TOC-shaped fields, block SDTs, comment extents, tracked block insertions and deletions, divisions) lands now.Two group variants, one per block flow.
SectionConstructGroupadmits heading children,ShapeConstructGroupdoes not — theSectionChild/ShapeChildsplit that already existed, since a shape carries no heading hierarchy. A list item's flow takes the shape-scoped variant, becauseListChildandShapeChildhave always admitted the same members. Constructs nest in and around every other group (aprovenanceinside acontentControlinside adivisionis a real docx shape), and are not legal as direct children of a slide, sheet, drawing page, or the package root — those hold containers and leaves, not block flow.Three tables, not three shapes.
layers/attachments/destinationsare typed as the same generic definitions tabledefinitionsalready uses, so no new entry shape is minted. They are separate root fields rather than three more tenants ofdefinitionsfor the reasonstylesis its own field despite being the facility's first tenant: separate key namespaces, so a layer and a destination may share a name. Thekinddiscriminator still earns its keep inside each, because each holds more than one tenant — layer definitions alongside their configuration, named destinations alongside outline entries (#24 names that table "navigation/destinations" for exactly that reason).No layer ref on tree nodes. ExaDev/pdf-codec#66 asks for "a layer reference on the extracted items/frames" — those are pdf-codec's own item model, which it owns outright since 4.0.0. No inventory asks for a layer ref on a content node, so the table carries definitions only; membership can land additively as a group-wrapper field when a codec actually needs it.
No
fieldTypeenum.instructionis required and verbatim, so nothing is lost without one. #24 asks for exactly one new vocabulary —link's internal targets — rather than one per kind, and the corpus gate all four inventories flag still stands: several field families have no real producer fixture in any repo, so a member set frozen now would be frozen from spec recollection. Adding one later is schema-additive.No residue channel. #22's channel 2 is a per-node and package-level
source: { format, xml }facility spanning the whole content model. Descriptors here are closed objects with no escape hatch, deliberately — a descriptor-only residue field would mint exactly the parallel shape the general facility exists to avoid. That work stays on #22.Sheet-scoped named ranges are definitions entries, not anchors. ExaDev/ooxml.js#65 verdicts xlsx defined names and tables as
anchor. That is unrealisable in the tree: a sheet group's children are its images and embedded documents, never a block flow, so there is no extent for an anchor to wrap. They ride a definitions-table entry naming their range — which is ExaDev/odf.js#59's own verdict for the identical construct (table:named-expressions).Guard ordering, fixed as part of this
Every group guard now tests its own node payload before walking its children. The order is load-bearing, not stylistic: a child predicate tries each group arm in turn, so a guard that walked the subtree first and only then rejected on the node made each arm pay for the whole subtree before failing — cost was arms-per-flow raised to the power of the tree's depth, and this PR adds a third arm to two of those predicates. Node payloads discriminate in constant time and no two group kinds share one, so checking the node first means at most one arm ever descends. The conjunctions are otherwise unchanged (both halves are pure predicates), and a thirty-deep construct chain whose verdict turns on its innermost leaf pins it: milliseconds under the current order, non-terminating under the old one (verified by reverting the order — the test times out).
Verification
pnpm lint,pnpm typecheck,pnpm build,pnpm test(265),pnpm test:workers(4),pnpm test:smoke(8) all green, uncached. Every commit is green in isolation, so the history bisects. Both published.schema.jsonfiles resolve every local$refwith nothing dangling, and the README's worked example was executed against the built output rather than written from memory.Not in scope, and where each is tracked
Everything else in #24's "as fields rather than kinds" list is a
ContentDocumentshape change, which this minor deliberately does not make: the chart union member onContentEmbeddedObject, the checked/task field and item identity on list membership, the blockquote container, the code-language annotation, and theContentSectionbreak-type and header/footer fields. Floating-image placement stays flagged contestable as ExaDev/ooxml.js#65 left it. Two contested rows need no schema change at all and can land as tenant entries today: xlsx conditional formatting and ODFcalcext:conditional-formatas aconditionalStyledefinitions entry, and PDF/PageLabelsas a numbering entry. The corpus gate every inventory flags — checking each semantic candidate against one real producer file before shapes freeze — is unchanged and still stands on the codec issues.Generated by Claude Code