diff --git a/README.md b/README.md index 0484c75..613753b 100644 --- a/README.md +++ b/README.md @@ -48,7 +48,7 @@ graph TD style schema fill:#f9a825,stroke:#333,stroke-width:3px ``` -`ContentDocument` (the semantic pivot) is a discriminated union of five kinds: `wordprocessing` (docx/odt sections of paragraphs/runs/tables/images), `presentation` (pptx/odp slides of shapes), `spreadsheet` (xlsx/ods sheets of cells, columns, rows, print settings), `drawing` (odg pages of shapes plus vector primitives — rect/ellipse/line/path), and `formula` (an equation carrying its own MathML node tree plus StarMath source when the producing format had one, extended with the two-layer math model: an optional verbatim-LaTeX `presentation` authoritative for rendering, an optional semantic `content: MathExpression` tree authoritative for computation, and provenance — neither layer stored derived from the other, so editing one never silently mutates the other). `ContentEmbeddedObjectSchema` lets any of the five embed another whole `ContentDocument`. Every paragraph/run/image/table/shape/vector/spreadsheet-cell leaf also carries its own canonical `headingLevel`-or-position fields directly: a `ContentParagraph`'s optional `headingLevel` (1 = the outermost heading, independent of the round-trip-only `styleId`), and every such leaf's optional `frames: LayoutFrame[]` — that node's own rendered page position(s) (`pageIndex` plus PDF user-space `xPt`/`yPt`/`widthPt`/`heightPt`), fused directly onto the content tree once a layout pass has run. `DocumentPackage` is the single hierarchical artefact — structure, layout, and content fused in one tree (see [The package tree](#the-package-tree)): the root carries `kind`, `metadata`, the optional document-level `symbolTable` and rendered `pages`, the optional package-level `styles`/`definitions` tables (see [Definitions tables and styles](#definitions-tables-and-styles)), and `children` — one group per top-level container with the content tree grouped inside it; the schema does not keep populated `frames` fields and `pages` in sync or detect staleness, and does not check that a tree's `style` refs name table entries (both are producer responsibilities, exactly as the frames/pages pairing always was). Every one of the five kinds also accepts an optional document-level `symbolTable` — the math curation layer mapping each written symbol glyph (within a scope) to its id, quantity kind, preferred unit, and definition source, alongside the unit registry (SI dimension-exponent vectors, exact rational conversions, per-unit-system normalisation contexts) that the `qty` nodes of lowered formulas resolve against. +`ContentDocument` (the semantic pivot) is a discriminated union of five kinds: `wordprocessing` (docx/odt sections of paragraphs/runs/tables/images), `presentation` (pptx/odp slides of shapes), `spreadsheet` (xlsx/ods sheets of cells, columns, rows, print settings), `drawing` (odg pages of shapes plus vector primitives — rect/ellipse/line/path), and `formula` (an equation carrying its own MathML node tree plus StarMath source when the producing format had one, extended with the two-layer math model: an optional verbatim-LaTeX `presentation` authoritative for rendering, an optional semantic `content: MathExpression` tree authoritative for computation, and provenance — neither layer stored derived from the other, so editing one never silently mutates the other). `ContentEmbeddedObjectSchema` lets any of the five embed another whole `ContentDocument`. Every paragraph/run/image/table/shape/vector/spreadsheet-cell leaf also carries its own canonical `headingLevel`-or-position fields directly: a `ContentParagraph`'s optional `headingLevel` (1 = the outermost heading, independent of the round-trip-only `styleId`), and every such leaf's optional `frames: LayoutFrame[]` — that node's own rendered page position(s) (`pageIndex` plus PDF user-space `xPt`/`yPt`/`widthPt`/`heightPt`), fused directly onto the content tree once a layout pass has run. `DocumentPackage` is the single hierarchical artefact — structure, layout, and content fused in one tree (see [The package tree](#the-package-tree)): the root carries `kind`, `metadata`, the optional document-level `symbolTable` and rendered `pages`, the optional package-level `styles`/`definitions`/`layers`/`attachments`/`destinations` tables (see [Definitions tables and styles](#definitions-tables-and-styles)), and `children` — one group per top-level container with the content tree grouped inside it, where a group's node may also be one of the six fidelity construct descriptors (see [Fidelity constructs](#fidelity-constructs)); the schema does not keep populated `frames` fields and `pages` in sync or detect staleness, and does not check that a tree's `style` refs name table entries (both are producer responsibilities, exactly as the frames/pages pairing always was). Every one of the five kinds also accepts an optional document-level `symbolTable` — the math curation layer mapping each written symbol glyph (within a scope) to its id, quantity kind, preferred unit, and definition source, alongside the unit registry (SI dimension-exponent vectors, exact rational conversions, per-unit-system normalisation contexts) that the `qty` nodes of lowered formulas resolve against. The `LayoutDocument` family (pages of positioned `LayoutItem`s — `text`/`image`/`rect`/`line`/`ellipse`/`path`/`link` in PDF user-space coordinates) no longer lives here: 4.0.0 demoted it to a pdf-codec-private model ([pdf-codec#65](https://github.com/ExaDev/pdf-codec/issues/65)), where the only codec that ever read or wrote it owns it outright. `documentFromJson` recognises old layout-document `$schema` URIs and throws a tombstone pointing at pdf-codec rather than failing as if the value were unrelated. Dependents stay on document-schema.js 3.x via semver until their own majors, so the demotion is not a cascade-breaker. @@ -90,7 +90,7 @@ const laidOut = DocumentPackageSchema.parse({ ...pkg, pages: [{ widthPt: 612, he `DocumentPackage` ([#20](https://github.com/ExaDev/document-schema.js/issues/20)) is the promoted single hierarchical artefact — one tree where 3.x carried `{ formatVersion, content, pages }` with the content flat. The tree's vocabulary is defined in `src/package-node.ts` and was proven first as [document-outline.js](https://github.com/ExaDev/document-outline.js)'s phase-1 `decompose`/`flatten` implementation ([document-outline.js#2](https://github.com/ExaDev/document-outline.js/issues/2)); this package's schemas are that shape's schema-home port, matching it node for node: -- **Groups** are `{ node, children }` where `node` embeds either an anchor paragraph (heading groups and list-item groups carry the full `ContentParagraph` — runs, formatting, frames — never a projected text label) or a container descriptor: `{ kind: 'section', pageSize, margins }`, `{ kind: 'slide', size, notes }`, `{ kind: 'sheet', name, cells, columns, rows, printSettings }`, `{ kind: 'drawPage', size }`, each tagged with a `kind` the flat container type does not carry, or a shape group's untagged frame descriptor. +- **Groups** are `{ node, children }` where `node` embeds either an anchor paragraph (heading groups and list-item groups carry the full `ContentParagraph` — runs, formatting, frames — never a projected text label) or a container descriptor: `{ kind: 'section', pageSize, margins }`, `{ kind: 'slide', size, notes }`, `{ kind: 'sheet', name, cells, columns, rows, printSettings }`, `{ kind: 'drawPage', size }`, each tagged with a `kind` the flat container type does not carry, or a shape group's untagged frame descriptor, or — since 4.1.0 — a **construct descriptor** (see [Fidelity constructs](#fidelity-constructs)). - **Bare leaves** carry their own `kind` and never `children`. Discrimination is structural on `node`+`children`, not on the presence of a `kind`. - **Section groups are mandatory** — one per `ContentSection` — because a section carries pre-layout page geometry (`pageSize`/`margins`) that a rendered `pages` array cannot hold. - **Grouping never crosses container boundaries**: a shape is its own group with its inner blocks grouped inside it (never a slide's paragraphs flattened across its shapes — that is a TOC projection, not a decomposition); a sheet's grid rides on the sheet node with images and embedded documents as children; embedded documents stay intact as one leaf. @@ -104,9 +104,69 @@ The flat `ContentDocument` and the tree are **one format, two encodings**, relat The codecs do not change: they keep producing flat `ContentDocument`s (their natural reading shape); decomposition runs once at the package boundary in documents.js and flatten runs once where a builder consumes a package. +## Fidelity constructs + +Every format carries constructs the flat content model drops — docx SDTs and field codes, ODF fields and tracked changes, PDF form widgets and link annotations, markdown footnotes and link titles. [#22](https://github.com/ExaDev/document-schema.js/issues/22) replaces those per-repo caveats with one **harmonised semantic vocabulary**: no format-specific node kinds, ever. Four codec inventories ([ooxml.js#65](https://github.com/ExaDev/ooxml.js/issues/65), [odf.js#59](https://github.com/ExaDev/odf.js/issues/59), [markdown-codec#63](https://github.com/ExaDev/markdown-codec/issues/63), [pdf-codec#66](https://github.com/ExaDev/pdf-codec/issues/66)) audited what each format actually carries, and 4.1.0 lands their answer additively ([#24](https://github.com/ExaDev/document-schema.js/issues/24)) — new descriptor kinds and package tables, no change to any existing node shape, so a 4.0.0 tree parses unchanged. + +A construct is a group like any other: `{ node: , children }`, where the children are the extent the construct spans. The tree was designed construct-capable from day one for exactly this, which is why the kinds could land in a minor rather than a second structural break. + +```ts +// A tracked insertion inside a docx content control, and a footnote marker whose body lives in the definitions table. +const pkg = DocumentPackageSchema.parse({ + kind: 'wordprocessing', + metadata: {}, + definitions: { n1: { kind: 'footnote', blocks: [{ kind: 'paragraph', runs: [{ text: 'The note body.' }] }] } }, + children: [ + { + node: { kind: 'section', pageSize: { widthPt: 612, heightPt: 792 }, margins: { topPt: 72, rightPt: 72, bottomPt: 72, leftPt: 72 } }, + children: [ + { + node: { kind: 'contentControl', controlType: 'richText', tag: 'ClientBlock', lock: 'container' }, + children: [ + { + node: { kind: 'provenance', change: 'insertion', author: 'A. Reviewer', dateIso: '2026-08-18T09:00:00Z' }, + children: [{ kind: 'paragraph', runs: [{ text: 'Inserted sentence.' }] }], + }, + { node: { kind: 'anchor', anchorType: 'footnote', name: '1', definition: 'n1' }, children: [] }, + ], + }, + ], + }, + ], +}); +``` + +The six kinds (`src/construct.ts`): + +| Kind | Carries | Where it comes from | +| --- | --- | --- | +| `contentControl` | `controlType`, `tag`, `alias`, `lock`, `value`, `checked`, `options` | docx block and inline SDTs, docx legacy `w:ffData` form fields, ODF `office:forms` controls and TOC/index wrappers, PDF AcroForm widgets and their field tree | +| `field` | `instruction`, `cachedResult` | docx `w:fldChar`/`w:instrText` and `w:fldSimple`, ODF field masters and simple fields, ODF cross-reference displays, pptx `a:fld` | +| `anchor` | `anchorType`, `name`, `definition` | docx bookmarks, comment extents, footnote/endnote references; ODF `text:bookmark`, `text:reference-mark*`, `text:note`, `office:annotation`; PDF sticky notes and markup annotations; markdown footnote markers | +| `link` | `target` (external URI or internal anchor name), `title` | docx `@w:anchor`, pptx slide jumps, PDF `GoTo`/`/Dest` and link annotations, markdown link/image titles | +| `provenance` | `change`, `author`, `dateIso` | docx `w:ins`/`w:del` and move tracking, ODF `text:tracked-changes`/`text:changed-region` | +| `division` | `name`, `columnCount`, `protected`, `source` | ODF `text:section` and `text:section-source`, tagged PDF `/Sect` and `/Div` | + +Four things bound the vocabulary, and each is a decision rather than an omission: + +- **Extents are block-scoped.** A construct group wraps the block flow of a section, heading group, shape, or list item. It cannot wrap a sub-sequence of one paragraph's runs, because a run-level extent is not expressible without changing `ContentParagraph`, and 4.1.0 changes no content shape. Run-level constructs keep their existing homes: **an external hyperlink stays on `ContentRun.hyperlink`** — `link` groups are for block-scoped and annotated extents a flat run field cannot express, never a replacement for it — and the inline field, bookmark, and tracked-change cases wait on a run-level mechanism rather than being forced into a wrapper that would split their paragraph. +- **Two group variants, one per block flow.** `SectionConstructGroup` sits in a section's or heading group's flow and admits heading children; `ShapeConstructGroup` sits in a shape's or list item's and does not — exactly the `SectionChild`/`ShapeChild` split that already existed. A construct nests in and around every other group, so a `provenance` wrapper inside a `contentControl` inside a `division` is a legal (and real) docx shape. Constructs are **not** legal as direct children of a slide, sheet, drawing page, or the package root: those hold containers and leaves, not block flow. +- **`division` is first-class, not degraded.** [#24](https://github.com/ExaDev/document-schema.js/issues/24) posed ODF `text:section` as a choice between a new generic kind and degrading to `contentControl` with the specifics in residue. It is first-class, on the odf inventory's own recommendation: `ContentSection` cannot host it (that is page geometry, one `pageSize`/`margins` pair, and it does not nest, while a division nests arbitrarily and usually changes no page geometry at all), and burying a structural container in the form-control vocabulary would make `contentControl` mean two unrelated things. It clears #22's no-format-specific-kinds bar on a real analogue — tagged PDF's `/Sect` and `/Div` are the same construct — not on ODF's say-so. It is spelled `division` rather than `section` because `{ kind: 'section' }` is already the page-geometry container descriptor. +- **Residue is not here.** #22's channel 2 — a per-node and package-level `source: { format, xml }` facility for what has no cross-format meaning — spans the whole content model and remains open on #22. Descriptors are closed objects with no escape hatch, deliberately: a descriptor-only residue field would mint exactly the parallel shape the general facility exists to avoid. + +Two harmonisations the inventories asked for are also deliberately absent. There is **no `fieldType` enum**: `instruction` is required and verbatim, so nothing is lost without one, and #24 asks for exactly one new vocabulary — `link`'s internal targets — rather than one per kind, with the four inventories' own corpus gate still standing before a field-type member set could be frozen honestly. And a **sheet-scoped named range** (xlsx defined names and tables, ODF `table:named-expressions`) is a definitions-table entry, not an `anchor`: a sheet group's children are its images and embedded documents, never a block flow, so there is no extent for an anchor to wrap — which is the odf inventory's own verdict for the identical construct. + ## Definitions tables and styles -The package root carries a generic definitions-table facility ([#21](https://github.com/ExaDev/document-schema.js/issues/21)): named tables whose entries tree nodes reference by string id. **Styles are the first tenant**; link and footnote definitions are future tenants of the same mechanism ([markdown-codec#63](https://github.com/ExaDev/markdown-codec/issues/63), [#22](https://github.com/ExaDev/document-schema.js/issues/22)) — which is why the tenant-generic `definitions` table (entries tagged with a `kind` discriminator and an open body, `src/definitions.ts`) sits alongside the styles-specific `styles` table rather than the facility being shaped around styles. +The package root carries a generic definitions-table facility ([#21](https://github.com/ExaDev/document-schema.js/issues/21)): named tables whose entries tree nodes reference by string id. **Styles were the first tenant**; link, footnote, and comment definitions ride the tenant-generic `definitions` table (entries tagged with a `kind` discriminator and an open body, `src/definitions.ts`) alongside it — which is why that generic table exists rather than the facility being shaped around styles. + +4.1.0 adds three more tables of that same generic type at the root ([#24](https://github.com/ExaDev/document-schema.js/issues/24), proposed by [pdf-codec#66](https://github.com/ExaDev/pdf-codec/issues/66)) — no new entry shape is minted anywhere: + +- `layers` — optional-content/layer definitions: PDF `/OCProperties` groups and their configuration, ODF Draw's layer model. Definitions only; which content belongs to which layer is a membership fact the producing codec carries on its own item model. +- `attachments` — package attachments: PDF `/Names /EmbeddedFiles`, `/FileAttachment`, `/EF`, `/AF`, and the docx/ODF package attachments that make the facility cross-format rather than PDF-specific. +- `destinations` — named destinations and the navigation tree that resolves against them: PDF `/Dests`, the `/Names` name tree, and `/Outlines`. This is the other end of a `link` construct's internal target. + +Each is its own root field rather than three more tenants of `definitions`, for the reason `styles` is its own field despite being the facility's first tenant: separate key namespaces, so a layer and a destination may share a name without colliding. The `kind` discriminator still earns its keep inside each, because each holds more than one tenant — a layers table carries group definitions alongside their configuration, and a destinations table carries named destinations alongside outline entries. Per-tenant entry fields stay the tenant's own, never this package's. A styles entry carries `{ paragraph?, run? }` sub-objects of **resolved canonical properties only**: paragraph `alignment`/`list`/`spacingBeforePt`/`spacingAfterPt`/`lineSpacing`/`indentLeftPt`/`indentFirstLinePt`, run `bold`/`italic`/`underline`/`strike`/`fontFamily`/`sizePt`/`color`. Never `frames`, never `sourcePath`, never `styleId` (per-node facts — a position is a fact about a node, not a style), never a `basedOn` graph (the table is a dictionary, not a program) — and the ban list is **enforced by schema shape** (strict objects that reject those keys outright), not merely documented. @@ -152,7 +212,7 @@ node_modules/document-schema.js/schemas/document-package.schema.json node_modules/document-schema.js/schemas/content-document.schema.json ``` -Each file's `$id` is a jsdelivr URL pinned to the exact npm version — immutable and live on publish, and (see [Versioning by `$schema`](#versioning-by-schema)) the version of anything stamped with it. Both files carry the same hand-authored `$defs` block (the same object emitted twice in one generator run, so the copies cannot drift), covering the recursive paragraph/table/embedded-object, MathML, and package-tree node models that Zod's converter cannot express directly; `content-json-schema-defs.ts` holds those fragments, and a regression test compares each fragment that has a real Zod counterpart against a live `z.toJSONSchema()` of that schema so a field changed without updating its fragment fails a test. The one deliberate cross-file `$ref` is the embedded-object cycle back to a whole `ContentDocument`. Fragments downstream of a `z.custom()` node (`ContentBlock`, `ContentTable`/`Cell`/`Row`, `ContentEmbeddedObject(Block)`, the seven package-tree group wrappers, `MathMlNode`/`Element`/`Attribute`, `ContentFormula`, `MathExpression` and its recursive variants) still need hand re-verification against `src/content.ts`/`src/package-node.ts`/`src/mathml.ts`/`src/math.ts` — see below. +Each file's `$id` is a jsdelivr URL pinned to the exact npm version — immutable and live on publish, and (see [Versioning by `$schema`](#versioning-by-schema)) the version of anything stamped with it. Both files carry the same hand-authored `$defs` block (the same object emitted twice in one generator run, so the copies cannot drift), covering the recursive paragraph/table/embedded-object, MathML, and package-tree node models that Zod's converter cannot express directly; `content-json-schema-defs.ts` holds those fragments, and a regression test compares each fragment that has a real Zod counterpart against a live `z.toJSONSchema()` of that schema so a field changed without updating its fragment fails a test. The one deliberate cross-file `$ref` is the embedded-object cycle back to a whole `ContentDocument`. Fragments downstream of a `z.custom()` node (`ContentBlock`, `ContentTable`/`Cell`/`Row`, `ContentEmbeddedObject(Block)`, the nine package-tree group wrappers, `MathMlNode`/`Element`/`Attribute`, `ContentFormula`, `MathExpression` and its recursive variants) still need hand re-verification — the construct descriptors themselves do not, since each is a plain `z.strictObject` reaching no opaque node and is held to the live comparison against `src/content.ts`/`src/package-node.ts`/`src/mathml.ts`/`src/math.ts` — see below. ### `z.custom()` vs `z.lazy()` for recursive schemas diff --git a/src/construct.test.ts b/src/construct.test.ts new file mode 100644 index 0000000..5bf5186 --- /dev/null +++ b/src/construct.test.ts @@ -0,0 +1,252 @@ +import { describe, expect, it } from 'vitest'; +import { + AnchorDescriptorSchema, + ConstructDescriptorSchema, + ContentControlDescriptorSchema, + DivisionDescriptorSchema, + FieldDescriptorSchema, + LinkDescriptorSchema, + ProvenanceDescriptorSchema, + type ConstructDescriptor, +} from './construct'; + +// Each accepted case below is a real construct from one of the four inventories, spelled with the fields that inventory's own verdict row said the kind has to carry -- not a synthetic minimal object -- so a field quietly dropped from a descriptor fails here rather than at the codec that needed it. + +describe('contentControl accepts the real control shapes the inventories name', () => { + it('accepts a docx block SDT with its tag, alias, and lock (ooxml.js#65: w:sdtPr alias/tag/lock)', () => { + const sdt: ConstructDescriptor = { + kind: 'contentControl', + controlType: 'richText', + tag: 'ClientName', + alias: 'Client name', + lock: 'container', + }; + expect(ContentControlDescriptorSchema.safeParse(sdt).success).toBe(true); + expect(ConstructDescriptorSchema.safeParse(sdt).success).toBe(true); + }); + + it('accepts a legacy w:ffData checkbox and a ddList with its options (ooxml.js#65: checkbox/ddList/textInput)', () => { + expect( + ContentControlDescriptorSchema.safeParse({ kind: 'contentControl', controlType: 'checkbox', checked: true }) + .success, + ).toBe(true); + expect( + ContentControlDescriptorSchema.safeParse({ + kind: 'contentControl', + controlType: 'dropDown', + options: ['Draft', 'Final'], + value: 'Draft', + }).success, + ).toBe(true); + }); + + it('accepts a PDF AcroForm text widget and a non-terminal field-tree node (pdf-codec#66: /FT /Tx with /V, /Fields recursion)', () => { + expect( + ContentControlDescriptorSchema.safeParse({ + kind: 'contentControl', + controlType: 'plainText', + tag: 'address.line1', + alias: 'Street address', + value: '10 Downing Street', + lock: 'content', + }).success, + ).toBe(true); + expect(ContentControlDescriptorSchema.safeParse({ kind: 'contentControl', controlType: 'group' }).success).toBe( + true, + ); + }); + + it('accepts an ODF index wrapper as a typed container (odf.js#59: text:table-of-content and kin)', () => { + expect( + ContentControlDescriptorSchema.safeParse({ kind: 'contentControl', controlType: 'index', alias: 'Table of Contents' }) + .success, + ).toBe(true); + }); + + it('rejects near-misses: an unknown control type, an unknown lock, a non-string option, and an extra key', () => { + expect(ContentControlDescriptorSchema.safeParse({ kind: 'contentControl', controlType: 'w:sdt' }).success).toBe(false); + expect( + ContentControlDescriptorSchema.safeParse({ kind: 'contentControl', controlType: 'richText', lock: 'readOnly' }) + .success, + ).toBe(false); + expect( + ContentControlDescriptorSchema.safeParse({ kind: 'contentControl', controlType: 'dropDown', options: [1, 2] }) + .success, + ).toBe(false); + expect( + ContentControlDescriptorSchema.safeParse({ kind: 'contentControl', controlType: 'richText', sdtPr: '' }) + .success, + ).toBe(false); + }); + + it('rejects a control with no controlType -- the kind exists to say what kind of control produced the content', () => { + expect(ContentControlDescriptorSchema.safeParse({ kind: 'contentControl' }).success).toBe(false); + }); +}); + +describe('field carries an instruction, with the cached scalar result beside it', () => { + it('accepts a docx complex field and a pptx a:fld (ooxml.js#65: w:instrText, a:fld slidenum)', () => { + expect( + FieldDescriptorSchema.safeParse({ kind: 'field', instruction: 'TOC \\o "1-3" \\h \\z \\u' }).success, + ).toBe(true); + expect(FieldDescriptorSchema.safeParse({ kind: 'field', instruction: 'slidenum', cachedResult: '7' }).success).toBe( + true, + ); + }); + + it('accepts an ODF simple field with its cached text (odf.js#59: text:page-number and the everyday family)', () => { + expect( + FieldDescriptorSchema.safeParse({ kind: 'field', instruction: 'text:page-number', cachedResult: '12' }).success, + ).toBe(true); + }); + + it('rejects a field with no instruction, and one carrying an invented harmonised type', () => { + expect(FieldDescriptorSchema.safeParse({ kind: 'field', cachedResult: '12' }).success).toBe(false); + expect( + FieldDescriptorSchema.safeParse({ kind: 'field', instruction: 'PAGE', fieldType: 'pageNumber' }).success, + ).toBe(false); + }); +}); + +describe('anchor names an extent or a reference site, and points at its definition when it has one', () => { + it('accepts a docx bookmark and an ODF text:bookmark (ooxml.js#65, odf.js#59)', () => { + expect(AnchorDescriptorSchema.safeParse({ kind: 'anchor', anchorType: 'bookmark', name: '_Toc12345' }).success).toBe( + true, + ); + }); + + it('accepts the marker-plus-definition split every inventory landed on: footnote, endnote, and comment markers', () => { + for (const anchorType of ['footnote', 'endnote', 'comment'] as const) { + expect( + AnchorDescriptorSchema.safeParse({ kind: 'anchor', anchorType, name: '1', definition: 'n1' }).success, + ).toBe(true); + } + }); + + it('rejects an unnamed anchor and an unknown anchor type', () => { + expect(AnchorDescriptorSchema.safeParse({ kind: 'anchor', anchorType: 'bookmark' }).success).toBe(false); + expect(AnchorDescriptorSchema.safeParse({ kind: 'anchor', anchorType: 'annotation', name: 'a' }).success).toBe(false); + }); +}); + +describe('link carries the target vocabulary a flat run field cannot express', () => { + it('accepts an external target with a markdown title (markdown-codec#63: link/image titles are dropped everywhere today)', () => { + expect( + LinkDescriptorSchema.safeParse({ + kind: 'link', + target: { kind: 'external', uri: 'https://example.com' }, + title: 'Example home page', + }).success, + ).toBe(true); + }); + + it('accepts an internal target -- docx @w:anchor, a pptx slide jump, a PDF GoTo//Dest (the one new vocabulary #24 asks for)', () => { + expect( + LinkDescriptorSchema.safeParse({ kind: 'link', target: { kind: 'internal', anchor: '_Toc12345' } }).success, + ).toBe(true); + }); + + it('rejects a target that names neither family, and a mixed target carrying both a uri and an anchor', () => { + expect(LinkDescriptorSchema.safeParse({ kind: 'link', target: { uri: 'https://example.com' } }).success).toBe(false); + expect( + LinkDescriptorSchema.safeParse({ + kind: 'link', + target: { kind: 'external', uri: 'https://example.com', anchor: '_Toc1' }, + }).success, + ).toBe(false); + }); +}); + +describe('provenance records who changed what, and when', () => { + it('accepts docx w:ins/w:del and ODF text:changed-region shapes, including the move pair', () => { + for (const change of ['insertion', 'deletion', 'moveFrom', 'moveTo', 'formatChange'] as const) { + expect( + ProvenanceDescriptorSchema.safeParse({ + kind: 'provenance', + change, + author: 'A. Reviewer', + dateIso: '2026-08-18T09:00:00Z', + }).success, + ).toBe(true); + } + }); + + it('rejects an unknown change class and an anonymous wrapper with no change at all', () => { + expect(ProvenanceDescriptorSchema.safeParse({ kind: 'provenance', change: 'w:ins' }).success).toBe(false); + expect(ProvenanceDescriptorSchema.safeParse({ kind: 'provenance', author: 'A. Reviewer' }).success).toBe(false); + }); +}); + +describe('division is the ODF text:section shape, decided first-class rather than degraded', () => { + it('accepts a named, protected, multi-column division (odf.js#59: columns, protection, naming)', () => { + expect( + DivisionDescriptorSchema.safeParse({ kind: 'division', name: 'Chapter1', columnCount: 2, protected: true }) + .success, + ).toBe(true); + }); + + it('accepts the odm external-chapter link (odf.js#59: text:section-source href plus text:section-name)', () => { + expect( + DivisionDescriptorSchema.safeParse({ + kind: 'division', + name: 'Chapter1', + source: { href: '../chapters/one.odt', sectionName: 'Body' }, + }).success, + ).toBe(true); + }); + + it('accepts a bare division -- every field is optional because only ODF requires a name', () => { + expect(DivisionDescriptorSchema.safeParse({ kind: 'division' }).success).toBe(true); + }); + + it('rejects a fractional or zero column count, a source with no href, and text:filter-name smuggled onto the source', () => { + expect(DivisionDescriptorSchema.safeParse({ kind: 'division', columnCount: 1.5 }).success).toBe(false); + expect(DivisionDescriptorSchema.safeParse({ kind: 'division', columnCount: 0 }).success).toBe(false); + expect(DivisionDescriptorSchema.safeParse({ kind: 'division', source: { sectionName: 'Body' } }).success).toBe(false); + expect( + DivisionDescriptorSchema.safeParse({ + kind: 'division', + source: { href: '../one.odt', filterName: 'writer8' }, + }).success, + ).toBe(false); + }); + + it('is not spelled "section" -- that kind is the page-geometry container descriptor, and one word cannot mean both', () => { + expect( + ConstructDescriptorSchema.safeParse({ + kind: 'section', + pageSize: { widthPt: 612, heightPt: 792 }, + margins: { topPt: 72, rightPt: 72, bottomPt: 72, leftPt: 72 }, + }).success, + ).toBe(false); + }); +}); + +describe('the construct descriptor union', () => { + it('admits exactly the six kinds and nothing else', () => { + const kinds = ['contentControl', 'field', 'anchor', 'link', 'provenance', 'division']; + const descriptors: ConstructDescriptor[] = [ + { kind: 'contentControl', controlType: 'richText' }, + { kind: 'field', instruction: 'PAGE' }, + { kind: 'anchor', anchorType: 'bookmark', name: 'b1' }, + { kind: 'link', target: { kind: 'external', uri: 'https://example.com' } }, + { kind: 'provenance', change: 'insertion' }, + { kind: 'division' }, + ]; + expect(descriptors.map((descriptor) => descriptor.kind)).toEqual(kinds); + for (const descriptor of descriptors) { + expect(ConstructDescriptorSchema.safeParse(descriptor).success).toBe(true); + } + expect(ConstructDescriptorSchema.safeParse({ kind: 'residue', xml: '' }).success).toBe(false); + }); + + it('survives a JSON round trip unchanged', () => { + const descriptor: ConstructDescriptor = { + kind: 'link', + target: { kind: 'internal', anchor: 'destination-3' }, + title: 'Chapter 3', + }; + const roundTripped: unknown = JSON.parse(JSON.stringify(descriptor)); + expect(ConstructDescriptorSchema.parse(roundTripped)).toEqual(descriptor); + }); +}); diff --git a/src/construct.ts b/src/construct.ts new file mode 100644 index 0000000..b78cb94 --- /dev/null +++ b/src/construct.ts @@ -0,0 +1,137 @@ +import { z } from 'zod'; + +// The harmonised semantic construct vocabulary (ExaDev/document-schema.js#22, landed additively per ExaDev/document-schema.js#24): the descriptor payloads a package-tree group may carry in place of a container descriptor or an anchor paragraph. The tree was designed construct-capable from day one -- a group is `{ node, children }`, and a construct is exactly that shape with a descriptor as its node and its extent as its children -- so these kinds land without a second structural break: a 4.0.0 tree contains none of them and parses identically under this release. + +// The vocabulary is format-agnostic by construction: every kind below is confirmed by at least two of the four codec inventories (ExaDev/ooxml.js#65, ExaDev/odf.js#59, ExaDev/markdown-codec#63, ExaDev/pdf-codec#66), and a construct with no cross-format analogue never gets a bespoke kind -- it degrades to the nearest kind here with its format-specific specifics quarantined in the residue channel that ExaDev/document-schema.js#22 still owns. This module deliberately does NOT define that residue channel: channel 2 is a per-node and package-level `source` facility spanning the whole content model, not a construct-descriptor field, and pre-empting it with a descriptor-only escape hatch would mint exactly the parallel shape the facility exists to avoid. + +// EXTENT SCOPE, stated once because it bounds every kind below: a construct group wraps BLOCK-scoped extents -- the block flow of a section, a heading group, a shape, or a list item. It does not wrap a sub-sequence of one paragraph's runs, because a run-level extent is not expressible without changing ContentParagraph's own shape, and the content model is unchanged by this release. The run-level constructs the formats do carry keep their existing homes: an external hyperlink stays on ContentRun.hyperlink (the standing reconciliation on ExaDev/document-schema.js#22 -- `link` groups are for block-scoped and annotated extents a flat run field cannot express, never a replacement for it), and the inline field/bookmark/tracked-change cases wait on a run-level extent mechanism rather than being forced into a block wrapper that would split the paragraph they sit inside. + +// A typed container of content: docx block and inline SDTs (`w:sdt`/`w:sdtContent`), docx legacy form fields (`w:ffData`), ODF `office:forms` controls in ordinary odt, ODF TOC and index wrappers as typed containers, and PDF AcroForm widgets with their field tree. The member set is the union of what those four producers actually spell, with each member named by a real inventory row rather than invented for symmetry: a control's own rendered content is its children, and this names what kind of control produced it. +export const ContentControlTypeSchema = z.enum([ + 'richText', // docx rich-text SDT at block and inline level; the general "container of arbitrary content" case + 'plainText', // docx plain-text SDT, docx `w:ffData` textInput, PDF AcroForm `/FT /Tx` + 'checkbox', // docx checkbox SDT, docx `w:ffData` checkbox, PDF AcroForm `/FT /Btn` checkbox, ODF form checkbox + 'dropDown', // docx `w:dropDownList` SDT, docx `w:ffData` ddList, PDF AcroForm `/FT /Ch` list box, ODF form listbox -- a closed list, no free text + 'comboBox', // docx `w:comboBox` SDT, PDF AcroForm `/FT /Ch` with the combo flag, ODF form combobox -- a list that also accepts free text + 'date', // docx date SDT + 'picture', // docx picture SDT + 'repeatingSection', // docx repeatingSection SDT + 'button', // PDF AcroForm push button, ODF form button + 'index', // ODF `text:table-of-content`/`text:alphabetical-index`/`text:bibliography`/`text:illustration-index`/`text:table-index`/`text:user-index`/`text:object-index`, and docx's TOC-as-SDT (`w:docPartObj` gallery) -- the wrapper, with its cached rendered entries as children + 'group', // a container of other controls carrying no value of its own: PDF AcroForm non-terminal `/Fields` nodes, ODF `office:forms` +]); +export type ContentControlType = z.infer; + +// What a producer locked, harmonised across the three spellings that exist: docx `w:lock` (`contentLocked`/`sdtLocked`/`sdtContentLocked`), PDF AcroForm's `/Ff` ReadOnly bit, and ODF form controls' read-only flag. Absent means nothing is locked -- there is deliberately no 'none' member, because an absent key and a key naming the absence of a lock are the same fact and two spellings of one fact is how tables drift. +export const ContentControlLockSchema = z.enum([ + 'content', // the contents cannot be edited, but the control itself can be removed -- docx `contentLocked`, AcroForm ReadOnly, ODF read-only + 'container', // the control cannot be removed, but its contents can be edited -- docx `sdtLocked` + 'both', // docx `sdtContentLocked` +]); +export type ContentControlLock = z.infer; + +export const ContentControlDescriptorSchema = z.strictObject({ + kind: z.literal('contentControl'), + controlType: ContentControlTypeSchema, + tag: z.string().optional(), // the machine-readable identifier a producer addresses this control by: docx `w:tag`, PDF AcroForm's partial field name `/T` + alias: z.string().optional(), // the human-readable label shown to an author: docx `w:alias`, PDF AcroForm's alternate description `/TU` + lock: ContentControlLockSchema.optional(), + value: z.string().optional(), // the control's current scalar value where it has one -- PDF AcroForm `/V`, a date control's date, a text input's text. A control whose value IS its rendered content carries that content in `children` and leaves this absent. + checked: z.boolean().optional(), // a checkbox or radio control's state, which is a boolean in every format that has one and would lose its type spelled through `value` + options: z.array(z.string()).optional(), // the choice list of a dropDown/comboBox control: docx `w:listItem` entries, PDF AcroForm `/Opt`, an ODF form control's list source +}); +export type ContentControlDescriptor = z.infer; + +// Instruction plus cached result, with the field's extent expressed as containment rather than as the marker pairs the formats serialise: docx `w:fldChar` begin/separate/end ranges and `w:fldSimple`, the ODF field-master families and the everyday simple-field set, ODF cross-reference displays, and pptx `a:fld` (which is what confirms the kind is genuinely cross-format rather than a docx-only shape). +// +// There is deliberately no harmonised `fieldType` enum here. `instruction` is required and verbatim, so nothing is lost without one; the harmonised type vocabulary is the kind of shape the four inventories' own corpus gate exists to settle (several field families have no real producer fixture in any repo yet), and ExaDev/document-schema.js#24 asks for exactly one new vocabulary -- the internal-target vocabulary on `link` below -- rather than one per kind. Minting a type enum from spec recollection now would freeze the member set before a single real file has been read against it. +export const FieldDescriptorSchema = z.strictObject({ + kind: z.literal('field'), + instruction: z.string(), // the producer's own field code, verbatim: docx `w:instrText` text or `w:fldSimple/@w:instr`, an ODF field element with its attributes, a pptx `a:fld/@type` + cachedResult: z.string().optional(), // the field's last-computed display text where the producer cached a scalar one (an ODF field element's own text content, a pptx `a:fld`'s `a:t`). A field whose result is block content carries that content in `children` and leaves this absent -- the two are the block and the scalar case of one fact, never two encodings of the same one. +}); +export type FieldDescriptor = z.infer; + +// What an anchor marks. A bookmark is a named target and nothing else; the other three are reference-site markers whose body lives in a definitions-table entry -- the marker-plus-definition split all four inventories independently landed on, and the reason a footnote is not a contiguous extent. +export const AnchorTypeSchema = z.enum([ + 'bookmark', // docx `w:bookmarkStart`/`w:bookmarkEnd`, ODF `text:bookmark` and `text:reference-mark*`, a PDF named destination's target site + 'footnote', // docx `w:footnoteReference`, ODF `text:note` with note-class footnote, a markdown `[^n]` marker + 'endnote', // docx `w:endnoteReference`, ODF `text:note` with note-class endnote + 'comment', // docx `w:commentRangeStart`/`End`, ODF `office:annotation`, a PDF sticky note or markup annotation +]); +export type AnchorType = z.infer; + +// A named extent or a reference-site marker. A point anchor -- a footnote reference, a bookmark with no range -- is a group with no children; a ranged anchor -- a docx bookmark pair, a comment extent -- wraps the blocks it spans. +// +// Scope note, because one inventory row cannot land as this kind: a sheet-scoped named range (xlsx defined names and tables, ODF `table:named-expressions`) has no block-flow extent to wrap -- a sheet group's children are its images and embedded documents, never a block flow -- so those ride a definitions-table entry naming their range, which is the odf inventory's own verdict for the identical construct. `anchor` covers block-flow extents. +export const AnchorDescriptorSchema = z.strictObject({ + kind: z.literal('anchor'), + anchorType: AnchorTypeSchema, + name: z.string(), // the anchor's own name: docx `w:name`, ODF `text:name`, a PDF destination name. Required -- an anchor nothing can address is not an anchor. + definition: z.string().optional(), // the definitions-table key holding this marker's body, for the note and comment cases; the entry's own tenant vocabulary carries the body, its author, and its date +}); +export type AnchorDescriptor = z.infer; + +// The internal-target vocabulary ExaDev/document-schema.js#24 asks for, spelled as the one place a link's two target families are distinguished: an external URI, or a name resolved inside this document. The internal arm is what docx `w:hyperlink/@w:anchor`, pptx slide jumps, and PDF `GoTo`/`/Dest` all need and none of them can express through a flat run field. +export const LinkTargetSchema = z.discriminatedUnion('kind', [ + z.strictObject({ + kind: z.literal('external'), + uri: z.string(), // the resolved external URI, the same value ContentRun.hyperlink carries for the run-level case + }), + z.strictObject({ + kind: z.literal('internal'), + anchor: z.string(), // the name of an `anchor` construct in this document, or of a `destinations` table entry -- one namespace, so a resolver has one place to look + }), +]); +export type LinkTarget = z.infer; + +// Target plus the extent it wraps, reserved for exactly what a flat `ContentRun.hyperlink` cannot express: a block-scoped extent (a PDF link annotation whose rect matches no recovered run), an annotated one (a markdown link or image title), and an internal target. Run-level external hyperlinks stay on ContentRun.hyperlink per the standing reconciliation on ExaDev/document-schema.js#22 -- this kind does not replace them, and a producer that emits both encodings for one link has emitted it twice. +export const LinkDescriptorSchema = z.strictObject({ + kind: z.literal('link'), + target: LinkTargetSchema, + title: z.string().optional(), // the annotation a run field has nowhere to put: a markdown link/image title, a PDF link annotation's contents +}); +export type LinkDescriptor = z.infer; + +// What kind of tracked change a provenance wrapper records. The five members are the union of docx's `w:ins`/`w:del`/`w:moveFrom`/`w:moveTo`/`w:rPrChange`-`w:pPrChange` and ODF's `text:changed-region` children (`text:insertion`, `text:deletion`, `text:format-change`); the move relation itself -- which moveFrom pairs with which moveTo -- has no ODF counterpart and stays residue, per the ooxml inventory's own verdict. +export const ProvenanceChangeSchema = z.enum(['insertion', 'deletion', 'moveFrom', 'moveTo', 'formatChange']); +export type ProvenanceChange = z.infer; + +// An author/date wrapper around content: docx `w:ins`/`w:del` and move tracking, ODF `text:tracked-changes`/`text:changed-region` with its inline markers. A deletion's children are the deleted content -- carried, not dropped, which is the whole point of the kind: today's readers merge insertions anonymously and drop deletions outright. +export const ProvenanceDescriptorSchema = z.strictObject({ + kind: z.literal('provenance'), + change: ProvenanceChangeSchema, + author: z.string().optional(), + dateIso: z.string().optional(), // ISO-8601, matching LayoutMetadata's own createdIso/modifiedIso spelling rather than minting a second date convention +}); +export type ProvenanceDescriptor = z.infer; + +// The external-chapter link of a division: ODF `text:section-source`, which the odm reader already reads verbatim as its chapter model. `text:filter-name` has no cross-format meaning and stays residue. +export const DivisionSourceSchema = z.strictObject({ + href: z.string(), // `xlink:href` -- the document this division's content is linked from + sectionName: z.string().optional(), // `text:section-name` -- which named division inside that document, when the link is to part of it +}); +export type DivisionSource = z.infer; + +// A named, arbitrarily nestable grouping of block flow: the ODF `text:section` shape, and the sixth kind ExaDev/document-schema.js#24 poses as a decision. Decided first-class rather than degraded to `contentControl`, on the odf inventory's own recommendation and its stated reasoning: ContentSection cannot host it (that is page geometry, one pageSize/margins pair, and it does not nest, while a division nests arbitrarily and usually changes no page geometry at all), and burying a structural container in the form-control vocabulary would make `contentControl` mean two unrelated things. It clears ExaDev/document-schema.js#22's no-format-specific-kinds bar on a real cross-format analogue rather than on ODF's say-so: tagged PDF's `/Sect` and `/Div` structure elements are the same construct, and docx spells the linked-chapter case through subdocuments. +// +// Named `division`, not `section`, because `{ kind: 'section' }` is already taken by the page-geometry container descriptor (src/package-node.ts) and one word cannot mean both. The name is tagged PDF's own for the same shape. +export const DivisionDescriptorSchema = z.strictObject({ + kind: z.literal('division'), + name: z.string().optional(), // ODF `text:name`; how the odm chapter model and cross-document links address a division + columnCount: z.number().int().positive().optional(), // the column count a division sets over its own flow, which is pre-layout geometry with no other home -- the styles table carries paragraph and run properties only + protected: z.boolean().optional(), // ODF `text:protected` -- the content is not editable in place + source: DivisionSourceSchema.optional(), +}); +export type DivisionDescriptor = z.infer; + +// The node payload of a construct group, discriminated on `kind` exactly as the container descriptors are. Adding a member here is schema-additive (a value carrying it simply starts parsing) and TS-breaking only for a consumer switching exhaustively over the union -- which is the whole reason these kinds could land in a minor after the tree major rather than needing one of their own. +export const ConstructDescriptorSchema = z.discriminatedUnion('kind', [ + ContentControlDescriptorSchema, + FieldDescriptorSchema, + AnchorDescriptorSchema, + LinkDescriptorSchema, + ProvenanceDescriptorSchema, + DivisionDescriptorSchema, +]); +export type ConstructDescriptor = z.infer; diff --git a/src/content-json-schema-defs.test.ts b/src/content-json-schema-defs.test.ts index 3ce5c07..82b1fee 100644 --- a/src/content-json-schema-defs.test.ts +++ b/src/content-json-schema-defs.test.ts @@ -26,6 +26,17 @@ import { ContentCellValueSchema, } from './content'; import { CONTENT_DEFS } from './content-json-schema-defs'; +import { + AnchorDescriptorSchema, + ConstructDescriptorSchema, + ContentControlDescriptorSchema, + DivisionDescriptorSchema, + DivisionSourceSchema, + FieldDescriptorSchema, + LinkDescriptorSchema, + LinkTargetSchema, + ProvenanceDescriptorSchema, +} from './construct'; import { DefinitionEntrySchema, StyleEntrySchema, StyleParagraphPropertiesSchema, StyleRunPropertiesSchema } from './definitions'; import { BoxSchema, LayoutFrameSchema, MarginsSchema, PageSizeSchema } from './geometry'; import { @@ -54,7 +65,7 @@ import { } from './package-node'; import { AlignmentSchema } from './style'; -// This is the regression test scripts/generate-json-schemas.mjs's own top comment calls for: the only structural defence that generator has against silently drifting away from src/content.ts/src/color.ts/src/geometry.ts/src/style.ts/src/math.ts/src/package-node.ts/src/definitions.ts, since CONTENT_DEFS (content-json-schema-defs.ts) is transcribed by hand rather than generated. Not every entry in CONTENT_DEFS can be checked this way -- ContentBlock/ContentTable/ContentTableRow/ContentTableCell/ContentEmbeddedObject(Block)/MathMlNode/MathMlElement/MathMlAttribute all sit downstream of one of the genuinely un-representable z.custom() nodes (ContentBlockSchema, ContentEmbeddedObjectSchema, MathMlNodeSchema), the seven package-tree group wrappers sit downstream of the tree's own per-kind group schemas (src/package-node.ts, z.custom over recursive guards, reached only through the hand fragments' own children pointers), and ContentFormula/MathExpression/MathApp/MathSum/MathProd/MathMatrix sit downstream of the fourth opaque node (MathExpressionSchema, reached through ContentFormulaSchema.content for the first and through the grammar's own recursion for the rest) -- see that module's own top comment -- so a bare z.toJSONSchema() call over their real schema counterpart either throws or degrades to `{}` for the recursive/custom part, which is exactly the problem CONTENT_DEFS exists to work around in the first place. What CAN be checked -- because a real, non-recursive, non-custom exported Zod schema exists for it -- is every leaf and near-leaf fragment: Color, Box, LayoutFrame, Alignment, ContentStrokeStyle, ContentBorder, ContentCellBorders, ContentListMembership, ContentRun, ContentParagraph, ContentImageBlock, ContentPageBreak, PageSize, Margins, SectionDescriptor, SlideDescriptor, SheetDescriptor, DrawPageDescriptor, ShapeDescriptor, HeadingParagraph, ListParagraph, ContentSheetCell, ContentCellValue, ContentSheetCellComment, ContentSheetColumn, ContentSheetRow, ContentSheetPrintSettings, ContentSheetPrintRange, ContentSheetRepeatRange, ContentSheetImage, ContentStroke, ContentPathPoint, ContentPathSegment, ContentSubpath, ContentVector, StyleParagraphProperties, StyleRunProperties, StyleEntry, DefinitionEntry, ExactRational, DimensionVector, MathPresentation, MathProvenance, MathUncertainty, MathNum, MathQty, MathSym, MathUnparsed, MathSymbolEntry, MathUnit, MathNormalisationContext, SymbolTable. None of these reaches ContentBlockSchema, ContentEmbeddedObjectSchema, MathMlNodeSchema, MathExpressionSchema, or a tree group schema from anywhere in its own field tree, so each can be generated live and compared directly. +// This is the regression test scripts/generate-json-schemas.mjs's own top comment calls for: the only structural defence that generator has against silently drifting away from src/content.ts/src/color.ts/src/geometry.ts/src/style.ts/src/math.ts/src/package-node.ts/src/definitions.ts, since CONTENT_DEFS (content-json-schema-defs.ts) is transcribed by hand rather than generated. Not every entry in CONTENT_DEFS can be checked this way -- ContentBlock/ContentTable/ContentTableRow/ContentTableCell/ContentEmbeddedObject(Block)/MathMlNode/MathMlElement/MathMlAttribute all sit downstream of one of the genuinely un-representable z.custom() nodes (ContentBlockSchema, ContentEmbeddedObjectSchema, MathMlNodeSchema), the nine package-tree group wrappers sit downstream of the tree's own per-kind group schemas (src/package-node.ts, z.custom over recursive guards, reached only through the hand fragments' own children pointers), and ContentFormula/MathExpression/MathApp/MathSum/MathProd/MathMatrix sit downstream of the fourth opaque node (MathExpressionSchema, reached through ContentFormulaSchema.content for the first and through the grammar's own recursion for the rest) -- see that module's own top comment -- so a bare z.toJSONSchema() call over their real schema counterpart either throws or degrades to `{}` for the recursive/custom part, which is exactly the problem CONTENT_DEFS exists to work around in the first place. What CAN be checked -- because a real, non-recursive, non-custom exported Zod schema exists for it -- is every leaf and near-leaf fragment: Color, Box, LayoutFrame, Alignment, ContentStrokeStyle, ContentBorder, ContentCellBorders, ContentListMembership, ContentRun, ContentParagraph, ContentImageBlock, ContentPageBreak, PageSize, Margins, SectionDescriptor, SlideDescriptor, SheetDescriptor, DrawPageDescriptor, ShapeDescriptor, HeadingParagraph, ListParagraph, the whole construct descriptor vocabulary (ContentControlDescriptor, FieldDescriptor, AnchorDescriptor, LinkTarget, LinkDescriptor, ProvenanceDescriptor, DivisionSource, DivisionDescriptor, and the ConstructDescriptor union over them -- each a plain z.strictObject or a union of them, reaching no opaque node), ContentSheetCell, ContentCellValue, ContentSheetCellComment, ContentSheetColumn, ContentSheetRow, ContentSheetPrintSettings, ContentSheetPrintRange, ContentSheetRepeatRange, ContentSheetImage, ContentStroke, ContentPathPoint, ContentPathSegment, ContentSubpath, ContentVector, StyleParagraphProperties, StyleRunProperties, StyleEntry, DefinitionEntry, ExactRational, DimensionVector, MathPresentation, MathProvenance, MathUncertainty, MathNum, MathQty, MathSym, MathUnparsed, MathSymbolEntry, MathUnit, MathNormalisationContext, SymbolTable. None of these reaches ContentBlockSchema, ContentEmbeddedObjectSchema, MathMlNodeSchema, MathExpressionSchema, or a tree group schema from anywhere in its own field tree, so each can be generated live and compared directly. // // Comparison strategy: a bare `z.toJSONSchema(SomeSchema)` call, run in isolation, would INLINE every nested schema it encounters (ColorSchema inside ContentRunSchema, AlignmentSchema inside ContentParagraphSchema, etc.) rather than emit the `{ $ref: '#/$defs/X' }` pointers CONTENT_DEFS itself uses -- because those nested schemas aren't registered anywhere. To reproduce the exact cross-reference shape CONTENT_DEFS hand-authors, this test registers the identical set of real schemas under the identical id strings CONTENT_DEFS uses as its own $defs keys, with a `uri` callback matching the `#/$defs/` convention CONTENT_DEFS was written against -- confirmed empirically (see this file's own construction) to make Zod's registry-based multi-schema generation emit exactly that $ref shape for every registered schema referenced from within another. Each per-schema result still carries its own top-level `$schema`/`$id` (since z.toJSONSchema(registry, ...) treats every registered schema as its own standalone root), which CONTENT_DEFS's own nested fragments never have -- those two keys are stripped before comparison, since they're an artefact of testing each fragment as a registry root rather than a real structural difference. @@ -80,6 +91,15 @@ const REGISTERED_SCHEMAS = { ShapeDescriptor: ShapeDescriptorSchema, HeadingParagraph: HeadingParagraphSchema, ListParagraph: ListParagraphSchema, + ContentControlDescriptor: ContentControlDescriptorSchema, + FieldDescriptor: FieldDescriptorSchema, + AnchorDescriptor: AnchorDescriptorSchema, + LinkTarget: LinkTargetSchema, + LinkDescriptor: LinkDescriptorSchema, + ProvenanceDescriptor: ProvenanceDescriptorSchema, + DivisionSource: DivisionSourceSchema, + DivisionDescriptor: DivisionDescriptorSchema, + ConstructDescriptor: ConstructDescriptorSchema, ContentSheetCell: ContentSheetCellSchema, ContentCellValue: ContentCellValueSchema, ContentSheetCellComment: ContentSheetCellCommentSchema, diff --git a/src/content-json-schema-defs.ts b/src/content-json-schema-defs.ts index 8d99ab1..7087f4f 100644 --- a/src/content-json-schema-defs.ts +++ b/src/content-json-schema-defs.ts @@ -5,11 +5,11 @@ import { schemaUriFor } from './schema-io'; // The hand-authored JSON Schema $defs fragments spliced into content-document.schema.json's `override()` callback (scripts/generate-json-schemas.mjs), lifted out into their own src module rather than staying inline in that script. The reason is single-sourcing, not tidiness: this exact object needs to be reachable from two places that cannot share an import graph -- // // 1. scripts/generate-json-schemas.mjs itself, which only ever runs against the freshly-built ../dist/ (it imports every other schema it needs the same way), so it imports CONTENT_DEFS from '../dist/content-json-schema-defs.js', the file tsdown emits for this module (entry: 'src/**/*.ts', one dist file per src file -- see tsdown.config.ts). -// 2. content-json-schema-defs.test.ts (src/, run directly by vitest's "unit" project against source, never against dist), which imports this exact same CONTENT_DEFS value straight from here and asserts it stays byte-for-byte in step with a live z.toJSONSchema() call over each fragment's real exported Zod schema counterpart (ContentParagraphSchema, ContentRunSchema, ContentListMembershipSchema, ContentImageBlockSchema, ContentPageBreakSchema, ColorSchema, BoxSchema, LayoutFrameSchema, PageSizeSchema, MarginsSchema, AlignmentSchema, ContentStrokeStyleSchema, ContentBorderSchema, ContentCellBordersSchema, the package tree's non-recursive descriptors and anchors from src/package-node.ts: SectionDescriptorSchema, SlideDescriptorSchema, SheetDescriptorSchema, DrawPageDescriptorSchema, ShapeDescriptorSchema, HeadingParagraphSchema, ListParagraphSchema, the sheet grid and vector leaves from src/content.ts: ContentSheetCellSchema, ContentCellValueSchema, ContentSheetCellCommentSchema, ContentSheetColumnSchema, ContentSheetRowSchema, ContentSheetPrintSettingsSchema, ContentSheetPrintRangeSchema, ContentSheetRepeatRangeSchema, ContentSheetImageSchema, ContentStrokeSchema, ContentPathPointSchema, ContentPathSegmentSchema, ContentSubpathSchema, ContentVectorSchema, and the definitions facility from src/definitions.ts: StyleParagraphPropertiesSchema, StyleRunPropertiesSchema, StyleEntrySchema, DefinitionEntrySchema, plus the non-recursive math leaves from src/math.ts: ExactRationalSchema, DimensionVectorSchema, MathPresentationSchema, MathProvenanceSchema, MathUncertaintySchema, MathNumSchema, MathQtySchema, MathSymSchema, MathUnparsedSchema, MathSymbolEntrySchema, MathUnitSchema, MathNormalisationContextSchema, SymbolTableSchema) -- see that test file's own top comment for why this is the only structural defence this generator has against silently drifting away from the schemas it's meant to describe. +// 2. content-json-schema-defs.test.ts (src/, run directly by vitest's "unit" project against source, never against dist), which imports this exact same CONTENT_DEFS value straight from here and asserts it stays byte-for-byte in step with a live z.toJSONSchema() call over each fragment's real exported Zod schema counterpart (ContentParagraphSchema, ContentRunSchema, ContentListMembershipSchema, ContentImageBlockSchema, ContentPageBreakSchema, ColorSchema, BoxSchema, LayoutFrameSchema, PageSizeSchema, MarginsSchema, AlignmentSchema, ContentStrokeStyleSchema, ContentBorderSchema, ContentCellBordersSchema, the package tree's non-recursive descriptors and anchors from src/package-node.ts: SectionDescriptorSchema, SlideDescriptorSchema, SheetDescriptorSchema, DrawPageDescriptorSchema, ShapeDescriptorSchema, HeadingParagraphSchema, ListParagraphSchema, the sheet grid and vector leaves from src/content.ts: ContentSheetCellSchema, ContentCellValueSchema, ContentSheetCellCommentSchema, ContentSheetColumnSchema, ContentSheetRowSchema, ContentSheetPrintSettingsSchema, ContentSheetPrintRangeSchema, ContentSheetRepeatRangeSchema, ContentSheetImageSchema, ContentStrokeSchema, ContentPathPointSchema, ContentPathSegmentSchema, ContentSubpathSchema, ContentVectorSchema, and the definitions facility from src/definitions.ts: StyleParagraphPropertiesSchema, StyleRunPropertiesSchema, StyleEntrySchema, DefinitionEntrySchema, the whole construct descriptor vocabulary from src/construct.ts: ContentControlDescriptorSchema, FieldDescriptorSchema, AnchorDescriptorSchema, LinkTargetSchema, LinkDescriptorSchema, ProvenanceDescriptorSchema, DivisionSourceSchema, DivisionDescriptorSchema, ConstructDescriptorSchema, plus the non-recursive math leaves from src/math.ts: ExactRationalSchema, DimensionVectorSchema, MathPresentationSchema, MathProvenanceSchema, MathUncertaintySchema, MathNumSchema, MathQtySchema, MathSymSchema, MathUnparsedSchema, MathSymbolEntrySchema, MathUnitSchema, MathNormalisationContextSchema, SymbolTableSchema) -- see that test file's own top comment for why this is the only structural defence this generator has against silently drifting away from the schemas it's meant to describe. // // If CONTENT_DEFS stayed inline in the .mjs script, only path 1 above would work: the script imports Zod schemas exclusively from '../dist/index.js' (a build artefact that may not exist, and per eslint.config.ts/tsconfig.json is deliberately excluded from both linting and typechecking, matching test/smoke.test.mjs's own precedent) -- a test that has to import through that path would only ever run after a build, which `pnpm test` (the "unit" vitest project, run standalone in CI's own "test" job, with no build step beforehand) never guarantees. Living here instead, this is an ordinary, fully typechecked and linted src module like any other -- CONTENT_DEFS just happens to be consumed by a script as well as by the package's own test suite. // -// The fragments below still cover exactly what scripts/generate-json-schemas.mjs's own top-of-file comment already explains: ContentBlockSchema, ContentEmbeddedObjectSchema, MathMlNodeSchema, and MathExpressionSchema are z.custom() predicates z.toJSONSchema() cannot introspect at all (recursion the pinned Zod version's z.lazy() can't express -- see src/content.ts's isContentBlock/isContentEmbeddedObject, src/mathml.ts's isMathMlNode, and src/math.ts's isMathExpression), so every schema reachable only through one of those four is transcribed by hand here, field-for-field, from the real Zod object definitions. The package tree added its own opaque set in the 4.0.0 major: DocumentPackageSchema's children reach the tree's per-kind group schemas (src/package-node.ts, all z.custom over recursive guards), so the whole PackageNode vocabulary -- container descriptors, anchor paragraphs, the seven group wrappers, and the sheet-image/vector leaves -- is transcribed here too, and the generator splices CONTENT_DEFS into document-package.schema.json as well as content-document.schema.json so both files resolve their local #/$defs pointers without depending on each other's file layout (the one deliberate cross-file ref stays $defs.ContentEmbeddedObject(Block)'s document pointer, CONTENT_DOCUMENT_URI). Three further schemas are transcribed despite being real z.objects themselves: ContentFormulaSchema (its mathml/content fields reach the opaque MathMlNodeSchema/MathExpressionSchema nodes, exactly like ContentTableCellSchema's blocks), SymbolTableSchema (transcribed so each ContentDocument arm's symbolTable field is one named $ref rather than five inlined copies of the whole unit-registry subtree), and now StyleEntrySchema/DefinitionEntrySchema (same five-copies reason for the package arms' styles/definitions fields) -- the generator's override() replaces each with a $ref to its fragment here. Anything transcribed here that DOES have a real, non-custom, exported Zod schema counterpart is exactly what content-json-schema-defs.test.ts holds to a live z.toJSONSchema() comparison; re-verify the rest (ContentTableCell/ContentTableRow/ContentTable, ContentEmbeddedObject(Block), the seven group wrappers, MathMlElement/MathMlNode, MathApp/MathSum/MathProd/MathMatrix/MathExpression, ContentFormula) against src/content.ts/src/package-node.ts/src/mathml.ts/src/math.ts by hand whenever those files' field shapes change, exactly as before. +// The fragments below still cover exactly what scripts/generate-json-schemas.mjs's own top-of-file comment already explains: ContentBlockSchema, ContentEmbeddedObjectSchema, MathMlNodeSchema, and MathExpressionSchema are z.custom() predicates z.toJSONSchema() cannot introspect at all (recursion the pinned Zod version's z.lazy() can't express -- see src/content.ts's isContentBlock/isContentEmbeddedObject, src/mathml.ts's isMathMlNode, and src/math.ts's isMathExpression), so every schema reachable only through one of those four is transcribed by hand here, field-for-field, from the real Zod object definitions. The package tree added its own opaque set in the 4.0.0 major: DocumentPackageSchema's children reach the tree's per-kind group schemas (src/package-node.ts, all z.custom over recursive guards), so the whole PackageNode vocabulary -- container descriptors, anchor paragraphs, the nine group wrappers (the seven of 4.0.0 plus 4.1.0's two construct groups), and the sheet-image/vector leaves -- is transcribed here too, and the generator splices CONTENT_DEFS into document-package.schema.json as well as content-document.schema.json so both files resolve their local #/$defs pointers without depending on each other's file layout (the one deliberate cross-file ref stays $defs.ContentEmbeddedObject(Block)'s document pointer, CONTENT_DOCUMENT_URI). Three further schemas are transcribed despite being real z.objects themselves: ContentFormulaSchema (its mathml/content fields reach the opaque MathMlNodeSchema/MathExpressionSchema nodes, exactly like ContentTableCellSchema's blocks), SymbolTableSchema (transcribed so each ContentDocument arm's symbolTable field is one named $ref rather than five inlined copies of the whole unit-registry subtree), and now StyleEntrySchema/DefinitionEntrySchema (same five-copies reason for the package arms' styles/definitions fields) -- the generator's override() replaces each with a $ref to its fragment here. Anything transcribed here that DOES have a real, non-custom, exported Zod schema counterpart is exactly what content-json-schema-defs.test.ts holds to a live z.toJSONSchema() comparison -- which is every construct descriptor fragment, since a descriptor is a plain z.strictObject reaching no opaque node; re-verify the rest (ContentTableCell/ContentTableRow/ContentTable, ContentEmbeddedObject(Block), the nine group wrappers, MathMlElement/MathMlNode, MathApp/MathSum/MathProd/MathMatrix/MathExpression, ContentFormula) against src/content.ts/src/package-node.ts/src/mathml.ts/src/math.ts by hand whenever those files' field shapes change, exactly as before. type JsonSchema = z.core.JSONSchema.JSONSchema; @@ -712,7 +712,7 @@ export const CONTENT_DEFS: Record = { required: ['objectKind', 'document', 'frame'], additionalProperties: false, }, - // The seven group wrappers, hand-verified alone (recursive through their children arrays): `{ node, style?, children }` where children's permitted members are exactly that group kind's own child types (src/package-node.ts's per-kind guards). A wordprocessing section's flow. + // The nine group wrappers, hand-verified alone (recursive through their children arrays): `{ node, style?, children }` where children's permitted members are exactly that group kind's own child types (src/package-node.ts's per-kind guards). A wordprocessing section's flow. SectionGroup: { type: 'object', properties: { @@ -720,7 +720,14 @@ export const CONTENT_DEFS: Record = { style: { type: 'string' }, children: { type: 'array', - items: { oneOf: [{ $ref: '#/$defs/HeadingGroup' }, { $ref: '#/$defs/ListGroup' }, { $ref: '#/$defs/ContentBlock' }] }, + items: { + oneOf: [ + { $ref: '#/$defs/HeadingGroup' }, + { $ref: '#/$defs/ListGroup' }, + { $ref: '#/$defs/SectionConstructGroup' }, + { $ref: '#/$defs/ContentBlock' }, + ], + }, }, }, required: ['node', 'children'], @@ -733,7 +740,14 @@ export const CONTENT_DEFS: Record = { style: { type: 'string' }, children: { type: 'array', - items: { oneOf: [{ $ref: '#/$defs/HeadingGroup' }, { $ref: '#/$defs/ListGroup' }, { $ref: '#/$defs/ContentBlock' }] }, + items: { + oneOf: [ + { $ref: '#/$defs/HeadingGroup' }, + { $ref: '#/$defs/ListGroup' }, + { $ref: '#/$defs/SectionConstructGroup' }, + { $ref: '#/$defs/ContentBlock' }, + ], + }, }, }, required: ['node', 'children'], @@ -746,7 +760,13 @@ export const CONTENT_DEFS: Record = { style: { type: 'string' }, children: { type: 'array', - items: { oneOf: [{ $ref: '#/$defs/ListGroup' }, { $ref: '#/$defs/ContentBlock' }] }, + items: { + oneOf: [ + { $ref: '#/$defs/ListGroup' }, + { $ref: '#/$defs/ShapeConstructGroup' }, + { $ref: '#/$defs/ContentBlock' }, + ], + }, }, }, required: ['node', 'children'], @@ -770,7 +790,13 @@ export const CONTENT_DEFS: Record = { style: { type: 'string' }, children: { type: 'array', - items: { oneOf: [{ $ref: '#/$defs/ListGroup' }, { $ref: '#/$defs/ContentBlock' }] }, + items: { + oneOf: [ + { $ref: '#/$defs/ListGroup' }, + { $ref: '#/$defs/ShapeConstructGroup' }, + { $ref: '#/$defs/ContentBlock' }, + ], + }, }, }, required: ['node', 'children'], @@ -804,7 +830,164 @@ export const CONTENT_DEFS: Record = { required: ['node', 'children'], additionalProperties: false, }, - // -- The definitions facility (src/definitions.ts), reached through DocumentPackageSchema's styles/definitions fields -- + // The two construct group wrappers (src/package-node.ts), hand-verified alone for the same reason as the seven above -- their children arrays recurse back through the same per-flow child unions. One per block flow: the section-scoped variant admits heading groups, the shape-scoped one does not, exactly as SectionChild and ShapeChild differ. A list item's flow takes the shape-scoped variant, since ListChild and ShapeChild admit the same members. + SectionConstructGroup: { + type: 'object', + properties: { + node: { $ref: '#/$defs/ConstructDescriptor' }, + style: { type: 'string' }, + children: { + type: 'array', + items: { + oneOf: [ + { $ref: '#/$defs/HeadingGroup' }, + { $ref: '#/$defs/ListGroup' }, + { $ref: '#/$defs/SectionConstructGroup' }, + { $ref: '#/$defs/ContentBlock' }, + ], + }, + }, + }, + required: ['node', 'children'], + additionalProperties: false, + }, + ShapeConstructGroup: { + type: 'object', + properties: { + node: { $ref: '#/$defs/ConstructDescriptor' }, + style: { type: 'string' }, + children: { + type: 'array', + items: { + oneOf: [ + { $ref: '#/$defs/ListGroup' }, + { $ref: '#/$defs/ShapeConstructGroup' }, + { $ref: '#/$defs/ContentBlock' }, + ], + }, + }, + }, + required: ['node', 'children'], + additionalProperties: false, + }, + // -- The construct descriptor vocabulary (src/construct.ts), the node payload of the two group wrappers above. Every fragment from here to ConstructDescriptor has a real, non-recursive, non-custom Zod counterpart, so all of them are held to the live z.toJSONSchema() comparison by content-json-schema-defs.test.ts rather than needing hand re-verification. -- + ContentControlDescriptor: { + type: 'object', + properties: { + kind: { type: 'string', const: 'contentControl' }, + controlType: { + type: 'string', + enum: [ + 'richText', + 'plainText', + 'checkbox', + 'dropDown', + 'comboBox', + 'date', + 'picture', + 'repeatingSection', + 'button', + 'index', + 'group', + ], + }, + tag: { type: 'string' }, + alias: { type: 'string' }, + lock: { type: 'string', enum: ['content', 'container', 'both'] }, + value: { type: 'string' }, + checked: { type: 'boolean' }, + options: { type: 'array', items: { type: 'string' } }, + }, + required: ['kind', 'controlType'], + additionalProperties: false, + }, + FieldDescriptor: { + type: 'object', + properties: { + kind: { type: 'string', const: 'field' }, + instruction: { type: 'string' }, + cachedResult: { type: 'string' }, + }, + required: ['kind', 'instruction'], + additionalProperties: false, + }, + AnchorDescriptor: { + type: 'object', + properties: { + kind: { type: 'string', const: 'anchor' }, + anchorType: { type: 'string', enum: ['bookmark', 'footnote', 'endnote', 'comment'] }, + name: { type: 'string' }, + definition: { type: 'string' }, + }, + required: ['kind', 'anchorType', 'name'], + additionalProperties: false, + }, + LinkTarget: { + oneOf: [ + { + type: 'object', + properties: { kind: { type: 'string', const: 'external' }, uri: { type: 'string' } }, + required: ['kind', 'uri'], + additionalProperties: false, + }, + { + type: 'object', + properties: { kind: { type: 'string', const: 'internal' }, anchor: { type: 'string' } }, + required: ['kind', 'anchor'], + additionalProperties: false, + }, + ], + }, + LinkDescriptor: { + type: 'object', + properties: { + kind: { type: 'string', const: 'link' }, + target: { $ref: '#/$defs/LinkTarget' }, + title: { type: 'string' }, + }, + required: ['kind', 'target'], + additionalProperties: false, + }, + ProvenanceDescriptor: { + type: 'object', + properties: { + kind: { type: 'string', const: 'provenance' }, + change: { type: 'string', enum: ['insertion', 'deletion', 'moveFrom', 'moveTo', 'formatChange'] }, + author: { type: 'string' }, + dateIso: { type: 'string' }, + }, + required: ['kind', 'change'], + additionalProperties: false, + }, + DivisionSource: { + type: 'object', + properties: { href: { type: 'string' }, sectionName: { type: 'string' } }, + required: ['href'], + additionalProperties: false, + }, + DivisionDescriptor: { + type: 'object', + properties: { + kind: { type: 'string', const: 'division' }, + name: { type: 'string' }, + columnCount: { type: 'integer', exclusiveMinimum: 0, maximum: MAX_SAFE_INTEGER }, + protected: { type: 'boolean' }, + source: { $ref: '#/$defs/DivisionSource' }, + }, + required: ['kind'], + additionalProperties: false, + }, + ConstructDescriptor: { + oneOf: [ + { $ref: '#/$defs/ContentControlDescriptor' }, + { $ref: '#/$defs/FieldDescriptor' }, + { $ref: '#/$defs/AnchorDescriptor' }, + { $ref: '#/$defs/LinkDescriptor' }, + { $ref: '#/$defs/ProvenanceDescriptor' }, + { $ref: '#/$defs/DivisionDescriptor' }, + ], + }, + // -- The definitions facility (src/definitions.ts), reached through DocumentPackageSchema's styles/definitions fields and, since 4.1.0, its layers/attachments/destinations tables -- StyleParagraphProperties: { type: 'object', properties: { diff --git a/src/definitions.ts b/src/definitions.ts index 326a56f..d7d5bb0 100644 --- a/src/definitions.ts +++ b/src/definitions.ts @@ -3,7 +3,7 @@ import { ColorSchema } from './color'; import { ContentListMembershipSchema, type ContentParagraph, type ContentRun } from './content'; import { AlignmentSchema } from './style'; -// The package-level definitions-table facility (ExaDev/document-schema.js#21): named tables at the DocumentPackage root whose entries tree nodes reference by string id, so repeated data is stated once and referenced many times. Styles are the first tenant (the StylesTableSchema below); link and footnote definitions are future tenants of the same mechanism (ExaDev/markdown-codec#63, ExaDev/document-schema.js#22) -- which is why the generic DefinitionsTableSchema exists alongside the styles-specific one rather than the facility being shaped around styles. This module defines the schemas and the pure resolution helpers only; minting entries (the frequency pass that factors repeated property tuples into table refs) is documents.js's boundary behaviour, not this package's. +// The package-level definitions-table facility (ExaDev/document-schema.js#21): named tables at the DocumentPackage root whose entries tree nodes reference by string id, so repeated data is stated once and referenced many times. Styles were the first tenant (the StylesTableSchema below), and the tenant-generic DefinitionsTableSchema beside it is what let every later tenant land without this module changing: link, footnote, and comment definitions (ExaDev/markdown-codec#63, ExaDev/document-schema.js#22) ride the `definitions` field, and 4.1.0's three construct tables -- `layers`, `attachments`, `destinations` (ExaDev/document-schema.js#24) -- are three more root fields of this same generic type rather than three parallel shapes. This module defines the schemas and the pure resolution helpers only; minting entries (the frequency pass that factors repeated property tuples into table refs) is documents.js's boundary behaviour, not this package's. // The paragraph half of a style entry: exactly the canonical ContentParagraph direct properties that a style may carry, and nothing else. Deliberately strict rather than plain: strictObject REJECTS a smuggled extra key instead of silently stripping it, which is what makes the ban list a schema-shape guarantee rather than a documented convention -- frames, sourcePath, and styleId are per-node facts (a position is a fact about a node, not a style; sourcePath and styleId identify the node and its producer-side style), so an entry carrying any of them fails validation outright instead of parsing to a value that quietly dropped them (ExaDev/document-schema.js#21's errata). export const StyleParagraphPropertiesSchema = z.strictObject({ @@ -40,7 +40,7 @@ export type StyleEntry = z.infer; export const StylesTableSchema = z.record(z.string(), StyleEntrySchema); export type StylesTable = z.infer; -// The tenant-generic half of the facility: any table of definitions whose entries are not styles. Each entry carries a `kind` string naming its tenant (a future link definition is { kind: 'link', url, ... }, a footnote definition { kind: 'footnote', ... }) and an open body belonging to that tenant's own vocabulary -- this package defines the mechanism and the discriminator, never the per-tenant fields, so a new tenant lands additively without this schema changing. Deliberately loose rather than strict: the whole point is that the body's keys are not this package's to enumerate, so unknown keys are preserved through a parse rather than stripped. +// The tenant-generic half of the facility: any table of definitions whose entries are not styles -- the type of the `definitions` root field and of the `layers`/`attachments`/`destinations` tables beside it (src/package.ts). Each entry carries a `kind` string naming its tenant (a link definition is { kind: 'link', url, ... }, a footnote definition { kind: 'footnote', ... }, a PDF optional-content group { kind: 'layer', ... }) and an open body belonging to that tenant's own vocabulary -- this package defines the mechanism and the discriminator, never the per-tenant fields, so a new tenant lands additively without this schema changing. Deliberately loose rather than strict: the whole point is that the body's keys are not this package's to enumerate, so unknown keys are preserved through a parse rather than stripped. export const DefinitionEntrySchema = z.looseObject({ kind: z.string(), }); diff --git a/src/index.ts b/src/index.ts index 908a452..b2c0eef 100644 --- a/src/index.ts +++ b/src/index.ts @@ -6,6 +6,7 @@ export * from './mathml'; export * from './math'; export * from './content'; export * from './definitions'; +export * from './construct'; export * from './package-node'; export * from './package'; export * from './codec'; diff --git a/src/package-node.test.ts b/src/package-node.test.ts index 89ab6a2..a304c69 100644 --- a/src/package-node.test.ts +++ b/src/package-node.test.ts @@ -10,7 +10,9 @@ import { PackageGroupSchema, PackageLeafSchema, PackageNodeSchema, + SectionConstructGroupSchema, SectionGroupSchema, + ShapeConstructGroupSchema, ShapeGroupSchema, SheetGroupSchema, SlideGroupSchema, @@ -18,7 +20,9 @@ import { type HeadingGroupNode, type ListGroupNode, type PackageNode, + type SectionConstructGroupNode, type SectionGroupNode, + type ShapeConstructGroupNode, type ShapeGroupNode, type SheetGroupNode, type SlideGroupNode, @@ -221,6 +225,210 @@ describe('the package tree accepts a real tree of every kind', () => { }); }); +// The construct groups (ExaDev/document-schema.js#24) at every tree position they are legal in, and every position they are not. The 4.0.0 trees above stand unchanged beside these: the kinds are additive, so nothing that parsed before parses differently now. +describe('construct groups wrap block extents wherever block flow runs', () => { + it('accepts a docx block SDT wrapping a heading group and its body, in a section flow', () => { + const control: SectionConstructGroupNode = { + node: { kind: 'contentControl', controlType: 'richText', tag: 'ClientBlock', lock: 'container' }, + children: [ + { + node: { kind: 'paragraph', headingLevel: 2, runs: [run('Client')] }, + children: [{ kind: 'paragraph', runs: [run('Acme Ltd')] }], + }, + ], + }; + expect(SectionConstructGroupSchema.safeParse(control).success).toBe(true); + const section: SectionGroupNode = { + node: { kind: 'section', pageSize: PAGE, margins: MARGINS }, + children: [control], + }; + expect(SectionGroupSchema.safeParse(section).success).toBe(true); + }); + + it("accepts a docx TOC field whose cached result is the extent it contains -- the kind's extent-as-containment shape", () => { + const field: SectionConstructGroupNode = { + node: { kind: 'field', instruction: 'TOC \\o "1-3" \\h' }, + children: [ + { kind: 'paragraph', runs: [run('1. Introduction\t3')] }, + { kind: 'paragraph', runs: [run('2. Method\t9')] }, + ], + }; + expect(SectionConstructGroupSchema.safeParse(field).success).toBe(true); + }); + + it('accepts a point anchor (a footnote marker, no children) and a ranged anchor (a comment extent) side by side', () => { + const section: SectionGroupNode = { + node: { kind: 'section', pageSize: PAGE, margins: MARGINS }, + children: [ + { node: { kind: 'anchor', anchorType: 'footnote', name: '1', definition: 'n1' }, children: [] }, + { + node: { kind: 'anchor', anchorType: 'comment', name: 'c1', definition: 'c1' }, + children: [{ kind: 'paragraph', runs: [run('The commented sentence.')] }], + }, + ], + }; + expect(SectionGroupSchema.safeParse(section).success).toBe(true); + }); + + it('accepts nested constructs -- a tracked deletion inside a content control inside a division', () => { + const division: SectionConstructGroupNode = { + node: { kind: 'division', name: 'Chapter1', columnCount: 2, protected: true }, + children: [ + { + node: { kind: 'contentControl', controlType: 'richText', tag: 'Body' }, + children: [ + { + node: { kind: 'provenance', change: 'deletion', author: 'A. Reviewer', dateIso: '2026-08-18T09:00:00Z' }, + children: [{ kind: 'paragraph', runs: [run('Struck sentence.')] }], + }, + ], + }, + ], + }; + expect(SectionConstructGroupSchema.safeParse(division).success).toBe(true); + expect(isPackageGroup(division)).toBe(true); + expect(isPackageNode(division)).toBe(true); + }); + + it("accepts a pptx a:fld and an internal slide-jump link inside a shape's flow", () => { + const shape: ShapeGroupNode = { + node: { + frame: { xPt: 10, yPt: 10, widthPt: 200, heightPt: 100 }, + insetLeftPt: 0, + insetTopPt: 0, + insetRightPt: 0, + insetBottomPt: 0, + }, + children: [ + { node: { kind: 'field', instruction: 'slidenum', cachedResult: '7' }, children: [] }, + { + node: { kind: 'link', target: { kind: 'internal', anchor: 'slide12' } }, + children: [{ kind: 'paragraph', runs: [run('Jump to the summary')] }], + }, + ], + }; + expect(ShapeGroupSchema.safeParse(shape).success).toBe(true); + expect(SlideGroupSchema.safeParse({ node: { kind: 'slide', size: PAGE, notes: '' }, children: [shape] }).success).toBe( + true, + ); + }); + + it("accepts a construct inside a list item's flow, which admits the shape-scoped variant", () => { + const bookmark: ShapeConstructGroupNode = { + node: { kind: 'anchor', anchorType: 'bookmark', name: 'item1' }, + children: [{ kind: 'paragraph', runs: [run('Item detail')] }], + }; + const list: ListGroupNode = { + node: { kind: 'paragraph', list: { level: 0 }, runs: [run('Item')] }, + children: [bookmark], + }; + expect(ListGroupSchema.safeParse(list).success).toBe(true); + expect(ShapeConstructGroupSchema.safeParse(bookmark).success).toBe(true); + }); + + it('carries a style ref on a construct group, exactly as on every other group wrapper', () => { + const styled: SectionConstructGroupNode = { + node: { kind: 'division', name: 'Chapter1' }, + style: 's1', + children: [], + }; + expect(SectionConstructGroupSchema.safeParse(styled).success).toBe(true); + }); + + it('round-trips a construct-bearing tree through JSON and revalidates identically', () => { + const section: SectionGroupNode = { + node: { kind: 'section', pageSize: PAGE, margins: MARGINS }, + children: [ + { + node: { kind: 'link', target: { kind: 'external', uri: 'https://example.com' }, title: 'Example' }, + children: [{ kind: 'paragraph', runs: [run('Linked block')] }], + }, + ], + }; + const roundTripped: unknown = JSON.parse(JSON.stringify(section)); + expect(SectionGroupSchema.parse(roundTripped)).toEqual(section); + expect(PackageGroupSchema.safeParse(roundTripped).success).toBe(true); + }); +}); + +// Pins the ordering invariant in package-node.ts's group guards: each tests its own node payload before walking its children, so at most one arm of a child predicate ever descends. Were a guard to walk first and reject on the node afterwards, every arm would pay for the whole subtree before failing and validating one tree would cost (arms per flow) to the power of its depth -- at the depth below, that is 3^30 predicate calls, so this test fails by timing out rather than by assertion if the order is ever reversed. +describe('validating a deeply nested tree stays linear in its size', () => { + // Thirty construct groups deep, with the only thing that decides the verdict sitting at the very bottom -- the case that forces the whole subtree to be walked before any answer is possible. + function nestConstructs(depth: number, innermostLeaf: unknown): unknown { + let node: unknown = { node: { kind: 'division', name: 'innermost' }, children: [innermostLeaf] }; + for (let level = 0; level < depth; level += 1) { + node = { node: { kind: 'contentControl', controlType: 'richText' }, children: [node] }; + } + return node; + } + + it('accepts a thirty-deep construct chain, and rejects the same chain with an illegal leaf at the bottom', () => { + expect(SectionConstructGroupSchema.safeParse(nestConstructs(30, { kind: 'paragraph', runs: [run('Leaf')] })).success).toBe( + true, + ); + // A style ref on a bare leaf: legal on a group wrapper, never on a leaf, and only discoverable at the deepest level. + expect( + SectionConstructGroupSchema.safeParse(nestConstructs(30, { kind: 'paragraph', runs: [run('Leaf')], style: 's1' })) + .success, + ).toBe(false); + }); +}); + +describe('construct groups reject the positions and shapes they are not legal in', () => { + it("rejects a heading group inside a shape-scoped construct -- a shape's flow carries no heading hierarchy", () => { + const broken = { + node: { kind: 'contentControl', controlType: 'richText' }, + children: [{ node: { kind: 'paragraph', headingLevel: 1, runs: [run('H')] }, children: [] }], + }; + expect(ShapeConstructGroupSchema.safeParse(broken).success).toBe(false); + expect(SectionConstructGroupSchema.safeParse(broken).success).toBe(true); + }); + + it('rejects a construct group as a direct child of a slide, a sheet, or a drawing page -- those hold containers and leaves, not block flow', () => { + const construct = { node: { kind: 'anchor', anchorType: 'bookmark', name: 'b1' }, children: [] }; + expect( + SlideGroupSchema.safeParse({ node: { kind: 'slide', size: PAGE, notes: '' }, children: [construct] }).success, + ).toBe(false); + expect(SheetGroupSchema.safeParse({ ...sheetGroup(), children: [construct] }).success).toBe(false); + expect(DrawPageGroupSchema.safeParse({ node: { kind: 'drawPage', size: PAGE }, children: [construct] }).success).toBe( + false, + ); + }); + + it('rejects a malformed descriptor at the node position -- an unknown control type does not become a bare wrapper', () => { + const broken = { node: { kind: 'contentControl', controlType: 'w:sdt' }, children: [] }; + expect(SectionConstructGroupSchema.safeParse(broken).success).toBe(false); + expect(isPackageGroup(broken)).toBe(false); + }); + + it('rejects a construct group carrying a key outside { node, style, children }', () => { + const broken = { + node: { kind: 'division', name: 'Chapter1' }, + residue: '', + children: [], + }; + expect(SectionConstructGroupSchema.safeParse(broken).success).toBe(false); + }); + + it('rejects a construct descriptor posed as a bare leaf -- a descriptor is a node payload, never a block', () => { + expect(isPackageLeaf({ kind: 'anchor', anchorType: 'bookmark', name: 'b1' })).toBe(false); + const section = sectionGroup(); + const withDescriptorLeaf = { + ...section, + children: [...section.children, { kind: 'anchor', anchorType: 'bookmark', name: 'b1' }], + }; + expect(SectionGroupSchema.safeParse(withDescriptorLeaf).success).toBe(false); + }); + + it('rejects a malformed leaf inside a construct extent, so a construct wrapper never launders bad content', () => { + const broken: unknown = { + node: { kind: 'provenance', change: 'insertion' }, + children: [{ kind: 'image', format: 'png', base64: 'aGk=', widthPt: 'wide', heightPt: 50 }], + }; + expect(SectionConstructGroupSchema.safeParse(broken).success).toBe(false); + }); +}); + describe('the package tree rejects near-misses', () => { it('rejects a group wrapper with no children array', () => { const broken = { node: { kind: 'section', pageSize: PAGE, margins: MARGINS } }; diff --git a/src/package-node.ts b/src/package-node.ts index dcf68a3..139ac22 100644 --- a/src/package-node.ts +++ b/src/package-node.ts @@ -18,8 +18,9 @@ import { type ContentSheetImage, type ContentVector, } from './content'; +import { ConstructDescriptorSchema, type ConstructDescriptor } from './construct'; -// The package tree's node vocabulary (ExaDev/document-schema.js#20's promoted DocumentPackage, as proven by document-outline.js's phase-1 reference implementation -- this module is that shape's schema-home port). Groups are `{ node, children }` where node embeds either an anchor paragraph (heading and list groups carry the full ContentParagraph, runs and formatting and frames included, never a projected text label) or a container descriptor (section / slide / sheet / drawPage, each tagged with a `kind` the flat container type does not carry). Bare leaves carry their own `kind` and never `children` -- discrimination is structural on node+children, because the earlier "anything with kind is a leaf" rule collided with `{ kind: 'slide' }` groups. Grouping never crosses container boundaries: a shape is its own group with its inner blocks grouped inside it, a sheet's grid rides on the sheet node, and an embedded document (the recursive ContentEmbeddedObject arm) stays intact as one leaf. A group may additionally carry `style` -- a string ref into the package's styles table (ExaDev/document-schema.js#21); refs exist only here, never on ContentDocument nodes, so the flat codec-exchange form is always fully materialised. +// The package tree's node vocabulary (ExaDev/document-schema.js#20's promoted DocumentPackage, as proven by document-outline.js's phase-1 reference implementation -- this module is that shape's schema-home port). Groups are `{ node, children }` where node embeds either an anchor paragraph (heading and list groups carry the full ContentParagraph, runs and formatting and frames included, never a projected text label) or a container descriptor (section / slide / sheet / drawPage, each tagged with a `kind` the flat container type does not carry). Bare leaves carry their own `kind` and never `children` -- discrimination is structural on node+children, because the earlier "anything with kind is a leaf" rule collided with `{ kind: 'slide' }` groups. Grouping never crosses container boundaries: a shape is its own group with its inner blocks grouped inside it, a sheet's grid rides on the sheet node, and an embedded document (the recursive ContentEmbeddedObject arm) stays intact as one leaf. A group may additionally carry `style` -- a string ref into the package's styles table (ExaDev/document-schema.js#21); refs exist only here, never on ContentDocument nodes, so the flat codec-exchange form is always fully materialised. Since 4.1.0 a group's node may also be a construct descriptor (src/construct.ts, ExaDev/document-schema.js#24) -- a contentControl, field, anchor, link, provenance, or division wrapping the block extent it spans -- which is what the tree's construct-capable-from-day-one design was for: the kinds landed additively, so a 4.0.0 tree carrying none of them parses identically under this release. // The descriptors are built from the content schemas themselves by omit+extend rather than re-declared field by field, so a field added to a container schema in a future release rides its descriptor automatically -- the zod-first spelling of the reference implementation's `Omit & { kind: 'section' }` types. Each is strict: the omitted array (the one whose members became the group's children) is rejected, not merely absent, so a raw flat container smuggled in as a descriptor fails validation instead of parsing to a descriptor that silently dropped its content. export const SectionDescriptorSchema = ContentSectionSchema.omit({ blocks: true }) @@ -61,14 +62,14 @@ export type ListParagraph = z.infer; // The leaf payloads of a package tree, across all five document kinds: wordprocessing/presentation/drawing block flow yields ContentBlock leaves, spreadsheets additionally yield sheet-anchored images (ContentSheetImage) and whole embedded documents (ContentEmbeddedObject, which is not itself a ContentBlock -- it has no `kind` discriminator), drawings yield textless vector primitives (ContentVector), and a formula document yields its single ContentFormula. One union, so one guard set serves every kind. export type PackageLeaf = ContentBlock | ContentSheetImage | ContentEmbeddedObject | ContentVector | ContentFormula; -// What a section's (or a heading group's) block flow holds: heading groups, list groups, and bare block leaves. Headings nest under headings and lists nest inside the open heading scope or under deeper list items; a plain paragraph is a leaf. -export type SectionChild = HeadingGroupNode | ListGroupNode | ContentBlock; +// What a section's (or a heading group's) block flow holds: heading groups, list groups, construct groups, and bare block leaves. Headings nest under headings and lists nest inside the open heading scope or under deeper list items; a plain paragraph is a leaf. +export type SectionChild = HeadingGroupNode | ListGroupNode | SectionConstructGroupNode | ContentBlock; -// What a shape's block flow holds: list groups and bare leaves only. Shapes carry no heading hierarchy of their own -- list.level is the only depth signal a slide or drawing shape's paragraphs actually carry -- so a paragraph with headingLevel but no list membership sits flat as a leaf here. -export type ShapeChild = ListGroupNode | ContentBlock; +// What a shape's block flow holds: list groups, construct groups, and bare leaves only. Shapes carry no heading hierarchy of their own -- list.level is the only depth signal a slide or drawing shape's paragraphs actually carry -- so a paragraph with headingLevel but no list membership sits flat as a leaf here. +export type ShapeChild = ListGroupNode | ShapeConstructGroupNode | ContentBlock; -// A list group's children: deeper list groups and block leaves. A heading never appears below a list group, because opening a heading resets the list nesting before it opens its own group. -export type ListChild = ListGroupNode | ContentBlock; +// A list group's children: deeper list groups, construct groups, and block leaves. A heading never appears below a list group, because opening a heading resets the list nesting before it opens its own group. The construct group here is the shape-scoped one for exactly that reason: a list item's flow and a shape's flow admit the same children, as ListChild and ShapeChild have always spelled identically. +export type ListChild = ListGroupNode | ShapeConstructGroupNode | ContentBlock; // What a sheet's children are: its anchored images and its whole embedded documents, in that order (the two live in sibling arrays with no cross-array ordering field, and flatten's type partition reverses this fixed order). Cells are addressable data, never children -- they ride the sheet descriptor. export type SheetChild = ContentSheetImage | ContentEmbeddedObject; @@ -118,6 +119,19 @@ export interface ListGroupNode { children: ListChild[]; } +// The construct groups (src/construct.ts, ExaDev/document-schema.js#24): the same `{ node, children }` wrapper every other group uses, with a construct descriptor as its node and the extent it spans as its children. Two variants, because a construct is transparent to the flow it wraps and the two block flows admit different children: one in a section's or heading group's flow (where headings may nest), one in a shape's or list item's (where they may not). A construct group nests in and around every other group -- a provenance wrapper inside a content control inside a division is a real docx shape -- because both variants are members of the very child unions their own children are drawn from. +export interface SectionConstructGroupNode { + readonly node: ConstructDescriptor; + style?: string; + children: SectionChild[]; +} + +export interface ShapeConstructGroupNode { + readonly node: ConstructDescriptor; + style?: string; + children: ShapeChild[]; +} + export type PackageGroup = | SectionGroupNode | SlideGroupNode @@ -125,7 +139,9 @@ export type PackageGroup = | DrawPageGroupNode | ShapeGroupNode | HeadingGroupNode - | ListGroupNode; + | ListGroupNode + | SectionConstructGroupNode + | ShapeConstructGroupNode; export type PackageNode = PackageGroup | PackageLeaf; @@ -148,15 +164,20 @@ function isLeafChild(schema: z.ZodType, value: unknown): boolean { } function isSectionChild(value: unknown): value is SectionChild { - return isHeadingGroupNode(value) || isListGroupNode(value) || isLeafChild(ContentBlockSchema, value); + return ( + isHeadingGroupNode(value) || + isListGroupNode(value) || + isSectionConstructGroupNode(value) || + isLeafChild(ContentBlockSchema, value) + ); } function isShapeChild(value: unknown): value is ShapeChild { - return isListGroupNode(value) || isLeafChild(ContentBlockSchema, value); + return isListGroupNode(value) || isShapeConstructGroupNode(value) || isLeafChild(ContentBlockSchema, value); } function isListChild(value: unknown): value is ListChild { - return isListGroupNode(value) || isLeafChild(ContentBlockSchema, value); + return isListGroupNode(value) || isShapeConstructGroupNode(value) || isLeafChild(ContentBlockSchema, value); } function isSheetChild(value: unknown): value is SheetChild { @@ -167,40 +188,48 @@ function isDrawPageChild(value: unknown): value is DrawPageChild { return isShapeGroupNode(value) || isLeafChild(ContentVectorSchema, value); } +// Every group guard tests its own node payload BEFORE walking its children, and the order is load-bearing rather than stylistic: a child predicate tries each group arm in turn, so a guard that walked the whole subtree first and only then rejected on the node would make each arm pay for the entire subtree before failing, and the cost of validating one tree would be (arms per flow) raised to the power of its depth. Node payloads discriminate in constant time and no two group kinds share one (container descriptors carry their own container kind, construct descriptors their construct kind, and the two paragraph anchors are the only pair that can both match one node), so checking the node first means at most one arm ever descends. The conjunction is otherwise unchanged -- both halves are pure predicates, so the order affects only how much work a rejection does. export function isSectionGroupNode(value: unknown): value is SectionGroupNode { return ( - isRecord(value) && - isGroupWrapper(value, isSectionChild) && - SectionDescriptorSchema.safeParse(value.node).success + isRecord(value) && SectionDescriptorSchema.safeParse(value.node).success && isGroupWrapper(value, isSectionChild) ); } export function isSlideGroupNode(value: unknown): value is SlideGroupNode { - return isRecord(value) && isGroupWrapper(value, isShapeGroupNode) && SlideDescriptorSchema.safeParse(value.node).success; + return isRecord(value) && SlideDescriptorSchema.safeParse(value.node).success && isGroupWrapper(value, isShapeGroupNode); } export function isSheetGroupNode(value: unknown): value is SheetGroupNode { - return isRecord(value) && isGroupWrapper(value, isSheetChild) && SheetDescriptorSchema.safeParse(value.node).success; + return isRecord(value) && SheetDescriptorSchema.safeParse(value.node).success && isGroupWrapper(value, isSheetChild); } export function isDrawPageGroupNode(value: unknown): value is DrawPageGroupNode { return ( - isRecord(value) && - isGroupWrapper(value, isDrawPageChild) && - DrawPageDescriptorSchema.safeParse(value.node).success + isRecord(value) && DrawPageDescriptorSchema.safeParse(value.node).success && isGroupWrapper(value, isDrawPageChild) ); } export function isShapeGroupNode(value: unknown): value is ShapeGroupNode { - return isRecord(value) && isGroupWrapper(value, isShapeChild) && ShapeDescriptorSchema.safeParse(value.node).success; + return isRecord(value) && ShapeDescriptorSchema.safeParse(value.node).success && isGroupWrapper(value, isShapeChild); } export function isHeadingGroupNode(value: unknown): value is HeadingGroupNode { - return isRecord(value) && isGroupWrapper(value, isSectionChild) && HeadingParagraphSchema.safeParse(value.node).success; + return isRecord(value) && HeadingParagraphSchema.safeParse(value.node).success && isGroupWrapper(value, isSectionChild); } export function isListGroupNode(value: unknown): value is ListGroupNode { - return isRecord(value) && isGroupWrapper(value, isListChild) && ListParagraphSchema.safeParse(value.node).success; + return isRecord(value) && ListParagraphSchema.safeParse(value.node).success && isGroupWrapper(value, isListChild); +} + +// The two construct-group guards. Nothing else in the vocabulary can be confused for one: every other group's node is either a container descriptor tagged with its own container kind or a ContentParagraph, and no construct descriptor's `kind` collides with any of those, so the child predicates' arm order carries no first-match hazard. +export function isSectionConstructGroupNode(value: unknown): value is SectionConstructGroupNode { + return ( + isRecord(value) && ConstructDescriptorSchema.safeParse(value.node).success && isGroupWrapper(value, isSectionChild) + ); +} + +export function isShapeConstructGroupNode(value: unknown): value is ShapeConstructGroupNode { + return isRecord(value) && ConstructDescriptorSchema.safeParse(value.node).success && isGroupWrapper(value, isShapeChild); } // Leaf validation delegates to the content model's own exported schemas rather than hand-rolling a second, parallel structural guard per payload -- the shapes are src/content.ts's to own, and a hand copy here would drift the first time a schema field changes. The union's first-match-wins order is safe because no leaf type is a structural subset of a later member that would change the verdict. @@ -224,7 +253,9 @@ export function isPackageGroup(value: unknown): value is PackageGroup { isDrawPageGroupNode(value) || isShapeGroupNode(value) || isHeadingGroupNode(value) || - isListGroupNode(value) + isListGroupNode(value) || + isSectionConstructGroupNode(value) || + isShapeConstructGroupNode(value) ); } @@ -240,6 +271,8 @@ export const DrawPageGroupSchema = z.custom(isDrawPageGroupNo export const ShapeGroupSchema = z.custom(isShapeGroupNode); export const HeadingGroupSchema = z.custom(isHeadingGroupNode); export const ListGroupSchema = z.custom(isListGroupNode); +export const SectionConstructGroupSchema = z.custom(isSectionConstructGroupNode); +export const ShapeConstructGroupSchema = z.custom(isShapeConstructGroupNode); export const PackageGroupSchema = z.custom(isPackageGroup); export const PackageLeafSchema = z.custom(isPackageLeaf); export const PackageNodeSchema = z.custom(isPackageNode); diff --git a/src/package.test.ts b/src/package.test.ts index b6dcf73..a09ccd6 100644 --- a/src/package.test.ts +++ b/src/package.test.ts @@ -196,3 +196,100 @@ describe('DocumentPackageSchema round trips (tree form)', () => { expect(parsed.symbolTable).toEqual({ symbols: [], units: [] }); }); }); + +// The 4.1.0 additions at the root (ExaDev/document-schema.js#24): three more tables of the same generic type `definitions` already uses, and the additivity guarantee that made them a minor. +describe('the construct tables at the package root', () => { + it('accepts layers, attachments, and destinations, each its own key namespace over kind-tagged entries', () => { + const withTables = { + ...wordprocessingPackage(), + layers: { + ocg1: { kind: 'layer', name: 'Watermark', defaultVisible: false }, + d: { kind: 'layerConfig', baseState: 'ON', order: ['ocg1'] }, + }, + attachments: { a1: { kind: 'attachment', fileName: 'source.csv', description: 'The source data' } }, + destinations: { + ch1: { kind: 'destination', pageIndex: 0 }, + o1: { kind: 'outline', title: 'Chapter 1', destination: 'ch1' }, + }, + }; + const parsed = DocumentPackageSchema.parse(withTables); + expect(parsed.layers?.ocg1).toEqual({ kind: 'layer', name: 'Watermark', defaultVisible: false }); + expect(parsed.attachments?.a1).toEqual({ kind: 'attachment', fileName: 'source.csv', description: 'The source data' }); + expect(parsed.destinations?.o1).toEqual({ kind: 'outline', title: 'Chapter 1', destination: 'ch1' }); + }); + + it('lets one name appear in more than one table without collision -- separate root fields are separate namespaces', () => { + const withTables = { + ...wordprocessingPackage(), + layers: { x: { kind: 'layer', name: 'Layer x' } }, + destinations: { x: { kind: 'destination', pageIndex: 3 } }, + }; + const parsed = DocumentPackageSchema.parse(withTables); + expect(parsed.layers?.x).toEqual({ kind: 'layer', name: 'Layer x' }); + expect(parsed.destinations?.x).toEqual({ kind: 'destination', pageIndex: 3 }); + }); + + it('requires the kind discriminator on every entry, exactly as the definitions table does', () => { + for (const field of ['layers', 'attachments', 'destinations']) { + const broken = { ...wordprocessingPackage(), [field]: { e1: { name: 'no kind here' } } }; + expect(DocumentPackageSchema.safeParse(broken).success).toBe(false); + } + }); + + it('preserves each tenant body through a JSON round trip, since the entry body is the tenant vocabulary and not this package to strip', () => { + const original = { + ...wordprocessingPackage(), + attachments: { a1: { kind: 'attachment', fileName: 'source.csv', bytesBase64: 'aGk=' } }, + }; + const roundTripped: unknown = JSON.parse(JSON.stringify(DocumentPackageSchema.parse(original))); + expect(DocumentPackageSchema.parse(roundTripped)).toEqual(original); + }); +}); + +describe('the construct kinds are additive over 4.0.0', () => { + it('parses a 4.0.0 tree carrying none of the new kinds or tables, unchanged and field for field', () => { + for (const original of [wordprocessingPackage(), spreadsheetPackage(), formulaPackage()]) { + const parsed = DocumentPackageSchema.parse(original); + expect(parsed).toEqual(original); + expect(parsed).not.toHaveProperty('layers'); + expect(parsed).not.toHaveProperty('attachments'); + expect(parsed).not.toHaveProperty('destinations'); + } + }); + + it('parses a package whose tree carries construct groups, at the root children position it belongs under', () => { + const withConstructs: DocumentPackage = { + kind: 'wordprocessing', + metadata: {}, + definitions: { n1: { kind: 'footnote', blocks: [] } }, + children: [ + { + node: { kind: 'section', pageSize: PAGE, margins: MARGINS }, + children: [ + { + node: { kind: 'division', name: 'Chapter1' }, + children: [ + { node: { kind: 'anchor', anchorType: 'footnote', name: '1', definition: 'n1' }, children: [] }, + { + node: { kind: 'provenance', change: 'insertion', author: 'A. Reviewer' }, + children: [{ kind: 'paragraph', runs: [{ text: 'Inserted sentence.' }] }], + }, + ], + }, + ], + }, + ], + }; + const roundTripped: unknown = JSON.parse(JSON.stringify(DocumentPackageSchema.parse(withConstructs))); + expect(DocumentPackageSchema.parse(roundTripped)).toEqual(withConstructs); + }); + + it('still rejects a construct group at the root children position -- a package holds containers, not extents', () => { + const broken = { + kind: 'wordprocessing', + metadata: {}, + children: [{ node: { kind: 'division', name: 'Chapter1' }, children: [] }], + }; + expect(DocumentPackageSchema.safeParse(broken).success).toBe(false); + }); +}); diff --git a/src/package.ts b/src/package.ts index 60df8ca..681001b 100644 --- a/src/package.ts +++ b/src/package.ts @@ -5,19 +5,23 @@ import { PageSizeSchema } from './geometry'; import { LayoutMetadataSchema } from './metadata'; import { DrawPageGroupSchema, SectionGroupSchema, SheetGroupSchema, SlideGroupSchema } from './package-node'; -// DocumentPackage is the single hierarchical artefact: structure, layout, and content fused in one tree (ExaDev/document-schema.js#20). The root carries what no tree node can -- the document kind (moved up from the retired flat `content` field; the empty documents are legal, so the kind cannot be inferred from the children and the envelope keeps it explicit), the required metadata, the optional document-level symbolTable (the same shared fields every ContentDocument arm spreads -- one declaration, spliced in from src/content.ts), and the envelope's three optional tables and arrays: `pages` (each rendered page's own size, indexed to match every content node's own `frames[].pageIndex` -- present once a layout pass has run, absent for a content-only package), `styles` and `definitions` (the package-level definitions-table facility, src/definitions.ts). Everything structural hangs off `children`: one group per top-level container (a section, slide, sheet, or draw page), each holding its own content tree -- see src/package-node.ts for the node vocabulary and its structural discrimination rule. +// DocumentPackage is the single hierarchical artefact: structure, layout, and content fused in one tree (ExaDev/document-schema.js#20). The root carries what no tree node can -- the document kind (moved up from the retired flat `content` field; the empty documents are legal, so the kind cannot be inferred from the children and the envelope keeps it explicit), the required metadata, the optional document-level symbolTable (the same shared fields every ContentDocument arm spreads -- one declaration, spliced in from src/content.ts), and the envelope's optional tables and arrays: `pages` (each rendered page's own size, indexed to match every content node's own `frames[].pageIndex` -- present once a layout pass has run, absent for a content-only package), and the package-level table facility of src/definitions.ts -- `styles`, `definitions`, and the three construct tables `layers`/`attachments`/`destinations` added in 4.1.0. Everything structural hangs off `children`: one group per top-level container (a section, slide, sheet, or draw page), each holding its own content tree -- see src/package-node.ts for the node vocabulary and its structural discrimination rule. // The package tree and the flat ContentDocument are one format in two encodings, related by three laws (stated on the issues and proven property-wise by document-outline.js's decompose/flatten over real corpus documents, with documents.js re-running the same assertions over its own corpus at the package boundary): (i) strict structural equality holds both directions for a table-free package -- decompose(flatten(pkg)) and flatten(decompose(pkg)) reproduce it exactly; (ii) effective-property equality holds universally -- once styles are resolved (resolve-then-compare, src/definitions.ts), a factored and an unfactored serialisation of one document compare equal; (iii) minting is idempotent -- factoring a second time mints the identical table. The codecs keep producing flat ContentDocuments (their natural reading shape); decomposition runs once at the package boundary in documents.js and flatten runs once where a builder consumes a package. // This is a genuinely breaking shape change from the previous `{ formatVersion, content, pages }` envelope, which is why it rides a major (4.0.0). The old envelope's `formatVersion` field is gone with no replacement field: a serialised package states its version through the release-pinned $schema URI its dumper stamped (documentPackageWithSchema, src/schema-io.ts), and an ingesting documentFromJson dispatches on that URI -- the URI is the version, not a hand-kept integer. ContentDocument (the flat codec-exchange form) survives unchanged in role minus its own retired formatVersion literal, and nothing about the content model itself changed: every block, run, cell, and frame field a 3.x package carried still validates in its old flat shape -- only the envelope around it moved. // The five arms duplicate their kind literals rather than factoring through a base schema, because z.discriminatedUnion() needs each member as a plain z.object carrying its own literal `kind` field in place (the same reason ContentDocumentSchema's own arms spread contentDocumentSharedFields); the children type is the one thing that differs per arm, and the union says exactly which root group each kind takes -- a wordprocessing package of section groups, a presentation of slide groups, a spreadsheet of sheet groups, a drawing of drawPage groups, and a formula package whose single child is the ContentFormula leaf itself (a formula has no container structure to group). +// The three package-level tables the pdf inventory proposed (ExaDev/pdf-codec#66, landed by ExaDev/document-schema.js#24) are additive optional root fields typed as the SAME generic definitions table the `definitions` field already uses -- kind-tagged loose entries, no new entry shape minted anywhere. Separate root fields rather than three more tenants of `definitions` for the reason `styles` is its own field despite being the facility's first tenant: each table is its own key namespace, so a layer and a destination may share a name without colliding, and a consumer reaches the table it wants without filtering. The `kind` discriminator still earns its keep inside each of them, because each table holds more than one tenant: a layers table carries optional-content group definitions alongside their configuration and radio-button-group entries, and a destinations table carries named destinations alongside the outline/navigation entries the same PDF names-tree walk produces -- ExaDev/document-schema.js#24 names that table "navigation/destinations" for exactly that reason. Per-tenant entry fields stay the tenant's own, never this package's, exactly as src/definitions.ts states. const packageEnvelopeFields = { metadata: LayoutMetadataSchema, ...contentDocumentSharedFields, pages: z.array(PageSizeSchema).optional(), styles: StylesTableSchema.optional(), definitions: DefinitionsTableSchema.optional(), + layers: DefinitionsTableSchema.optional(), // optional-content / layer definitions: PDF `/OCProperties` groups and their configuration, ODF Draw's layer model. Definitions only -- which content belongs to which layer is a membership fact the producing codec carries on its own item model (pdf-codec owns that model since 4.0.0), and no inventory asks for a layer ref on a content-tree node. + attachments: DefinitionsTableSchema.optional(), // package attachments: PDF `/Names /EmbeddedFiles`, `/FileAttachment`, `/EF`, `/AF`, and the docx/ODF package attachments that make the facility cross-format rather than PDF-specific. + destinations: DefinitionsTableSchema.optional(), // named destinations and the navigation tree that resolves against them: PDF `/Dests`, the `/Names` name tree, and `/Outlines`. This is the other end of a `link` construct's internal target (src/construct.ts) -- an internal target names either an `anchor` construct or an entry here. }; export const DocumentPackageSchema = z.discriminatedUnion('kind', [ diff --git a/test/smoke.test.mjs b/test/smoke.test.mjs index 41ea93d..09783ca 100644 --- a/test/smoke.test.mjs +++ b/test/smoke.test.mjs @@ -10,6 +10,8 @@ describe('smoke: ESM/CJS parity', () => { const esm = await import('../dist/index.js'); expect(typeof esm.DocumentPackageSchema.parse).toBe('function'); expect(typeof esm.PackageNodeSchema.safeParse).toBe('function'); + expect(typeof esm.ConstructDescriptorSchema.safeParse).toBe('function'); + expect(typeof esm.SectionConstructGroupSchema.safeParse).toBe('function'); expect(typeof esm.resolveStyleChain).toBe('function'); expect(esm.LayoutDocumentSchema).toBeUndefined(); }); @@ -19,6 +21,8 @@ describe('smoke: ESM/CJS parity', () => { const cjs = require('../dist/index.cjs'); expect(typeof cjs.DocumentPackageSchema.parse).toBe('function'); expect(typeof cjs.PackageNodeSchema.safeParse).toBe('function'); + expect(typeof cjs.ConstructDescriptorSchema.safeParse).toBe('function'); + expect(typeof cjs.SectionConstructGroupSchema.safeParse).toBe('function'); expect(typeof cjs.resolveStyleChain).toBe('function'); expect(cjs.LayoutDocumentSchema).toBeUndefined(); }); @@ -59,6 +63,10 @@ describe('smoke: generated JSON Schema files', () => { expect(variant.properties.pages.type).toBe('array'); expect(variant.properties.styles.additionalProperties.$ref).toBe('#/$defs/StyleEntry'); expect(variant.properties.definitions.additionalProperties.$ref).toBe('#/$defs/DefinitionEntry'); + // The three construct tables reuse the generic definitions entry rather than minting parallel entry shapes. + for (const table of ['layers', 'attachments', 'destinations']) { + expect(variant.properties[table].additionalProperties.$ref).toBe('#/$defs/DefinitionEntry'); + } } // The per-kind root children: sections, slides, sheets, draw pages, and the formula leaf. const byKind = Object.fromEntries(documentPackage.oneOf.map((variant) => [variant.properties.kind.const, variant])); @@ -70,10 +78,11 @@ describe('smoke: generated JSON Schema files', () => { // The tree fragments resolve file-locally: both published files carry the same $defs block (the same object emitted twice in one generator run). expect(Object.keys(documentPackage.$defs)).toContain('SectionGroup'); expect(Object.keys(documentPackage.$defs)).toContain('StyleEntry'); - // The recursion itself: a section group's children point back at the shared HeadingGroup/ListGroup definitions, and those at ContentBlock. + // The recursion itself: a section group's children point back at the shared HeadingGroup/ListGroup/SectionConstructGroup definitions, and those at ContentBlock. expect(documentPackage.$defs.SectionGroup.properties.children.items.oneOf).toEqual([ { $ref: '#/$defs/HeadingGroup' }, { $ref: '#/$defs/ListGroup' }, + { $ref: '#/$defs/SectionConstructGroup' }, { $ref: '#/$defs/ContentBlock' }, ]); // Style entries enforce the ban list by shape: additionalProperties false on entry and both halves, with no frames/sourcePath/styleId field anywhere. @@ -83,6 +92,46 @@ describe('smoke: generated JSON Schema files', () => { expect(documentPackage.$defs.StyleRunProperties.properties.sourcePath).toBeUndefined(); }); + it('document-package.schema.json publishes the construct vocabulary: six descriptor kinds and the two group wrappers that carry them', () => { + const documentPackage = readSchema('document-package.schema.json'); + expect(documentPackage.$defs.ConstructDescriptor.oneOf).toEqual([ + { $ref: '#/$defs/ContentControlDescriptor' }, + { $ref: '#/$defs/FieldDescriptor' }, + { $ref: '#/$defs/AnchorDescriptor' }, + { $ref: '#/$defs/LinkDescriptor' }, + { $ref: '#/$defs/ProvenanceDescriptor' }, + { $ref: '#/$defs/DivisionDescriptor' }, + ]); + // Both group wrappers take a construct descriptor as their node, and each recurses into its own flow's child union -- the section-scoped one admitting heading groups, the shape-scoped one not. + for (const wrapper of ['SectionConstructGroup', 'ShapeConstructGroup']) { + expect(documentPackage.$defs[wrapper].properties.node.$ref).toBe('#/$defs/ConstructDescriptor'); + expect(documentPackage.$defs[wrapper].required).toEqual(['node', 'children']); + expect(documentPackage.$defs[wrapper].additionalProperties).toBe(false); + } + expect(documentPackage.$defs.SectionConstructGroup.properties.children.items.oneOf).toContainEqual({ + $ref: '#/$defs/HeadingGroup', + }); + expect(documentPackage.$defs.ShapeConstructGroup.properties.children.items.oneOf).not.toContainEqual({ + $ref: '#/$defs/HeadingGroup', + }); + // Every descriptor is a closed object, so a producer cannot smuggle format-specific residue onto one -- channel 2 is its own facility, not a descriptor escape hatch. + for (const descriptor of [ + 'ContentControlDescriptor', + 'FieldDescriptor', + 'AnchorDescriptor', + 'LinkDescriptor', + 'ProvenanceDescriptor', + 'DivisionDescriptor', + 'DivisionSource', + ]) { + expect(documentPackage.$defs[descriptor].additionalProperties).toBe(false); + } + expect(documentPackage.$defs.FieldDescriptor.required).toEqual(['kind', 'instruction']); + expect(documentPackage.$defs.AnchorDescriptor.required).toEqual(['kind', 'anchorType', 'name']); + expect(documentPackage.$defs.LinkDescriptor.properties.target.$ref).toBe('#/$defs/LinkTarget'); + expect(documentPackage.$defs.LinkTarget.oneOf).toHaveLength(2); + }); + it("content-document.schema.json's flat arms keep their symbol-table $ref and drop the retired formatVersion field", () => { const contentDocument = readSchema('content-document.schema.json'); expect(contentDocument.oneOf).toHaveLength(5); diff --git a/test/workers/document-schema.test.ts b/test/workers/document-schema.test.ts index b9528be..c595eca 100644 --- a/test/workers/document-schema.test.ts +++ b/test/workers/document-schema.test.ts @@ -57,4 +57,36 @@ describe('document-schema.js under the Cloudflare Workers runtime', () => { const resolved = resolveStyleChain(parsed.styles ?? {}, ['s1']); expect(resolved.paragraph).toEqual({ alignment: 'justify' }); }); + + it('DocumentPackageSchema parses a construct-bearing tree and the construct tables at the root', () => { + const parsed = DocumentPackageSchema.parse({ + kind: 'wordprocessing', + metadata: {}, + definitions: { n1: { kind: 'footnote', blocks: [] } }, + destinations: { ch1: { kind: 'destination', pageIndex: 0 } }, + children: [ + { + node: { + kind: 'section', + pageSize: { widthPt: 612, heightPt: 792 }, + margins: { topPt: 0, rightPt: 0, bottomPt: 0, leftPt: 0 }, + }, + children: [ + { + node: { kind: 'division', name: 'Chapter1' }, + children: [ + { node: { kind: 'anchor', anchorType: 'footnote', name: '1', definition: 'n1' }, children: [] }, + { + node: { kind: 'link', target: { kind: 'internal', anchor: 'ch1' } }, + children: [{ kind: 'paragraph', runs: [{ text: 'Jump to chapter 1.' }] }], + }, + ], + }, + ], + }, + ], + }); + expect(parsed.destinations?.ch1).toEqual({ kind: 'destination', pageIndex: 0 }); + expect(parsed.children[0]?.children).toHaveLength(1); + }); });