Skip to content

Fidelity constructs: implement the semantic-verdict rows (annotations, AcroForm, embedded files, layers, outlines, destinations, tagged structure) #69

Description

@Mearman

Implementation half of the construct inventory (verdict table in the latest comment on #66, verified against origin/main — method: every dictionary key the reader touches was enumerated via git grep "dictGet(" across src/read.ts, src/document.ts, src/interpret.ts, then each candidate construct's format marker grepped to establish carried / partially carried / dropped), feeding the harmonised vocabulary in ExaDev/document-schema.js#22. The key inventory is small enough to restate: catalog /Type /Pages; page /MediaBox /Rotate /Resources /Contents /Annots; annotation /Subtype /Rect /T /Contents /A(/S /URI); trailer /Root /Info /ID /Encrypt; resource /XObject. Everything below that line is untouched, and everything here is gated on ExaDev/document-schema.js#24 landing the construct descriptor kinds and the package-level tables (layers, attachments, navigation/destinations) additively.

The hidden presenter-notes channel (/Subtype /Text with /T = "documents.js:notes", /F 2) is semantic already and stays unchanged — a writer/reader convention, not a real PDF feature — src/read.ts:383-405; src/write.ts:242-256; consumed via reconstructPresentation in documents.js.

Semantic rows — reading work, by target in the harmonised vocabulary:

  • Internal links: /Dest (direct and named) and GoTo/GoToR actions — link with an anchor-referenced internal target; readLinkUri returns undefined for anything without /A /S /URI today, so internal links vanish — src/read.ts:374-381 (the one action shape read); needs the internal-target vocabulary and the destinations table below
  • External /A /S /URI links — already LayoutLink items, so the work is reconciliation at the content boundary: ContentRun.hyperlink where the rect matches recovered runs, else a block-scoped link extent (exactly the case the build(deps): bump document-schema.js to 2.6.0 #22 reconciliation reserves the node kind for). Note the content layer drops these today — documents.js src/layout/reconstruct.ts:521 documents link items being ignored by every reconstruction direction — src/read.ts:338-381; src/write.ts:228-236; fixture src/roundtrip.test.ts:79-84
  • Genuine third-party sticky notes (/Subtype /Text without our notes marker) — anchor for the marker (author/date on it, provenance-style) plus a definitions-table entry for the body, the same marker/definition split the schema settles for footnotes; deliberately skipped today — src/read.ts:383-405 skips any /TNOTES_ANNOTATION_AUTHOR (src/write.ts:245); discrimination proven by src/read.test.ts:206-211 + fixture src/test-support/pdf.ts:255-260
  • Markup annotations (FreeText, Highlight/Underline/StrikeOut/Squiggly with /QuadPoints, Popup pairing, /CAs//IT states, author /T + date /M) — the comment analogue of docx/odf annotations: anchor plus a definitions entry, with /QuadPoints ranges matched against reconstructed runs (real work, scoped here) — src/read.ts:346-350 skips every non-Link annot; zero hits for FreeText/QuadPoints/Popup in src/
  • AcroForm: /AcroForm field tree (/Fields recursion, /FT per type, /V values, /Ff flags, choice /Opt, /DR//DA, merged-field/widget /Parent split) — contentControl (the assignment build(deps): bump document-schema.js to 2.6.0 #22 already makes for PDF form widgets): /V rides the control's value, field type/flags map onto the control's typing; zero handling confirmed (no AcroForm/Widget/FT/V hits anywhere in src/) — catalog walk reads only /Pages at src/document.ts:151,167-175; src/read.ts:340-405 is the whole annotation reader
  • Embedded files: /Names /EmbeddedFiles tree, /FileAttachment annotations, /EmbeddedFile streams (/EF, /Desc, /F//UF), ISO 32000-2 /AF — a package-level attachments table riding the same package-table mechanism as definitions and styles, not a node kind; zero handling confirmed — src/document.ts:167-175; no names-tree walker exists anywhere in src/
  • Optional content groups: /OCProperties (/OCGs, /D config with /BaseState /ON /OFF /Order /RBGroups), /OC in BDC spans and XObject/image dicts — a package-level layers table plus a layer reference on the extracted items/frames within the /OC BDC…EMC span; a cross-format analogue exists (ODF Draw's layer model), which is what makes this semantic rather than residue; today BDC/BMC are skipped outright so content an author hid in an OFF layer extracts as if visible — src/interpret.ts:753; zero OCProperties hits
  • /Outlines — the document's bookmark tree: a navigation table at package level plus the anchor ids /Dest-bearing entries point at; the one construct here with an obvious consumer in every interface package; zero handling — catalog walk src/document.ts:167
  • /Dests + /Names name tree — named destinations as a package-level table; a hard prerequisite: internal-link resolution and outline targets both need the name-tree walk, and one walker serves embedded files, destinations, and (if ever wanted) JavaScript names — no walker exists
  • Tagged structure (/StructTreeRoot + role map, BDC /StructTreeRefs, /ActualText, /Alt, per-element /Lang) — tagged PDF's logical structure onto the schema tree: reading order, heading levels, lists, tables, alt text — semantic in the strongest sense (the one place PDF carries semantics natively rather than forcing reconstruction to infer them), and the largest new-read item here by a wide margin; phased honestly: /ActualText + /Alt + per-element /Lang on already-extracted items first, full structure re-parenting later — zero handling
  • /Lang (catalog) — document language: one metadata field, with a cross-format analogue in every other format's metadata; zero handling
  • XMP /Metadata (catalog stream, and per-object) — split: the Dublin Core / standard fields mirroring LayoutMetadata become semantic metadata (in real PDF/A files these live only in XMP), the raw packet goes to residue; today only the /Info subset is read (Title Author Subject Keywords Creator Producer CreationDate ModDate), with /Trapped and custom keys dropped — src/read.ts:436-460; fixture src/read.test.ts:194-203; XMP's DC subset additionally wants a minimal namespace-aware parser this dependency-minimal repo does not have, hence residue-first

Residue rows — quarantined source channel (per-node or package-level, per the #22 two-channel model):

  • Stamp / Ink / Polygon / PolyLine / Caret / Redact annotations — opaque source channel: drawing-as-commentary with no cross-format comment shape — same skip path src/read.ts:346-350; real-world frequency unverified
  • Signature widgets (/FT /Sig, /ByteRange, /SigFlags, /Perms) — residue: certification binds to bytes a semantic pivot never reproduces; signer name/date could surface as provenance metadata if wanted — zero Sig/ByteRange/Perms hits
  • Output intents (/OutputIntents with DestOutputProfile ICC stream, /OutputConditionIdentifier, /GTS_PDFA1) — print-condition blob plus ICC bytes as opaque package-level residue: PDF/A identification depends on it, content does not — zero hits
  • XMP raw packet — package-level residue, opaque XML per the schema issue's channel 2 (semantic half above)
  • Viewer/session behaviour (/ViewerPreferences, /PageMode, /PageLayout, /OpenAction, /AA, page /Trans /Dur /B beads) — residue: behaviour, not content
  • Private/application data (/PieceInfo Illustrator/InDesign live data, /Legal, /Collection portfolios (shape uncertain — no real-file evidence), trailer /ID read only for key derivation at src/document.ts:29-34 and never surfaced) — residue
  • Launch / Named / embedded-GoTo actions (the link targets with no cross-format analogue) — residue alongside the internal-link row above — src/read.ts:374-381

Derivable, dropped without loss once the semantic parent exists: widget/annotation appearance streams (/AP /N, /NeedAppearances — cached renderings of the value the control/anchor already carries), link annotation /Border (the writer already emits zero-width as an invisible region, src/write.ts:233), and Popup annotation rectangles (derivable from parent plus offset).

Decisions needed — contested rows, held rather than implemented (both hinge on decisions ExaDev/document-schema.js#22/#24 own rather than on further evidence this repo could supply):

  • Embedded files: semantic-as-package-table vs residue — the inventory's read is that docx/odf attachments make a package attachments facility cross-format, clearing the no-format-specific bar as a table rather than a node; the honest alternative is residue if the schema declines.
  • Optional content groups: semantic layer table vs visibility-residue — same hinge; this one changes currently-extracted output, so it wants fixtures with known ON/OFF state before landing.
  • /PageLabels (page-numbering scheme, roman front matter and kin) — semantic, small (a numbering definition in a package table), but may be better as a shared cross-inventory page-numbering decision; not checked against the other three inventories.
  • /CropBox (and /BleedBox /TrimBox /ArtBox) — page frame fields within the media box; today only /MediaBox is read (src/read.ts:83-93) so content outside the crop box extracts as if visible (the same active-misfidelity shape as OCGs, in geometry), but it straddles this issue's scope boundary and likely belongs to the item-layer work instead.

A corpus gate stands before shapes freeze: this repo's fixture corpus is entirely generated (src/test-support/pdf.ts builds PDFs by literal construction), so no real-world file exercises AcroForm, OCGs, XMP, output intents, outlines, or tagged structure — acquiring a real-world corpus (a PDF/A sample would cover four rows at once) is part of the read work above, not prior art.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions