Fidelity construct inventory: annotations, AcroForm fields, embedded files, optional content groups

[Mearman]

Research audit feeding the schema's fidelity vocabulary (ExaDev/document-schema.js#22, part of the DocumentPackage promotion ExaDev/document-schema.js#20). PDF is the opposite pole from markdown: its entire semantic layer is inferred by reconstruction, and its item layer stays codec-private per #65 -- so this inventory covers the real objects the item stream does not represent:

• Annotations -- link annotations (onto the link vocabulary), text/comment annotations (anchor vocabulary with author/date), widget annotations.
• AcroForm fields and values -- onto the contentControl vocabulary (form fields).
• Embedded files (file annotations, names tree) -- residue or a package-level attachment list, to be decided on evidence.
• Optional content groups (visibility layers) -- verdict needed; candidates are a visibility property on nodes or residue.
• Document metadata beyond what is carried (XMP, output intents) -- metadata or residue.

Deliverable: the verdict table with real-file examples per semantic candidate. Reading these requires extending readPdf beyond the item stream -- the inventory should scope that read work honestly rather than presuming it exists.

[Mearman]

Verified against origin/main (7f1648b..4f0f0e2 at time of audit; greps re-run after fetch), not spec recollection. Method: enumerated every dictionary key the reader actually touches (git grep "dictGet(" across src/read.ts, src/document.ts, src/interpret.ts), then grepped each candidate construct's format marker in src/ to establish carried / partially carried / dropped. The complete key inventory is small enough to state in one line: catalog /Type /Pages; page /MediaBox /Rotate /Resources /Contents /Annots; annotation /Subtype /Rect /T /Contents /A(/S /URI); trailer /Root /Info /ID /Encrypt; resource /XObject. Everything below that line is untouched by the reader.

One caveat before the table: this repo's fixture corpus is entirely generated (src/test-support/pdf.ts builds PDFs by literal construction; the worker/smoke tests do the same). No real-world file in the repo exercises AcroForm, OCGs, XMP, output intents, outlines, or tagged structure — so where the issue asks for real-file examples per semantic candidate, the only in-repo evidence is the round-trip fixtures cited below, and acquiring a real-world corpus (a PDF/A sample would cover four rows at once) is part of the read work, not prior art.

Construct	Carried today	Verdict	Harmonisation target	Evidence
Link annotation, /A /S /URI form	Yes — LayoutLink items on the page	Semantic (already)	link; run-level reconciliation to ContentRun.hyperlink where the rect matches recovered runs, else a block-scoped link extent — exactly the case ExaDev/document-schema.js#22's comment reserves the node kind for	Read: src/read.ts:338-381. Write: src/write.ts:228-236. Fixture: src/roundtrip.test.ts:79-84. Note the content layer drops these today: documents.js src/layout/reconstruct.ts:521 documents link items being ignored by every reconstruction direction (filters at reconstruct.ts:255, :361 take only text/image)
Link annotation, other targets: /Dest (direct and named), GoTo/GoToR actions	No — readLinkUri returns undefined for anything without /A /S /URI, so internal links vanish	Semantic	link with an anchor-referenced internal target	src/read.ts:374-381 (the one action shape read); Launch/Named/embedded-GoTo actions have no cross-format analogue — residue
Text annotation, genuine third-party sticky note (/Subtype /Text without our marker)	No — deliberately skipped	Semantic	anchor for the marker (author/date on it, provenance-style) + the note body as a definitions-table entry — the same marker/definition split ExaDev/document-schema.js#22's comment settles for footnotes	src/read.ts:383-405 skips any /T ≠ NOTES_ANNOTATION_AUTHOR (src/write.ts:245); the discrimination is proven by src/read.test.ts:206-211 + fixture src/test-support/pdf.ts:255-260
Markup annotations: FreeText, Highlight/Underline/StrikeOut/Squiggly (with /QuadPoints), Popup pairing, /CAs//IT states, author /T + date /M	No	Semantic	comment analogue of docx/odf annotations: anchor (author/date, provenance fields) + definitions entry for the comment body; /QuadPoints ranges need matching against reconstructed runs — real work, scoped below	No code path: src/read.ts:346-350 skips every non-Link annot and readPageNotes skips unmarked Text; zero hits for FreeText/QuadPoints/Popup in src/
Stamp / Ink / Polygon / PolyLine / Caret / Redact annotations	No	Residue	opaque source channel (drawing-as-commentary with no cross-format comment shape)	same skip path as above; flagging rather than guessing — I have not verified real-world frequency of these
Widget annotations + AcroForm field tree (/AcroForm → /Fields recursion, /FT per type, /V values, /Ff flags, choice /Opt, /DR, /DA, merged-field/widget /Parent split)	Zero handling — confirmed (no AcroForm, Widget, FT, V hits anywhere in src/)	Semantic	contentControl (the assignment ExaDev/document-schema.js#22 already makes for PDF form widgets); current /V rides the control's value; field type/flags map onto the control's typing	Catalog walk reads only /Pages: src/document.ts:151,167-175; src/read.ts:340-405 is the whole annotation reader
Signature widgets (/FT /Sig, /ByteRange, /SigFlags, /Perms)	No	Residue	certification binds to bytes a semantic pivot never reproduces; signer name/date could surface as provenance metadata if wanted	zero Sig/ByteRange/Perms hits
Embedded files: /Names /EmbeddedFiles tree, /FileAttachment annotations, /EmbeddedFile streams (/EF, /Desc, /F//UF), ISO 32000-2 /AF associated files	Zero handling — confirmed (no EmbeddedFile/Names//AF hits)	Semantic	package-level attachment list, riding the same #20-major package-table mechanism as the definitions and styles tables — not a node kind. Honest alternative: residue if ExaDev/document-schema.js#22 declines a package attachments facility; my read of the evidence is that docx/odf attachments make it cross-format, so it clears the "no format-specific node" bar as a table, not a node	src/document.ts:167-175 (catalog → pages only); no names-tree walker exists anywhere in src/
Optional content groups: /OCProperties (/OCGs, /D config with /BaseState /ON /OFF /Order /RBGroups), /OC in BDC spans and XObject/image dicts	Zero handling — confirmed; worse than dropped metadata: BDC/BMC are skipped outright (src/interpret.ts:753), so content an author hid in an OFF layer is extracted as if visible	Semantic	layer definitions in a package-level layers table + a layer reference on the extracted items/frames within the /OC BDC…EMC span. Cross-format analogue exists (ODF Draw's layer model), which is what makes this semantic rather than residue. If #22 declines layers, falls back to residue — contested, flagged below	src/interpret.ts:753 (marked-content skip); zero OCProperties hits; catalog walk src/document.ts:167
XMP metadata: /Metadata stream on catalog (and per-object)	No — only the /Info subset (Title Author Subject Keywords Creator Producer CreationDate ModDate → LayoutMetadata) is read; /Trapped, custom /Info keys, and all XMP dropped	Split	Dublin Core / standard XMP fields that mirror LayoutMetadata → semantic metadata (in real PDF/A files these live only in XMP — format knowledge, not repo-verified); the raw packet → residue at package level, opaque XML per channel 2. Initial cut could honestly be residue-only	src/read.ts:436-460 (/Info reader); fixture src/read.test.ts:194-203; zero Metadata-stream hits beyond the LayoutMetadata type import (src/read.ts:4)
Output intents: /OutputIntents (/DestOutputProfile ICC stream, /OutputConditionIdentifier, /GTS_PDFA1…)	Zero handling — confirmed	Residue	print-condition blob + ICC bytes as opaque package-level residue; PDF/A identification depends on it, content does not	zero OutputIntents hits; catalog walk src/document.ts:167-175
Hidden presenter-notes channel (/Subtype /Text with /T = "documents.js:notes", /F 2)	Yes — LayoutPage.notes → slide notes through reconstructPresentation	Semantic (already; stays)	unchanged — it is a writer/reader convention, not a real PDF feature, per the schema's own comment	Read src/read.ts:383-405, write src/write.ts:242-256, schema document-schema.js src/layout.ts:140-141, consumption documents.js src/layout/reconstruct.ts:379-380

Derivable (recomputable, dropped without loss once the semantic parent exists): widget/annotation appearance streams (/AP /N etc.) and /NeedAppearances — renderings of the field value or comment the control/anchor already carries, i.e. cached results; link annotation /Border (our writer already emits zero-width as an invisible region, src/write.ts:233); Popup annotation rectangles (derivable from parent + offset). Counts below treat these as derivable, not residue.

Missed by the issue's candidate list

Walked the catalog and page-dictionary vocabulary against the key inventory above; these have zero handling (git grep zero hits for each, all confirmed in the same pass):

• /Outlines — the document's bookmark tree. Semantic: navigation table at package level + the anchor ids /Dest-bearing entries point at. The one construct here with an obvious consumer in every interface package.
• /Dests + /Names name tree — named destinations. Semantic, and a hard prerequisite: internal-link resolution and outline targets both need the name-tree walk. One walker serves embedded files, destinations, and (if ever wanted) JavaScript names.
• /StructTreeRoot + /OCG-adjacent tagged structure (BDC /StructTreeRefs, /ActualText, /Alt, per-element /Lang) — tagged PDF's logical structure: reading order, heading levels, lists, tables, alt text. Semantic in the strongest sense — it is the one place PDF carries semantics natively rather than forcing reconstruction to infer them, and it maps directly onto the Promote DocumentPackage to the single hierarchical tree: structure, layout, and content in one artefact document-schema.js#20 tree. Also the largest new-read item here by a wide margin (structure-tree walk + role-map + marked-content correlation); the honest scoping is phased: /ActualText + /Alt + per-element /Lang on already-extracted items first, full structure re-parenting later.
• /Lang (catalog) — document language. Semantic: one metadata field, cross-format analogue in every other format's metadata.
• /PageLabels — page-numbering scheme (roman front matter etc.). Semantic, small: a numbering definition in a package table. Honest flag: I have not checked how the other three inventories treat page numbering, so this may be better as a shared decision there.
• /CropBox (and /BleedBox /TrimBox /ArtBox) — document.ts walks /CropBox through page-tree inheritance (src/document.ts:26) but nothing ever consumes it; only /MediaBox is read (src/read.ts:83-93), so content outside the crop box is extracted as if visible — the same active-misfidelity shape as OCGs, in geometry. Verdict semantic (page frame fields within the media box), while noting it straddles this issue's stated scope boundary ("objects the item stream does not represent") — geometry is the item stream's business, so it may belong to the item-layer work instead.
• Viewer/session behaviour — /ViewerPreferences, /PageMode, /PageLayout, /OpenAction, /AA, page /Trans /Dur /B (beads). Residue: behaviour, not content.
• Private/application data — /PieceInfo (Illustrator/InDesign live data), /Legal, /Collection (portfolios), trailer /ID (read only for key derivation, src/document.ts:29-34, never surfaced). Residue; /Collection marked uncertain — rare enough I have no real-file evidence of its shape.

One scope note for completeness: clipping paths, shadings, tiling patterns, and ExtGState (transparency, blend modes) are also skipped by the item stream (src/interpret.ts:753), but those are item-layer gaps inside this issue's own exclusion ("constructs the item stream does not represent" they are not), except insofar as OCG membership rides BDC — which is why the OCG row above has one foot in each layer.

New generic kinds proposed

None. No construct here needs a sixth node kind: forms are contentControl (already assigned), links and internal targets are link + anchor, sticky notes and markup comments reuse the footnote pattern (anchor marker + definitions-table body), and the remaining semantic weight — layers, attachments, navigation, page labels — is package-level tables riding the same ExaDev/document-schema.js#20/#21 mechanism as definitions and styles, not tree nodes. What the evidence does demand beyond the existing five kinds is three new package tables (layers, attachments, navigation/destinations) and one name-tree walker in document.ts that all of embedded-files, destinations, and outlines share.

Read-work scoping (what extending readPdf beyond the item stream actually requires)

In rough order of cost: XMP /Metadata and output intents are trivial dict/stream fetches (stream decode already exists, src/read.ts:10) — XMP's DC subset additionally wants a minimal namespace-aware parser, which this dependency-minimal repo does not have, hence the residue-first suggestion. Embedded files, /Dests, and outlines need the shared names-tree walker plus stream extraction. Sticky notes and markup need annotation-subtype dispatch plus /M date parsing (parsePdfDate exists, src/read.ts:423) and, for markup ranges, /QuadPoints-to-run matching. AcroForm needs catalog access, field-tree recursion with the merged-widget /Parent split, per-/FT value decoding, and /DR font resolution — the biggest item after tagged structure. OCGs need /OCProperties parsing plus threading /OC membership through interpret.ts's state machine so items inside a BDC span carry their layer — moderate, and it changes currently-extracted output, so it wants fixtures with known ON/OFF state before landing.

Contested verdicts, for the record: embedded files (semantic-as-table vs residue) and OCGs (semantic layer table vs visibility-residue) both hinge on decisions ExaDev/document-schema.js#22 owns rather than on further evidence this repo could supply; /PageLabels may belong to a cross-inventory decision; and /CropBox likely belongs to the item-layer issue rather than this one. Everything else in the table follows from the code as it stands.