Fidelity constructs: implement the semantic-verdict rows (definitions table, footnotes, link titles, task lists, math, HTML residue)

[Mearman]

Implementation half of the construct inventory (verdict table in the latest comment on #63, verified against the reader on origin/main — per-construct greps with file:line citations on that ref, not spec recollection), feeding the harmonised vocabulary in ExaDev/document-schema.js#22. The pole-case finding holds: markdown needs no new generic construct kind — every semantic verdict lands on a mechanism the schema major already defines (definitions table, anchor, link, ContentFormula, list-membership/tree fields). Everything below is gated on ExaDev/document-schema.js#24 landing the construct descriptor kinds and package tables additively.

Already carried and untouched: GFM tables with alignment (ContentTable plus per-cell ContentParagraph.alignment), strikethrough (ContentRun.strike), autolinks (ContentRun.hyperlink, with the <...> spelling re-derived on emit when text equals destination). Derivable, dropped without loss: the reference-link form itself (the destination survives on hyperlink; label spelling is a serialisation choice restorable from the definitions table), entity spelling (emit re-escapes), serialisation choices (ATX/setext, bullet glyph, delimiters, fence char, CRLF, tight/loose), and soft line breaks — with the honest caveat that soft breaks are the single largest conformance-exclusion bucket by count.

Semantic rows — reading work, by target in the harmonised vocabulary:

• Reference link definitions ([label]: dest "title") — package-level definitions table, the same facility the styles table rides; the map is built and returned today but dies at lowering (lowerParsedMarkdown reads only parsed.document.children), and a definitions-only paragraph leaves no block behind — src/block/definitions.ts:66 (extract + consume); src/block/block.ts:782 (map returned); src/lower/lower.ts:258,275 (map ignored); fixtures src/block/block.test.ts:139-158,233
• Footnotes ([^n] marker + definition) — done in feat: read and write GitHub footnotes, with definitions as anchor constructs #69. The mapping this row proposed turned out backwards on both halves: the DEFINITION is the anchor construct (its body rides the construct's own extent, since AnchorDescriptor.definition names a definitions-table key and a flat ContentDocument has no root table to key into), and the MARKER cannot be an anchor at all — a construct's extent is block-scoped, and a reference site sits between two runs inside a paragraph. The reference ships as a marked run (md/footnote-reference-preserved-as-text) and needs a run-level extent mechanism in document-schema.js to close properly. See the comment below for the full reading
• Link/image titles — link node: exactly the annotated link extent a flat run field cannot express, the case the standing reconciliation on build(deps): bump document-schema.js to 2.6.1 #22 reserves the node kind for; dropped today on every path (inline links, images both run-degraded and resolved-block, and the definitions table's own titles, parsed then dead with the map) — md/link-title-dropped; src/lower/inline.ts:72-84; src/lower/lower.ts:98; src/test-support/conformance-exclusions.ts (LINK_TITLE)
• GFM task lists — a real checked/task field on the list-membership representation, replacing today's degraded encoding (a checkbox-glyph text run prepended to the first paragraph plus a task bit encoded into the minted numId; state lost entirely when the item's first block is a table or resolved image) — src/ast/ast.ts:81; src/block/block.ts:685-700; src/lower/lower.ts:165-208 (applyTaskCheckbox, false-return case documented)
• Code fence info string — the language word as an annotation on the code representation (a field, not a node kind); the remainder goes to residue (see the residue checklist); dropped today when non-empty, diagnosed — md/code-block-info-string-dropped; src/lower/lower.ts:117-124
• Math ($$ blocks, \(...\) inline, Parse $$ display-math blocks and \( \) inline math #53) — ContentFormula with the canonical-plus-source pattern: LaTeX as the source spelling beside whatever canonical form the schema settles; today's preserved-as-text (MathBlock styleId, inline marker fontFamily) is the interim, not the destination — src/lower/lower.ts:143-152; src/lower/inline.ts:64-67; the MarkdownMathInlineNode comment in src/ast/ast.ts
• Frontmatter, the unmapped half — mapped keys already reach ContentDocument.metadata; the unmapped remainder (md/front-matter-key-unmapped plus drop) and the non-flat shapes the flat reader silently skips (nested mappings, block lists, multi-line scalars) become a raw front-matter block in the package-level residue channel — src/lower/front-matter.ts:66-87 (switch + default-drop); emit inverse src/emit/front-matter.ts
• Multi-block list item boundary — item identity on the list-membership model (the schema's list work, not a construct-vocabulary node); today ContentListMembership carries numId/level only, so "one item, several blocks" and "several items" are indistinguishable, and tables/images in an item lose membership entirely — md/list-item-multi-block-flattened, md/list-item-block-unlisted; lowerListItem in src/lower/lower.ts
• Blockquote container / nesting depth — a container representation in the schema tree, sibling of the item-boundary gap; carried degraded today as indentLeftPt plus a Quote styleId, with depth beyond 1 approximate and the container boundary lost — md/blockquote-nested-depth; lowerBlockquote in src/lower/lower.ts; conformance-exclusions.ts (BLOCKQUOTE_STRUCTURE, ADJACENT_SAME_DEPTH)

Residue rows — quarantined source channel (per the #22 two-channel model):

• Raw HTML blocks (all seven kinds) — per-node source: { format, ... }, restorable on re-emission; the canonical residue-channel use case from build(deps): bump document-schema.js to 2.6.1 #22, and it also fixes a live ambiguity: literal text that happens to look like a tag is indistinguishable from preserved HTML today (RAW_TEXT_TAG_AMBIGUITY exclusion). Current behaviour is preserve-as-literal-text (default), HTMLPreformatted styleId, or drop under rawHtml: 'drop' — never parsed — src/html/html.ts (recogniser, all 7 block kinds plus the inline grammar); src/lower/lower.ts:130-141; codes md/raw-html-preserved-as-text, md/raw-html-dropped; fixtures src/diagnostics/diagnostics.test.ts:110-121, src/lower/lower.test.ts:281-290
• Raw HTML inline — per-run source channel, same treatment and same diagnostics — src/lower/inline.ts:57-62
• Code fence info-string remainder — pandoc-style {.class #id} attributes quarantined beside the semantic language word (see the info-string row above)

Decisions needed — uncertainty to resolve before the split rows freeze:

• Info-string split: the language-word/residue-remainder divide is a design proposal, not an observed distribution — real-world prevalence of pandoc-style attribute syntax versus bare language words is unsurveyed. Measure before freezing.
• Frontmatter: the flat key: value reader matches the de facto convention, but how often real documents use nested shapes it silently skips is unmeasured; the residue half needs the package channel from Add the fidelity construct descriptor kinds additively (contentControl / field / anchor / link / provenance; package-level tables for layers/attachments/destinations) document-schema.js#24.
• Pandoc inline footnotes (^[...]) have no GFM analogue — the inventory treats the marker-plus-tail-definition pair as the scope and inline notes as out of scope; confirm or widen deliberately rather than by omission.

[Mearman]

Checked the gate (document-schema.js 4.1.0 is published, npm and #24's release comment both confirm) and set up a worktree off origin/main, which already carries the ^4.1.0 bump (ffc708b). Before touching src/lower/src/emit I read the actual shipped 4.1.0 surface — dist/*.d.ts plus the README fetched at the v4.1.0 tag — to confirm where the new construct vocabulary and definitions table actually live, and it's not reachable from where this package operates.

Every 4.1.0 addition — the six construct descriptors (contentControl/field/anchor/link/provenance/division) and the definitions/layers/attachments/destinations tables — is wired exclusively into DocumentPackage/PackageNode (package.d.ts, package-node-DLUobcDQ.d.ts: SectionConstructGroupNode/ShapeConstructGroupNode are {node: ConstructDescriptor, children}, and definitions etc. are root fields of DocumentPackageSchema). ContentDocument and its ContentBlock/ContentRun types (content-CiZf6Fqn.d.ts) gained nothing — grepping that file for Construct, anchor, definitions, residue all return zero hits. The 4.1.0 README says this is deliberate, not an oversight: "The codecs do not change: they keep producing flat ContentDocuments... decomposition runs once at the package boundary in documents.js" and, in its own references list, "readMarkdown/writeMarkdown read and write this package's ContentDocument directly." documents.js's decompose is the only thing that ever builds a DocumentPackage, and it's a generic structural promotion (grouping headings/lists/sections) with no format-specific knowledge — it can't manufacture an anchor-plus-definitions-table pair for a footnote that has nowhere to live on the flat ContentDocument it's promoting from.

Concretely, this blocks all three rows, not just the two the task anticipated might block:

• Footnotes — no field anywhere on ContentBlock/ContentRun to place an anchor marker at the reference site or reach a definitions table for the tail definition. Adding one would mean inventing a new ContentDocument-level channel on top of a schema that 4.1.0 explicitly left unchanged.
• Raw HTML residue — confirmed the residue channel (build(deps): bump document-schema.js to 2.6.1 #22's channel 2, source: {format, xml}) did not land in 4.1.0; the README says outright "Residue is not here... remains open on build(deps): bump document-schema.js to 2.6.1 #22."
• Reference-link definitions — same definitions-table-is-DocumentPackage-only gap.

No code changes landed in src/; the throwaway branch/worktree were removed. This isn't a markdown-codec-side gap — it needs a document-schema.js-side decision (a new issue, since #22/#24 are both closed) on how a codec's flat ContentDocument surface is meant to reach the construct/definitions vocabulary it now has no path to: either ContentDocument gains its own construct fields, or documents.js's decompose gains per-format hooks a codec can register. The handover plan's Wave W4 phrase "execute high-value constructs... as codec lanes" assumed the former or something like it; the shipped 4.1.0 README's "the codecs do not change" invariant says the latter, or neither yet. That reconciliation is outside what this issue (or this package) can resolve unilaterally.

[Mearman]

Footnotes row is done in #69 (reader + writer + round-trip tests), now that document-schema.js 4.2.0 has landed construct boundary markers on ContentBlock. That closes the gap the earlier comment on this issue correctly stopped at — the construct vocabulary being reachable only from DocumentPackage — but only for the block-scoped half. The rest of this is what actually shipped and what did not, since the row's original wording ("marker is anchor, definition is a definitions-table entry") turned out to be exactly backwards on both halves.

The definition is the anchor, not the marker. Reading AnchorDescriptorSchema on the published 4.2.0 source: definition is z.string().optional() and its comment says it is "the definitions-table key holding this marker's body" — a key, not the body. DefinitionsTableSchema hangs off DocumentPackageSchema's root; ContentDocumentSchema has no root fields to carry one, so there is no table to key into and definition stays absent. That is not a size problem with the field: a footnote body is block content (several paragraphs, a code block, a list), which a string could not have carried whichever table it pointed at. So the body rides the construct's own extent, which is what AnchorDescriptorSchema's own note describes — "a ranged anchor … wraps the blocks it spans", and a bodyless [^1]: is the point anchor in the same sentence.

The reference site cannot be an anchor at all, and that is still open. src/construct.ts binds every construct kind to block scope ("It does not wrap a sub-sequence of one paragraph's runs, because a run-level extent is not expressible without changing ContentParagraph's own shape") and names this exact case as deferred: "the inline field/bookmark/tracked-change cases wait on a run-level extent mechanism". A reference sits between two runs inside a paragraph, and ContentRun has no field for it either — no styleId, and hyperlink is a resolved external URI by its own definition.

It ships as a marked run (FOOTNOTE_REFERENCE_FONT_MARKER on the run carrying the [^1] spelling) with md/footnote-reference-preserved-as-text reporting it. The marker is not cosmetic: escapeMarkdownText escapes [, ^, and ], so a genuine reference and a deliberately-escaped literal \[^1\] are the same run text by the time the writer sees them — the identical trap that got the pattern-based approach to inline math reverted. This needs a run-level extent mechanism in document-schema.js to close properly; it is worth a schema-side issue rather than living on as a font-family sentinel indefinitely.

Two further restrictions fell out of the marker contract (ContentBlockSchema's "balance is necessary but not sufficient" note), both deliberate and both documented in the README:

• Definitions are recognised only at the document's top level. Inside a list item the body would have to carry the item's ContentListMembership on every block, and a body table or image cannot — closing the item's list scope from inside the pair, which is precisely what a producer is told never to emit.
• A heading inside a definition body is flattened to literal ATX text (md/footnote-body-heading-flattened): it both closes a heading scope opened outside the extent and leaves one standing at the closing marker.

Still open on this issue, untouched by #69:

• Raw HTML blocks and inline raw HTML — the residue channel (Two-channel fidelity: harmonised semantic construct vocabulary plus a quarantined residue channel document-schema.js#22's channel 2) has not shipped. src/construct.ts says outright that it deliberately does not define one: it is a per-node and package-level source facility spanning the whole content model, not a construct descriptor field.
• Frontmatter's unmapped half — same residue-channel dependency.
• Reference link definitions — same definitions-table-is-DocumentPackage-only gap that keeps AnchorDescriptor.definition absent above. If the schema grows a way for a flat ContentDocument to carry a definitions table, this row and that field both unblock together.
• Link/image titles, task lists, info strings, math, list-item boundary, blockquote container — all unchanged.