diff --git a/README.md b/README.md index a173f54..25a8bc8 100644 --- a/README.md +++ b/README.md @@ -51,7 +51,7 @@ graph TD ## Status -The scanner, block parser, and inline parser are complete hand-written implementations of CommonMark 0.31.2's two-phase algorithm plus GFM's table/strikethrough/autolink/task-list-item extensions. `readMarkdown`/`writeMarkdown`/`markdownCodec` are wired and real. Conformance suites measure the full public surface (`readMarkdown` → `writeMarkdown` → reparse → render to HTML) against the vendored CommonMark/GFM corpora — see [Fidelity](#fidelity) for why the rate is below 100% (dominated by what `ContentDocument` can represent, not parsing gaps). +The scanner, block parser, and inline parser are complete hand-written implementations of CommonMark 0.31.2's two-phase algorithm plus GFM's table/strikethrough/autolink/task-list-item extensions and GitHub's footnotes (see [Footnotes](#footnotes)). `readMarkdown`/`writeMarkdown`/`markdownCodec` are wired and real. Conformance suites measure the full public surface (`readMarkdown` → `writeMarkdown` → reparse → render to HTML) against the vendored CommonMark/GFM corpora — see [Fidelity](#fidelity) for why the rate is below 100% (dominated by what `ContentDocument` can represent, not parsing gaps). ## Getting started @@ -80,6 +80,7 @@ import { readMarkdown, writeMarkdown } from 'markdown-codec'; const { document, diagnostics } = readMarkdown('# Title\n\nSome **bold** text with a [link](https://example.com).', { frontMatter: true, // parse a leading YAML front matter block into ContentDocument.metadata + footnotes: true, // recognise [^label] markers and [^label]: definitions (default; see Footnotes) images: (destination) => undefined, // a synchronous MarkdownImageResolver port for non-data: URI images }); @@ -112,8 +113,8 @@ Modelled on `pdf-codec`'s own layering, aimed at CommonMark+GFM instead of PDF: - **`src/ast/`** — markdown AST node types (document/block/inline union), Zod-first. - **`src/options/`** / **`src/defaults/`** — read/write options (GFM toggles, sink, `AbortSignal`, write-side style) and defaults. - **`src/scan/`** — CommonMark line/character scanner, plus `entity-table.ts` (generated from `assets/html-entities/entities.json`). -- **`src/block/`** — CommonMark block-structure algorithm (open-block stack, continuation matching): paragraphs, headings, code blocks, block quotes, lists (incl. GFM task-list-item), thematic breaks, link references, GFM tables. -- **`src/inline/`** — emphasis, code spans, links, autolinks, raw HTML, GFM strikethrough, line breaks. +- **`src/block/`** — CommonMark block-structure algorithm (open-block stack, continuation matching): paragraphs, headings, code blocks, block quotes, lists (incl. GFM task-list-item), thematic breaks, link references, footnote definitions, GFM tables. +- **`src/inline/`** — emphasis, code spans, links, autolinks, raw HTML, GFM strikethrough, footnote references, line breaks. `link.ts` and `footnote.ts` hold the label grammars the block phase shares. - **`src/html/`** — raw HTML recognition (bounded rules, not a general parser) plus `render.ts` (conformance oracle; internal only). - **`src/image/`** — PNG/JPEG dimension reader and base64 codec, shared by `src/lower/` and `src/emit/`. - **`src/shared/`** — string-shape conventions `src/lower`/`src/emit` agree on (`style-constants.ts`, `list-id.ts`'s opaque `numId`). Re-exported so `documents.js`'s `MarkdownEditor` reuses the identical grammar. @@ -173,6 +174,23 @@ Every construct `src/lower`/`src/emit` cannot represent losslessly is a document - **`md/paragraph-indent-dropped`** — `indentLeftPt` without a recognised styleId; indent dropped, paragraph renders. - **`md/list-numid-fallback`** — a foreign or absent `numId` (depth-only `ContentListMembership`) falls back to a plain bullet list. - **`md/table-cell-formatting-dropped`** / **`md/table-cell-multi-paragraph-joined`** — GFM cells have no rich-formatting or multi-paragraph representation. +- **`md/duplicate-footnote-definition`** — two definitions share a label; every reference resolves to the first, both are kept as written. +- **`md/footnote-reference-preserved-as-text`** — a reference site is a marked run, not an `anchor` construct; see [Footnotes](#footnotes). +- **`md/footnote-body-heading-flattened`** — a heading inside a definition body is carried as literal ATX text, since a construct extent may not open or close a heading scope. +- **`md/construct-unrepresented`** — a construct kind markdown has no syntax for renders transparently: its extent still appears, the construct itself does not. + +## Footnotes + +GitHub's footnote extension (`[^label]` markers, `[^label]: body` definitions) is on by default, alongside the four GFM toggles — switch it off with `footnotes: false`. Neither CommonMark nor the GFM spec document defines footnotes, so both spellings are ordinary text with it off. + +The two halves of a footnote map onto **two different mechanisms**, and that split is structural rather than a choice: + +- **A definition becomes an `anchor` construct.** `readMarkdown` emits document-schema.js 4.2.0's construct boundary markers — a `constructStart` carrying `{ kind: 'anchor', anchorType: 'footnote', name }`, the definition's own lowered body blocks, and a `constructEnd`. The body rides the construct's extent rather than `AnchorDescriptor.definition`, which names a key in a package-level definitions table a flat `ContentDocument` has no root to carry; a body is genuinely block content (several paragraphs, a code block, a list) that a string field could not have held either way. A bodyless `[^1]:` lowers to the point anchor the same descriptor describes: a pair with nothing between it. +- **A reference site stays a marked run.** A construct's extent is block-scoped by document-schema.js's own definition, and a reference sits between two runs inside a paragraph, so no block-level boundary marker can bracket it without splitting the paragraph in two. The schema names this gap itself and parks the inline-anchor case on a run-level extent mechanism it has not shipped. Until it does, the reference is a `ContentRun` keeping its own `[^label]` spelling and carrying `FOOTNOTE_REFERENCE_FONT_MARKER`, reported through `md/footnote-reference-preserved-as-text`. + +Definitions are recognised only at the document's own top level. Inside a block quote or a list item, the pair's extent would sit inside a scope the enclosing container had already opened, which the marker contract forbids a producer from emitting — so the text stays an ordinary paragraph there. A heading inside a definition body is flattened to literal ATX text for the same reason. + +`writeMarkdown` is the inverse and validates first: a section's markers must pair as balanced brackets (checked through document-schema.js's own `findConstructMarkerImbalance`, the shared definition every codec and `decompose` agree on) or it throws `MarkdownUnbalancedConstructMarkersError`. A construct kind with no markdown syntax — a bookmark, a division, a tracked change — renders transparently: its extent still appears in place, only the construct's own identity is lost. ## Fidelity diff --git a/dist/ast-DbjiuYr8.d.cts b/dist/ast-8XCbjRQT.d.cts similarity index 78% rename from dist/ast-DbjiuYr8.d.cts rename to dist/ast-8XCbjRQT.d.cts index bc89b83..b9d2a1b 100644 --- a/dist/ast-DbjiuYr8.d.cts +++ b/dist/ast-8XCbjRQT.d.cts @@ -6,7 +6,7 @@ interface MarkdownPosition { readonly endColumn: number; } type MarkdownNode = MarkdownBlockNode | MarkdownInlineNode; -type MarkdownBlockNode = MarkdownDocumentNode | MarkdownParagraphNode | MarkdownHeadingNode | MarkdownBlockquoteNode | MarkdownListNode | MarkdownListItemNode | MarkdownCodeBlockNode | MarkdownThematicBreakNode | MarkdownHtmlBlockNode | MarkdownTableNode | MarkdownTableRowNode | MarkdownTableCellNode | MarkdownMathBlockNode; +type MarkdownBlockNode = MarkdownDocumentNode | MarkdownParagraphNode | MarkdownHeadingNode | MarkdownBlockquoteNode | MarkdownListNode | MarkdownListItemNode | MarkdownCodeBlockNode | MarkdownThematicBreakNode | MarkdownHtmlBlockNode | MarkdownTableNode | MarkdownTableRowNode | MarkdownTableCellNode | MarkdownMathBlockNode | MarkdownFootnoteDefinitionNode; interface MarkdownDocumentNode { readonly type: 'document'; readonly children: MarkdownBlockNode[]; @@ -89,7 +89,13 @@ interface MarkdownMathBlockNode { readonly literal: string; readonly position?: MarkdownPosition; } -type MarkdownInlineNode = MarkdownTextNode | MarkdownEmphasisNode | MarkdownStrongNode | MarkdownStrikethroughNode | MarkdownCodeSpanNode | MarkdownLinkNode | MarkdownImageNode | MarkdownAutolinkNode | MarkdownHardBreakNode | MarkdownSoftBreakNode | MarkdownRawHtmlNode | MarkdownEntityNode | MarkdownMathInlineNode; +interface MarkdownFootnoteDefinitionNode { + readonly type: 'footnoteDefinition'; + readonly label: string; + readonly children: MarkdownBlockNode[]; + readonly position?: MarkdownPosition; +} +type MarkdownInlineNode = MarkdownTextNode | MarkdownEmphasisNode | MarkdownStrongNode | MarkdownStrikethroughNode | MarkdownCodeSpanNode | MarkdownLinkNode | MarkdownImageNode | MarkdownAutolinkNode | MarkdownHardBreakNode | MarkdownSoftBreakNode | MarkdownRawHtmlNode | MarkdownEntityNode | MarkdownMathInlineNode | MarkdownFootnoteReferenceNode; type MarkdownEmphasisMarker = '_' | '*'; interface MarkdownTextNode { readonly type: 'text'; @@ -164,7 +170,12 @@ interface MarkdownMathInlineNode { readonly literal: string; readonly position?: MarkdownPosition; } +interface MarkdownFootnoteReferenceNode { + readonly type: 'footnoteReference'; + readonly label: string; + readonly position?: MarkdownPosition; +} declare function isMarkdownBlockNode(node: MarkdownNode): node is MarkdownBlockNode; declare function isMarkdownInlineNode(node: MarkdownNode): node is MarkdownInlineNode; //#endregion -export { MarkdownStrongNode as A, MarkdownNode as C, MarkdownRawHtmlNode as D, MarkdownPosition as E, MarkdownTextNode as F, MarkdownThematicBreakNode as I, isMarkdownBlockNode as L, MarkdownTableCellNode as M, MarkdownTableNode as N, MarkdownSoftBreakNode as O, MarkdownTableRowNode as P, isMarkdownInlineNode as R, MarkdownMathInlineNode as S, MarkdownParagraphNode as T, MarkdownLinkNode as _, MarkdownCodeBlockNode as a, MarkdownListNode as b, MarkdownEmphasisMarker as c, MarkdownHardBreakNode as d, MarkdownHeadingNode as f, MarkdownInlineNode as g, MarkdownImageNode as h, MarkdownBulletMarker as i, MarkdownTableAlignment as j, MarkdownStrikethroughNode as k, MarkdownEmphasisNode as l, MarkdownHtmlBlockNode as m, MarkdownBlockNode as n, MarkdownCodeSpanNode as o, MarkdownHeadingStyle as p, MarkdownBlockquoteNode as r, MarkdownDocumentNode as s, MarkdownAutolinkNode as t, MarkdownEntityNode as u, MarkdownListItemNode as v, MarkdownOrderedListDelimiter as w, MarkdownMathBlockNode as x, MarkdownListMarkerType as y }; \ No newline at end of file +export { MarkdownSoftBreakNode as A, isMarkdownInlineNode as B, MarkdownMathBlockNode as C, MarkdownParagraphNode as D, MarkdownOrderedListDelimiter as E, MarkdownTableNode as F, MarkdownTableRowNode as I, MarkdownTextNode as L, MarkdownStrongNode as M, MarkdownTableAlignment as N, MarkdownPosition as O, MarkdownTableCellNode as P, MarkdownThematicBreakNode as R, MarkdownListNode as S, MarkdownNode as T, MarkdownImageNode as _, MarkdownCodeBlockNode as a, MarkdownListItemNode as b, MarkdownEmphasisMarker as c, MarkdownFootnoteDefinitionNode as d, MarkdownFootnoteReferenceNode as f, MarkdownHtmlBlockNode as g, MarkdownHeadingStyle as h, MarkdownBulletMarker as i, MarkdownStrikethroughNode as j, MarkdownRawHtmlNode as k, MarkdownEmphasisNode as l, MarkdownHeadingNode as m, MarkdownBlockNode as n, MarkdownCodeSpanNode as o, MarkdownHardBreakNode as p, MarkdownBlockquoteNode as r, MarkdownDocumentNode as s, MarkdownAutolinkNode as t, MarkdownEntityNode as u, MarkdownInlineNode as v, MarkdownMathInlineNode as w, MarkdownListMarkerType as x, MarkdownLinkNode as y, isMarkdownBlockNode as z }; \ No newline at end of file diff --git a/dist/ast-DbjiuYr8.d.ts b/dist/ast-8XCbjRQT.d.ts similarity index 78% rename from dist/ast-DbjiuYr8.d.ts rename to dist/ast-8XCbjRQT.d.ts index bc89b83..b9d2a1b 100644 --- a/dist/ast-DbjiuYr8.d.ts +++ b/dist/ast-8XCbjRQT.d.ts @@ -6,7 +6,7 @@ interface MarkdownPosition { readonly endColumn: number; } type MarkdownNode = MarkdownBlockNode | MarkdownInlineNode; -type MarkdownBlockNode = MarkdownDocumentNode | MarkdownParagraphNode | MarkdownHeadingNode | MarkdownBlockquoteNode | MarkdownListNode | MarkdownListItemNode | MarkdownCodeBlockNode | MarkdownThematicBreakNode | MarkdownHtmlBlockNode | MarkdownTableNode | MarkdownTableRowNode | MarkdownTableCellNode | MarkdownMathBlockNode; +type MarkdownBlockNode = MarkdownDocumentNode | MarkdownParagraphNode | MarkdownHeadingNode | MarkdownBlockquoteNode | MarkdownListNode | MarkdownListItemNode | MarkdownCodeBlockNode | MarkdownThematicBreakNode | MarkdownHtmlBlockNode | MarkdownTableNode | MarkdownTableRowNode | MarkdownTableCellNode | MarkdownMathBlockNode | MarkdownFootnoteDefinitionNode; interface MarkdownDocumentNode { readonly type: 'document'; readonly children: MarkdownBlockNode[]; @@ -89,7 +89,13 @@ interface MarkdownMathBlockNode { readonly literal: string; readonly position?: MarkdownPosition; } -type MarkdownInlineNode = MarkdownTextNode | MarkdownEmphasisNode | MarkdownStrongNode | MarkdownStrikethroughNode | MarkdownCodeSpanNode | MarkdownLinkNode | MarkdownImageNode | MarkdownAutolinkNode | MarkdownHardBreakNode | MarkdownSoftBreakNode | MarkdownRawHtmlNode | MarkdownEntityNode | MarkdownMathInlineNode; +interface MarkdownFootnoteDefinitionNode { + readonly type: 'footnoteDefinition'; + readonly label: string; + readonly children: MarkdownBlockNode[]; + readonly position?: MarkdownPosition; +} +type MarkdownInlineNode = MarkdownTextNode | MarkdownEmphasisNode | MarkdownStrongNode | MarkdownStrikethroughNode | MarkdownCodeSpanNode | MarkdownLinkNode | MarkdownImageNode | MarkdownAutolinkNode | MarkdownHardBreakNode | MarkdownSoftBreakNode | MarkdownRawHtmlNode | MarkdownEntityNode | MarkdownMathInlineNode | MarkdownFootnoteReferenceNode; type MarkdownEmphasisMarker = '_' | '*'; interface MarkdownTextNode { readonly type: 'text'; @@ -164,7 +170,12 @@ interface MarkdownMathInlineNode { readonly literal: string; readonly position?: MarkdownPosition; } +interface MarkdownFootnoteReferenceNode { + readonly type: 'footnoteReference'; + readonly label: string; + readonly position?: MarkdownPosition; +} declare function isMarkdownBlockNode(node: MarkdownNode): node is MarkdownBlockNode; declare function isMarkdownInlineNode(node: MarkdownNode): node is MarkdownInlineNode; //#endregion -export { MarkdownStrongNode as A, MarkdownNode as C, MarkdownRawHtmlNode as D, MarkdownPosition as E, MarkdownTextNode as F, MarkdownThematicBreakNode as I, isMarkdownBlockNode as L, MarkdownTableCellNode as M, MarkdownTableNode as N, MarkdownSoftBreakNode as O, MarkdownTableRowNode as P, isMarkdownInlineNode as R, MarkdownMathInlineNode as S, MarkdownParagraphNode as T, MarkdownLinkNode as _, MarkdownCodeBlockNode as a, MarkdownListNode as b, MarkdownEmphasisMarker as c, MarkdownHardBreakNode as d, MarkdownHeadingNode as f, MarkdownInlineNode as g, MarkdownImageNode as h, MarkdownBulletMarker as i, MarkdownTableAlignment as j, MarkdownStrikethroughNode as k, MarkdownEmphasisNode as l, MarkdownHtmlBlockNode as m, MarkdownBlockNode as n, MarkdownCodeSpanNode as o, MarkdownHeadingStyle as p, MarkdownBlockquoteNode as r, MarkdownDocumentNode as s, MarkdownAutolinkNode as t, MarkdownEntityNode as u, MarkdownListItemNode as v, MarkdownOrderedListDelimiter as w, MarkdownMathBlockNode as x, MarkdownListMarkerType as y }; \ No newline at end of file +export { MarkdownSoftBreakNode as A, isMarkdownInlineNode as B, MarkdownMathBlockNode as C, MarkdownParagraphNode as D, MarkdownOrderedListDelimiter as E, MarkdownTableNode as F, MarkdownTableRowNode as I, MarkdownTextNode as L, MarkdownStrongNode as M, MarkdownTableAlignment as N, MarkdownPosition as O, MarkdownTableCellNode as P, MarkdownThematicBreakNode as R, MarkdownListNode as S, MarkdownNode as T, MarkdownImageNode as _, MarkdownCodeBlockNode as a, MarkdownListItemNode as b, MarkdownEmphasisMarker as c, MarkdownFootnoteDefinitionNode as d, MarkdownFootnoteReferenceNode as f, MarkdownHtmlBlockNode as g, MarkdownHeadingStyle as h, MarkdownBulletMarker as i, MarkdownStrikethroughNode as j, MarkdownRawHtmlNode as k, MarkdownEmphasisNode as l, MarkdownHeadingNode as m, MarkdownBlockNode as n, MarkdownCodeSpanNode as o, MarkdownHardBreakNode as p, MarkdownBlockquoteNode as r, MarkdownDocumentNode as s, MarkdownAutolinkNode as t, MarkdownEntityNode as u, MarkdownInlineNode as v, MarkdownMathInlineNode as w, MarkdownListMarkerType as x, MarkdownLinkNode as y, isMarkdownBlockNode as z }; \ No newline at end of file diff --git a/dist/ast/ast.cjs b/dist/ast/ast.cjs index 181bcec..3282869 100644 --- a/dist/ast/ast.cjs +++ b/dist/ast/ast.cjs @@ -13,7 +13,8 @@ const BLOCK_NODE_TYPES = /* @__PURE__ */ new Set([ "table", "tableRow", "tableCell", - "mathBlock" + "mathBlock", + "footnoteDefinition" ]); function isMarkdownBlockNode(node) { return BLOCK_NODE_TYPES.has(node.type); diff --git a/dist/ast/ast.d.cts b/dist/ast/ast.d.cts index f9f760e..28ca273 100644 --- a/dist/ast/ast.d.cts +++ b/dist/ast/ast.d.cts @@ -1,2 +1,2 @@ -import { A as MarkdownStrongNode, C as MarkdownNode, D as MarkdownRawHtmlNode, E as MarkdownPosition, F as MarkdownTextNode, I as MarkdownThematicBreakNode, L as isMarkdownBlockNode, M as MarkdownTableCellNode, N as MarkdownTableNode, O as MarkdownSoftBreakNode, P as MarkdownTableRowNode, R as isMarkdownInlineNode, S as MarkdownMathInlineNode, T as MarkdownParagraphNode, _ as MarkdownLinkNode, a as MarkdownCodeBlockNode, b as MarkdownListNode, c as MarkdownEmphasisMarker, d as MarkdownHardBreakNode, f as MarkdownHeadingNode, g as MarkdownInlineNode, h as MarkdownImageNode, i as MarkdownBulletMarker, j as MarkdownTableAlignment, k as MarkdownStrikethroughNode, l as MarkdownEmphasisNode, m as MarkdownHtmlBlockNode, n as MarkdownBlockNode, o as MarkdownCodeSpanNode, p as MarkdownHeadingStyle, r as MarkdownBlockquoteNode, s as MarkdownDocumentNode, t as MarkdownAutolinkNode, u as MarkdownEntityNode, v as MarkdownListItemNode, w as MarkdownOrderedListDelimiter, x as MarkdownMathBlockNode, y as MarkdownListMarkerType } from "../ast-DbjiuYr8.cjs"; -export { MarkdownAutolinkNode, MarkdownBlockNode, MarkdownBlockquoteNode, MarkdownBulletMarker, MarkdownCodeBlockNode, MarkdownCodeSpanNode, MarkdownDocumentNode, MarkdownEmphasisMarker, MarkdownEmphasisNode, MarkdownEntityNode, MarkdownHardBreakNode, MarkdownHeadingNode, MarkdownHeadingStyle, MarkdownHtmlBlockNode, MarkdownImageNode, MarkdownInlineNode, MarkdownLinkNode, MarkdownListItemNode, MarkdownListMarkerType, MarkdownListNode, MarkdownMathBlockNode, MarkdownMathInlineNode, MarkdownNode, MarkdownOrderedListDelimiter, MarkdownParagraphNode, MarkdownPosition, MarkdownRawHtmlNode, MarkdownSoftBreakNode, MarkdownStrikethroughNode, MarkdownStrongNode, MarkdownTableAlignment, MarkdownTableCellNode, MarkdownTableNode, MarkdownTableRowNode, MarkdownTextNode, MarkdownThematicBreakNode, isMarkdownBlockNode, isMarkdownInlineNode }; \ No newline at end of file +import { A as MarkdownSoftBreakNode, B as isMarkdownInlineNode, C as MarkdownMathBlockNode, D as MarkdownParagraphNode, E as MarkdownOrderedListDelimiter, F as MarkdownTableNode, I as MarkdownTableRowNode, L as MarkdownTextNode, M as MarkdownStrongNode, N as MarkdownTableAlignment, O as MarkdownPosition, P as MarkdownTableCellNode, R as MarkdownThematicBreakNode, S as MarkdownListNode, T as MarkdownNode, _ as MarkdownImageNode, a as MarkdownCodeBlockNode, b as MarkdownListItemNode, c as MarkdownEmphasisMarker, d as MarkdownFootnoteDefinitionNode, f as MarkdownFootnoteReferenceNode, g as MarkdownHtmlBlockNode, h as MarkdownHeadingStyle, i as MarkdownBulletMarker, j as MarkdownStrikethroughNode, k as MarkdownRawHtmlNode, l as MarkdownEmphasisNode, m as MarkdownHeadingNode, n as MarkdownBlockNode, o as MarkdownCodeSpanNode, p as MarkdownHardBreakNode, r as MarkdownBlockquoteNode, s as MarkdownDocumentNode, t as MarkdownAutolinkNode, u as MarkdownEntityNode, v as MarkdownInlineNode, w as MarkdownMathInlineNode, x as MarkdownListMarkerType, y as MarkdownLinkNode, z as isMarkdownBlockNode } from "../ast-8XCbjRQT.cjs"; +export { MarkdownAutolinkNode, MarkdownBlockNode, MarkdownBlockquoteNode, MarkdownBulletMarker, MarkdownCodeBlockNode, MarkdownCodeSpanNode, MarkdownDocumentNode, MarkdownEmphasisMarker, MarkdownEmphasisNode, MarkdownEntityNode, MarkdownFootnoteDefinitionNode, MarkdownFootnoteReferenceNode, MarkdownHardBreakNode, MarkdownHeadingNode, MarkdownHeadingStyle, MarkdownHtmlBlockNode, MarkdownImageNode, MarkdownInlineNode, MarkdownLinkNode, MarkdownListItemNode, MarkdownListMarkerType, MarkdownListNode, MarkdownMathBlockNode, MarkdownMathInlineNode, MarkdownNode, MarkdownOrderedListDelimiter, MarkdownParagraphNode, MarkdownPosition, MarkdownRawHtmlNode, MarkdownSoftBreakNode, MarkdownStrikethroughNode, MarkdownStrongNode, MarkdownTableAlignment, MarkdownTableCellNode, MarkdownTableNode, MarkdownTableRowNode, MarkdownTextNode, MarkdownThematicBreakNode, isMarkdownBlockNode, isMarkdownInlineNode }; \ No newline at end of file diff --git a/dist/ast/ast.d.ts b/dist/ast/ast.d.ts index 931f9bc..266ae50 100644 --- a/dist/ast/ast.d.ts +++ b/dist/ast/ast.d.ts @@ -1,2 +1,2 @@ -import { A as MarkdownStrongNode, C as MarkdownNode, D as MarkdownRawHtmlNode, E as MarkdownPosition, F as MarkdownTextNode, I as MarkdownThematicBreakNode, L as isMarkdownBlockNode, M as MarkdownTableCellNode, N as MarkdownTableNode, O as MarkdownSoftBreakNode, P as MarkdownTableRowNode, R as isMarkdownInlineNode, S as MarkdownMathInlineNode, T as MarkdownParagraphNode, _ as MarkdownLinkNode, a as MarkdownCodeBlockNode, b as MarkdownListNode, c as MarkdownEmphasisMarker, d as MarkdownHardBreakNode, f as MarkdownHeadingNode, g as MarkdownInlineNode, h as MarkdownImageNode, i as MarkdownBulletMarker, j as MarkdownTableAlignment, k as MarkdownStrikethroughNode, l as MarkdownEmphasisNode, m as MarkdownHtmlBlockNode, n as MarkdownBlockNode, o as MarkdownCodeSpanNode, p as MarkdownHeadingStyle, r as MarkdownBlockquoteNode, s as MarkdownDocumentNode, t as MarkdownAutolinkNode, u as MarkdownEntityNode, v as MarkdownListItemNode, w as MarkdownOrderedListDelimiter, x as MarkdownMathBlockNode, y as MarkdownListMarkerType } from "../ast-DbjiuYr8.js"; -export { MarkdownAutolinkNode, MarkdownBlockNode, MarkdownBlockquoteNode, MarkdownBulletMarker, MarkdownCodeBlockNode, MarkdownCodeSpanNode, MarkdownDocumentNode, MarkdownEmphasisMarker, MarkdownEmphasisNode, MarkdownEntityNode, MarkdownHardBreakNode, MarkdownHeadingNode, MarkdownHeadingStyle, MarkdownHtmlBlockNode, MarkdownImageNode, MarkdownInlineNode, MarkdownLinkNode, MarkdownListItemNode, MarkdownListMarkerType, MarkdownListNode, MarkdownMathBlockNode, MarkdownMathInlineNode, MarkdownNode, MarkdownOrderedListDelimiter, MarkdownParagraphNode, MarkdownPosition, MarkdownRawHtmlNode, MarkdownSoftBreakNode, MarkdownStrikethroughNode, MarkdownStrongNode, MarkdownTableAlignment, MarkdownTableCellNode, MarkdownTableNode, MarkdownTableRowNode, MarkdownTextNode, MarkdownThematicBreakNode, isMarkdownBlockNode, isMarkdownInlineNode }; \ No newline at end of file +import { A as MarkdownSoftBreakNode, B as isMarkdownInlineNode, C as MarkdownMathBlockNode, D as MarkdownParagraphNode, E as MarkdownOrderedListDelimiter, F as MarkdownTableNode, I as MarkdownTableRowNode, L as MarkdownTextNode, M as MarkdownStrongNode, N as MarkdownTableAlignment, O as MarkdownPosition, P as MarkdownTableCellNode, R as MarkdownThematicBreakNode, S as MarkdownListNode, T as MarkdownNode, _ as MarkdownImageNode, a as MarkdownCodeBlockNode, b as MarkdownListItemNode, c as MarkdownEmphasisMarker, d as MarkdownFootnoteDefinitionNode, f as MarkdownFootnoteReferenceNode, g as MarkdownHtmlBlockNode, h as MarkdownHeadingStyle, i as MarkdownBulletMarker, j as MarkdownStrikethroughNode, k as MarkdownRawHtmlNode, l as MarkdownEmphasisNode, m as MarkdownHeadingNode, n as MarkdownBlockNode, o as MarkdownCodeSpanNode, p as MarkdownHardBreakNode, r as MarkdownBlockquoteNode, s as MarkdownDocumentNode, t as MarkdownAutolinkNode, u as MarkdownEntityNode, v as MarkdownInlineNode, w as MarkdownMathInlineNode, x as MarkdownListMarkerType, y as MarkdownLinkNode, z as isMarkdownBlockNode } from "../ast-8XCbjRQT.js"; +export { MarkdownAutolinkNode, MarkdownBlockNode, MarkdownBlockquoteNode, MarkdownBulletMarker, MarkdownCodeBlockNode, MarkdownCodeSpanNode, MarkdownDocumentNode, MarkdownEmphasisMarker, MarkdownEmphasisNode, MarkdownEntityNode, MarkdownFootnoteDefinitionNode, MarkdownFootnoteReferenceNode, MarkdownHardBreakNode, MarkdownHeadingNode, MarkdownHeadingStyle, MarkdownHtmlBlockNode, MarkdownImageNode, MarkdownInlineNode, MarkdownLinkNode, MarkdownListItemNode, MarkdownListMarkerType, MarkdownListNode, MarkdownMathBlockNode, MarkdownMathInlineNode, MarkdownNode, MarkdownOrderedListDelimiter, MarkdownParagraphNode, MarkdownPosition, MarkdownRawHtmlNode, MarkdownSoftBreakNode, MarkdownStrikethroughNode, MarkdownStrongNode, MarkdownTableAlignment, MarkdownTableCellNode, MarkdownTableNode, MarkdownTableRowNode, MarkdownTextNode, MarkdownThematicBreakNode, isMarkdownBlockNode, isMarkdownInlineNode }; \ No newline at end of file diff --git a/dist/ast/ast.js b/dist/ast/ast.js index 8b29c9b..8cd36e2 100644 --- a/dist/ast/ast.js +++ b/dist/ast/ast.js @@ -12,7 +12,8 @@ const BLOCK_NODE_TYPES = /* @__PURE__ */ new Set([ "table", "tableRow", "tableCell", - "mathBlock" + "mathBlock", + "footnoteDefinition" ]); function isMarkdownBlockNode(node) { return BLOCK_NODE_TYPES.has(node.type); diff --git a/dist/block/block.cjs b/dist/block/block.cjs index eb483d9..53bd029 100644 --- a/dist/block/block.cjs +++ b/dist/block/block.cjs @@ -1,4 +1,5 @@ Object.defineProperty(exports, Symbol.toStringTag, { value: "Module" }); +const require_inline_footnote = require("../inline/footnote.cjs"); require("../defaults/defaults.cjs"); const require_diagnostics_diagnostics = require("../diagnostics/diagnostics.cjs"); const require_html_html = require("../html/html.cjs"); @@ -14,7 +15,7 @@ const TASK_LIST_MARKER_PATTERN = /^\[([ xX])\][ \t]/; const NUL_REPLACEMENT = "�"; const NUL_PATTERN = /\0/g; const LINE_ENDING_PATTERN = /\r\n|\n|\r/; -const MAYBE_SPECIAL_PATTERN = /^[#$`~*+_=<>0-9|:-]/; +const MAYBE_SPECIAL_PATTERN = /^[#$`~*+_=<>[0-9|:-]/; const ATX_MARKER_PATTERN = /^#{1,6}(?:[ \t]+|$)/; const ATX_ONLY_CLOSING_SEQUENCE_PATTERN = /^[ \t]*#+[ \t]*$/; const ATX_TRAILING_CLOSING_SEQUENCE_PATTERN = /[ \t]+#+[ \t]*$/; @@ -46,8 +47,10 @@ function headingLevelOf(hashes) { } var BlockParser = class { references = /* @__PURE__ */ new Map(); + footnotes = /* @__PURE__ */ new Set(); document = new require_block_node.BlockNode("document", 1); tables; + footnotesEnabled; sink; maxNesting; tip = this.document; @@ -59,6 +62,7 @@ var BlockParser = class { nestingDepth = 0; constructor(options) { this.tables = options.gfmTables ?? true; + this.footnotesEnabled = options.footnotes ?? true; this.sink = options.sink ?? require_diagnostics_diagnostics.NOOP_MARKDOWN_DIAGNOSTIC_SINK; this.maxNesting = options.maxNesting ?? 250; } @@ -129,6 +133,7 @@ var BlockParser = class { case "list": return "matched"; case "blockquote": return this.continueBlockquote(); case "listItem": return this.continueListItem(node); + case "footnoteDefinition": return this.continueFootnoteDefinition(node); case "codeBlock": return this.continueCodeBlock(node); case "mathBlock": return this.continueMathBlock(node); case "htmlBlock": return this.line.blank && HTML_BLOCK_BLANK_LINE_END_TYPES.includes(node.htmlBlockType) ? "not-matched" : "matched"; @@ -163,6 +168,18 @@ var BlockParser = class { } return "not-matched"; } + continueFootnoteDefinition(node) { + if (this.line.blank) { + if (node.children.length === 0) return "not-matched"; + this.line.advanceToNextNonspace(); + return "matched"; + } + if (this.line.indent >= 4) { + this.line.advance(4); + return "matched"; + } + return "not-matched"; + } continueCodeBlock(node) { if (!node.fenced) { if (this.line.indent >= 4) { @@ -216,6 +233,7 @@ var BlockParser = class { () => this.tryAtxHeadingStart(), () => this.tryCodeFenceStart(), () => this.tryMathBlockStart(), + () => this.tryFootnoteDefinitionStart(container), () => this.tryHtmlBlockStart(container), () => this.tryPromoteParagraph(container), () => this.tryThematicBreakStart(), @@ -273,6 +291,29 @@ var BlockParser = class { this.line.advance(MATH_BLOCK_MARKER_LENGTH); return "leaf"; } + tryFootnoteDefinitionStart(container) { + if (!this.footnotesEnabled || this.line.indented || !this.footnoteDefinitionMayOpenIn(container)) return "none"; + const marker = require_inline_footnote.matchFootnoteDefinitionMarker(this.line.restFromNextNonspace()); + if (marker === void 0) return "none"; + if (this.footnotes.has(marker.label)) this.sink({ + code: require_diagnostics_diagnostics.MarkdownDiagnosticCodes.DUPLICATE_FOOTNOTE_DEFINITION, + severity: "warning", + message: `footnote "${marker.label}" was already defined earlier in the document; every reference resolves to the first definition, and both definitions are kept as written`, + line: this.lineNumber + }); + this.footnotes.add(marker.label); + this.line.advanceToNextNonspace(); + this.closeUnmatchedBlocks(); + const node = this.addChild("footnoteDefinition"); + node.footnoteLabel = marker.label; + this.line.advance(marker.markerLength); + return "container"; + } + footnoteDefinitionMayOpenIn(container) { + let node = container; + while (node?.kind === "list") node = node.parent; + return node?.kind === "document"; + } tryHtmlBlockStart(container) { if (this.line.indented || this.line.peekNextNonspace() !== "<") return "none"; const interruptsParagraph = container.kind === "paragraph" || !this.allClosed && !this.line.blank && this.tip.kind === "paragraph"; @@ -429,15 +470,15 @@ var BlockParser = class { node.literal = breakIndex === -1 ? "" : node.content.slice(breakIndex + 1); } }; -function toInlineChildren(content, references, options) { - return require_inline_inline.parseInlines(content.trim(), references, options); +function toInlineChildren(content, context) { + return require_inline_inline.parseInlines(content.trim(), context.references, context.footnotes, context.options); } -function toHeadingNode(node, references, options) { +function toHeadingNode(node, context) { return { type: "heading", level: node.level, style: node.setext ? "setext" : "atx", - children: toInlineChildren(node.content, references, options) + children: toInlineChildren(node.content, context) }; } function extractTaskListMarker(itemChildren) { @@ -448,19 +489,19 @@ function extractTaskListMarker(itemChildren) { first.content = first.content.slice(match[0].length); return match[1] !== " "; } -function toListItemNode(item, references, options) { - const checked = options.gfmTaskLists ?? true ? extractTaskListMarker(item.children) : void 0; +function toListItemNode(item, context) { + const checked = context.options.gfmTaskLists ?? true ? extractTaskListMarker(item.children) : void 0; return checked === void 0 ? { type: "listItem", - children: toAstBlocks(item.children, references, options) + children: toAstBlocks(item.children, context) } : { type: "listItem", checked, - children: toAstBlocks(item.children, references, options) + children: toAstBlocks(item.children, context) }; } -function toListNode(node, references, options) { - const children = node.children.map((item) => toListItemNode(item, references, options)); +function toListNode(node, context) { + const children = node.children.map((item) => toListItemNode(item, context)); const data = node.listData; if (data?.type === "ordered") return { type: "list", @@ -478,20 +519,20 @@ function toListNode(node, references, options) { children }; } -function toTableRow(cells, header, references, options) { +function toTableRow(cells, header, context) { return { type: "tableRow", header, children: cells.map((cell) => ({ type: "tableCell", - children: toInlineChildren(cell, references, options) + children: toInlineChildren(cell, context) })) }; } -function toTableNode(node, references, options) { - const sink = options.sink ?? require_diagnostics_diagnostics.NOOP_MARKDOWN_DIAGNOSTIC_SINK; +function toTableNode(node, context) { + const sink = context.options.sink ?? require_diagnostics_diagnostics.NOOP_MARKDOWN_DIAGNOSTIC_SINK; const columnCount = node.alignments.length; - const rows = [toTableRow(require_block_table.fitRowToColumns(require_block_table.splitTableRow(node.headerLine), columnCount), true, references, options)]; + const rows = [toTableRow(require_block_table.fitRowToColumns(require_block_table.splitTableRow(node.headerLine), columnCount), true, context)]; for (const rowLine of node.content.split("\n")) { if (rowLine.trim().length === 0) continue; const cells = require_block_table.splitTableRow(rowLine); @@ -501,7 +542,7 @@ function toTableNode(node, references, options) { message: `table row has ${String(cells.length)} cell(s), but the header row declares ${String(columnCount)}; the row is padded with empty cells or truncated to fit`, line: node.startLine }); - rows.push(toTableRow(require_block_table.fitRowToColumns(cells, columnCount), false, references, options)); + rows.push(toTableRow(require_block_table.fitRowToColumns(cells, columnCount), false, context)); } return { type: "table", @@ -509,18 +550,23 @@ function toTableNode(node, references, options) { children: rows }; } -function toAstBlock(node, references, options) { +function toAstBlock(node, context) { switch (node.kind) { case "paragraph": return { type: "paragraph", - children: toInlineChildren(node.content, references, options) + children: toInlineChildren(node.content, context) }; - case "heading": return toHeadingNode(node, references, options); + case "heading": return toHeadingNode(node, context); case "blockquote": return { type: "blockquote", - children: toAstBlocks(node.children, references, options) + children: toAstBlocks(node.children, context) + }; + case "list": return toListNode(node, context); + case "footnoteDefinition": return { + type: "footnoteDefinition", + label: node.footnoteLabel, + children: toAstBlocks(node.children, context) }; - case "list": return toListNode(node, references, options); case "codeBlock": return node.fenced ? { type: "codeBlock", fenced: true, @@ -541,15 +587,15 @@ function toAstBlock(node, references, options) { type: "mathBlock", literal: node.literal }; - case "table": return toTableNode(node, references, options); + case "table": return toTableNode(node, context); case "document": case "listItem": return; } } -function toAstBlocks(nodes, references, options) { +function toAstBlocks(nodes, context) { const blocks = []; for (const node of nodes) { - const converted = toAstBlock(node, references, options); + const converted = toAstBlock(node, context); if (converted !== void 0) blocks.push(converted); } return blocks; @@ -557,13 +603,18 @@ function toAstBlocks(nodes, references, options) { function parseMarkdown(source, options = {}) { const parser = new BlockParser(options); const root = parser.parse(source); - const references = parser.references; + const context = { + references: parser.references, + footnotes: parser.footnotes, + options + }; return { document: { type: "document", - children: toAstBlocks(root.children, references, options) + children: toAstBlocks(root.children, context) }, - references + references: context.references, + footnotes: context.footnotes }; } //#endregion diff --git a/dist/block/block.d.cts b/dist/block/block.d.cts index 6061c80..8332b7e 100644 --- a/dist/block/block.d.cts +++ b/dist/block/block.d.cts @@ -1,17 +1,20 @@ -import { s as MarkdownDocumentNode } from "../ast-DbjiuYr8.cjs"; -import { i as MarkdownDiagnosticSink } from "../diagnostics-B72W0P_E.cjs"; +import { s as MarkdownDocumentNode } from "../ast-8XCbjRQT.cjs"; +import { r as FootnoteLabelSet } from "../footnote-CKk4JbLk.cjs"; +import { i as MarkdownDiagnosticSink } from "../diagnostics-BuO5-SW1.cjs"; import { n as LinkReferenceMap } from "../link-Dv4kxVjk.cjs"; -import { t as InlineParseOptions } from "../inline-uVHJ5xzT.cjs"; +import { t as InlineParseOptions } from "../inline-CXVQWQnW.cjs"; //#region src/block/block.d.ts interface MarkdownParseOptions extends InlineParseOptions { readonly gfmTables?: boolean; readonly gfmTaskLists?: boolean; + readonly footnotes?: boolean; readonly maxNesting?: number; readonly sink?: MarkdownDiagnosticSink; } interface ParsedMarkdown { readonly document: MarkdownDocumentNode; readonly references: LinkReferenceMap; + readonly footnotes: FootnoteLabelSet; } declare function parseMarkdown(source: string, options?: MarkdownParseOptions): ParsedMarkdown; //#endregion diff --git a/dist/block/block.d.ts b/dist/block/block.d.ts index e4d752a..fd6ddbe 100644 --- a/dist/block/block.d.ts +++ b/dist/block/block.d.ts @@ -1,17 +1,20 @@ -import { s as MarkdownDocumentNode } from "../ast-DbjiuYr8.js"; -import { i as MarkdownDiagnosticSink } from "../diagnostics-B72W0P_E.js"; +import { s as MarkdownDocumentNode } from "../ast-8XCbjRQT.js"; +import { r as FootnoteLabelSet } from "../footnote-CKk4JbLk.js"; +import { i as MarkdownDiagnosticSink } from "../diagnostics-BuO5-SW1.js"; import { n as LinkReferenceMap } from "../link-Dv4kxVjk.js"; -import { t as InlineParseOptions } from "../inline-TuBQ2TUr.js"; +import { t as InlineParseOptions } from "../inline-B_V7bs5j.js"; //#region src/block/block.d.ts interface MarkdownParseOptions extends InlineParseOptions { readonly gfmTables?: boolean; readonly gfmTaskLists?: boolean; + readonly footnotes?: boolean; readonly maxNesting?: number; readonly sink?: MarkdownDiagnosticSink; } interface ParsedMarkdown { readonly document: MarkdownDocumentNode; readonly references: LinkReferenceMap; + readonly footnotes: FootnoteLabelSet; } declare function parseMarkdown(source: string, options?: MarkdownParseOptions): ParsedMarkdown; //#endregion diff --git a/dist/block/block.js b/dist/block/block.js index 6064516..077ddfd 100644 --- a/dist/block/block.js +++ b/dist/block/block.js @@ -1,3 +1,4 @@ +import { matchFootnoteDefinitionMarker } from "../inline/footnote.js"; import "../defaults/defaults.js"; import { MarkdownDiagnosticCodes, MarkdownNestingLimitExceededError, NOOP_MARKDOWN_DIAGNOSTIC_SINK } from "../diagnostics/diagnostics.js"; import { matchHtmlBlockStart, matchesHtmlBlockEnd } from "../html/html.js"; @@ -13,7 +14,7 @@ const TASK_LIST_MARKER_PATTERN = /^\[([ xX])\][ \t]/; const NUL_REPLACEMENT = "�"; const NUL_PATTERN = /\0/g; const LINE_ENDING_PATTERN = /\r\n|\n|\r/; -const MAYBE_SPECIAL_PATTERN = /^[#$`~*+_=<>0-9|:-]/; +const MAYBE_SPECIAL_PATTERN = /^[#$`~*+_=<>[0-9|:-]/; const ATX_MARKER_PATTERN = /^#{1,6}(?:[ \t]+|$)/; const ATX_ONLY_CLOSING_SEQUENCE_PATTERN = /^[ \t]*#+[ \t]*$/; const ATX_TRAILING_CLOSING_SEQUENCE_PATTERN = /[ \t]+#+[ \t]*$/; @@ -45,8 +46,10 @@ function headingLevelOf(hashes) { } var BlockParser = class { references = /* @__PURE__ */ new Map(); + footnotes = /* @__PURE__ */ new Set(); document = new BlockNode("document", 1); tables; + footnotesEnabled; sink; maxNesting; tip = this.document; @@ -58,6 +61,7 @@ var BlockParser = class { nestingDepth = 0; constructor(options) { this.tables = options.gfmTables ?? true; + this.footnotesEnabled = options.footnotes ?? true; this.sink = options.sink ?? NOOP_MARKDOWN_DIAGNOSTIC_SINK; this.maxNesting = options.maxNesting ?? 250; } @@ -128,6 +132,7 @@ var BlockParser = class { case "list": return "matched"; case "blockquote": return this.continueBlockquote(); case "listItem": return this.continueListItem(node); + case "footnoteDefinition": return this.continueFootnoteDefinition(node); case "codeBlock": return this.continueCodeBlock(node); case "mathBlock": return this.continueMathBlock(node); case "htmlBlock": return this.line.blank && HTML_BLOCK_BLANK_LINE_END_TYPES.includes(node.htmlBlockType) ? "not-matched" : "matched"; @@ -162,6 +167,18 @@ var BlockParser = class { } return "not-matched"; } + continueFootnoteDefinition(node) { + if (this.line.blank) { + if (node.children.length === 0) return "not-matched"; + this.line.advanceToNextNonspace(); + return "matched"; + } + if (this.line.indent >= 4) { + this.line.advance(4); + return "matched"; + } + return "not-matched"; + } continueCodeBlock(node) { if (!node.fenced) { if (this.line.indent >= 4) { @@ -215,6 +232,7 @@ var BlockParser = class { () => this.tryAtxHeadingStart(), () => this.tryCodeFenceStart(), () => this.tryMathBlockStart(), + () => this.tryFootnoteDefinitionStart(container), () => this.tryHtmlBlockStart(container), () => this.tryPromoteParagraph(container), () => this.tryThematicBreakStart(), @@ -272,6 +290,29 @@ var BlockParser = class { this.line.advance(MATH_BLOCK_MARKER_LENGTH); return "leaf"; } + tryFootnoteDefinitionStart(container) { + if (!this.footnotesEnabled || this.line.indented || !this.footnoteDefinitionMayOpenIn(container)) return "none"; + const marker = matchFootnoteDefinitionMarker(this.line.restFromNextNonspace()); + if (marker === void 0) return "none"; + if (this.footnotes.has(marker.label)) this.sink({ + code: MarkdownDiagnosticCodes.DUPLICATE_FOOTNOTE_DEFINITION, + severity: "warning", + message: `footnote "${marker.label}" was already defined earlier in the document; every reference resolves to the first definition, and both definitions are kept as written`, + line: this.lineNumber + }); + this.footnotes.add(marker.label); + this.line.advanceToNextNonspace(); + this.closeUnmatchedBlocks(); + const node = this.addChild("footnoteDefinition"); + node.footnoteLabel = marker.label; + this.line.advance(marker.markerLength); + return "container"; + } + footnoteDefinitionMayOpenIn(container) { + let node = container; + while (node?.kind === "list") node = node.parent; + return node?.kind === "document"; + } tryHtmlBlockStart(container) { if (this.line.indented || this.line.peekNextNonspace() !== "<") return "none"; const interruptsParagraph = container.kind === "paragraph" || !this.allClosed && !this.line.blank && this.tip.kind === "paragraph"; @@ -428,15 +469,15 @@ var BlockParser = class { node.literal = breakIndex === -1 ? "" : node.content.slice(breakIndex + 1); } }; -function toInlineChildren(content, references, options) { - return parseInlines(content.trim(), references, options); +function toInlineChildren(content, context) { + return parseInlines(content.trim(), context.references, context.footnotes, context.options); } -function toHeadingNode(node, references, options) { +function toHeadingNode(node, context) { return { type: "heading", level: node.level, style: node.setext ? "setext" : "atx", - children: toInlineChildren(node.content, references, options) + children: toInlineChildren(node.content, context) }; } function extractTaskListMarker(itemChildren) { @@ -447,19 +488,19 @@ function extractTaskListMarker(itemChildren) { first.content = first.content.slice(match[0].length); return match[1] !== " "; } -function toListItemNode(item, references, options) { - const checked = options.gfmTaskLists ?? true ? extractTaskListMarker(item.children) : void 0; +function toListItemNode(item, context) { + const checked = context.options.gfmTaskLists ?? true ? extractTaskListMarker(item.children) : void 0; return checked === void 0 ? { type: "listItem", - children: toAstBlocks(item.children, references, options) + children: toAstBlocks(item.children, context) } : { type: "listItem", checked, - children: toAstBlocks(item.children, references, options) + children: toAstBlocks(item.children, context) }; } -function toListNode(node, references, options) { - const children = node.children.map((item) => toListItemNode(item, references, options)); +function toListNode(node, context) { + const children = node.children.map((item) => toListItemNode(item, context)); const data = node.listData; if (data?.type === "ordered") return { type: "list", @@ -477,20 +518,20 @@ function toListNode(node, references, options) { children }; } -function toTableRow(cells, header, references, options) { +function toTableRow(cells, header, context) { return { type: "tableRow", header, children: cells.map((cell) => ({ type: "tableCell", - children: toInlineChildren(cell, references, options) + children: toInlineChildren(cell, context) })) }; } -function toTableNode(node, references, options) { - const sink = options.sink ?? NOOP_MARKDOWN_DIAGNOSTIC_SINK; +function toTableNode(node, context) { + const sink = context.options.sink ?? NOOP_MARKDOWN_DIAGNOSTIC_SINK; const columnCount = node.alignments.length; - const rows = [toTableRow(fitRowToColumns(splitTableRow(node.headerLine), columnCount), true, references, options)]; + const rows = [toTableRow(fitRowToColumns(splitTableRow(node.headerLine), columnCount), true, context)]; for (const rowLine of node.content.split("\n")) { if (rowLine.trim().length === 0) continue; const cells = splitTableRow(rowLine); @@ -500,7 +541,7 @@ function toTableNode(node, references, options) { message: `table row has ${String(cells.length)} cell(s), but the header row declares ${String(columnCount)}; the row is padded with empty cells or truncated to fit`, line: node.startLine }); - rows.push(toTableRow(fitRowToColumns(cells, columnCount), false, references, options)); + rows.push(toTableRow(fitRowToColumns(cells, columnCount), false, context)); } return { type: "table", @@ -508,18 +549,23 @@ function toTableNode(node, references, options) { children: rows }; } -function toAstBlock(node, references, options) { +function toAstBlock(node, context) { switch (node.kind) { case "paragraph": return { type: "paragraph", - children: toInlineChildren(node.content, references, options) + children: toInlineChildren(node.content, context) }; - case "heading": return toHeadingNode(node, references, options); + case "heading": return toHeadingNode(node, context); case "blockquote": return { type: "blockquote", - children: toAstBlocks(node.children, references, options) + children: toAstBlocks(node.children, context) + }; + case "list": return toListNode(node, context); + case "footnoteDefinition": return { + type: "footnoteDefinition", + label: node.footnoteLabel, + children: toAstBlocks(node.children, context) }; - case "list": return toListNode(node, references, options); case "codeBlock": return node.fenced ? { type: "codeBlock", fenced: true, @@ -540,15 +586,15 @@ function toAstBlock(node, references, options) { type: "mathBlock", literal: node.literal }; - case "table": return toTableNode(node, references, options); + case "table": return toTableNode(node, context); case "document": case "listItem": return; } } -function toAstBlocks(nodes, references, options) { +function toAstBlocks(nodes, context) { const blocks = []; for (const node of nodes) { - const converted = toAstBlock(node, references, options); + const converted = toAstBlock(node, context); if (converted !== void 0) blocks.push(converted); } return blocks; @@ -556,13 +602,18 @@ function toAstBlocks(nodes, references, options) { function parseMarkdown(source, options = {}) { const parser = new BlockParser(options); const root = parser.parse(source); - const references = parser.references; + const context = { + references: parser.references, + footnotes: parser.footnotes, + options + }; return { document: { type: "document", - children: toAstBlocks(root.children, references, options) + children: toAstBlocks(root.children, context) }, - references + references: context.references, + footnotes: context.footnotes }; } //#endregion diff --git a/dist/block/definitions.d.cts b/dist/block/definitions.d.cts index a988d01..9ecab99 100644 --- a/dist/block/definitions.d.cts +++ b/dist/block/definitions.d.cts @@ -1,4 +1,4 @@ -import { i as MarkdownDiagnosticSink } from "../diagnostics-B72W0P_E.cjs"; +import { i as MarkdownDiagnosticSink } from "../diagnostics-BuO5-SW1.cjs"; import { t as LinkReferenceDefinition } from "../link-Dv4kxVjk.cjs"; //#region src/block/definitions.d.ts declare function extractDefinitions(content: string, references: Map, sink?: MarkdownDiagnosticSink, startLine?: number): string; diff --git a/dist/block/definitions.d.ts b/dist/block/definitions.d.ts index 8b6318b..248c928 100644 --- a/dist/block/definitions.d.ts +++ b/dist/block/definitions.d.ts @@ -1,4 +1,4 @@ -import { i as MarkdownDiagnosticSink } from "../diagnostics-B72W0P_E.js"; +import { i as MarkdownDiagnosticSink } from "../diagnostics-BuO5-SW1.js"; import { t as LinkReferenceDefinition } from "../link-Dv4kxVjk.js"; //#region src/block/definitions.d.ts declare function extractDefinitions(content: string, references: Map, sink?: MarkdownDiagnosticSink, startLine?: number): string; diff --git a/dist/block/node.cjs b/dist/block/node.cjs index bb6750e..84348cc 100644 --- a/dist/block/node.cjs +++ b/dist/block/node.cjs @@ -22,6 +22,7 @@ var BlockNode = class { tight = true; alignments = []; headerLine = ""; + footnoteLabel = ""; constructor(kind, startLine) { this.kind = kind; this.startLine = startLine; @@ -52,6 +53,7 @@ var BlockNode = class { }; function canContain(parent, child) { switch (parent) { + case "footnoteDefinition": return child !== "listItem" && child !== "footnoteDefinition"; case "document": case "blockquote": case "listItem": return child !== "listItem"; diff --git a/dist/block/node.d.cts b/dist/block/node.d.cts index 75f36bb..498bece 100644 --- a/dist/block/node.d.cts +++ b/dist/block/node.d.cts @@ -1,7 +1,7 @@ -import { i as MarkdownBulletMarker, j as MarkdownTableAlignment, w as MarkdownOrderedListDelimiter } from "../ast-DbjiuYr8.cjs"; +import { E as MarkdownOrderedListDelimiter, N as MarkdownTableAlignment, i as MarkdownBulletMarker } from "../ast-8XCbjRQT.cjs"; import { t as HtmlBlockType } from "../html-bkz2QTuq.cjs"; //#region src/block/node.d.ts -type BlockNodeKind = 'document' | 'paragraph' | 'heading' | 'blockquote' | 'list' | 'listItem' | 'codeBlock' | 'htmlBlock' | 'thematicBreak' | 'table' | 'mathBlock'; +type BlockNodeKind = 'document' | 'paragraph' | 'heading' | 'blockquote' | 'list' | 'listItem' | 'codeBlock' | 'htmlBlock' | 'thematicBreak' | 'table' | 'mathBlock' | 'footnoteDefinition'; type BlockHeadingLevel = 1 | 2 | 3 | 4 | 5 | 6; interface ListMarkerData { readonly type: 'bullet' | 'ordered'; @@ -33,6 +33,7 @@ declare class BlockNode { tight: boolean; alignments: MarkdownTableAlignment[]; headerLine: string; + footnoteLabel: string; constructor(kind: BlockNodeKind, startLine: number); get lastChild(): BlockNode | undefined; appendChild(child: BlockNode): void; diff --git a/dist/block/node.d.ts b/dist/block/node.d.ts index 2e36796..2c6c844 100644 --- a/dist/block/node.d.ts +++ b/dist/block/node.d.ts @@ -1,7 +1,7 @@ -import { i as MarkdownBulletMarker, j as MarkdownTableAlignment, w as MarkdownOrderedListDelimiter } from "../ast-DbjiuYr8.js"; +import { E as MarkdownOrderedListDelimiter, N as MarkdownTableAlignment, i as MarkdownBulletMarker } from "../ast-8XCbjRQT.js"; import { t as HtmlBlockType } from "../html-bkz2QTuq.js"; //#region src/block/node.d.ts -type BlockNodeKind = 'document' | 'paragraph' | 'heading' | 'blockquote' | 'list' | 'listItem' | 'codeBlock' | 'htmlBlock' | 'thematicBreak' | 'table' | 'mathBlock'; +type BlockNodeKind = 'document' | 'paragraph' | 'heading' | 'blockquote' | 'list' | 'listItem' | 'codeBlock' | 'htmlBlock' | 'thematicBreak' | 'table' | 'mathBlock' | 'footnoteDefinition'; type BlockHeadingLevel = 1 | 2 | 3 | 4 | 5 | 6; interface ListMarkerData { readonly type: 'bullet' | 'ordered'; @@ -33,6 +33,7 @@ declare class BlockNode { tight: boolean; alignments: MarkdownTableAlignment[]; headerLine: string; + footnoteLabel: string; constructor(kind: BlockNodeKind, startLine: number); get lastChild(): BlockNode | undefined; appendChild(child: BlockNode): void; diff --git a/dist/block/node.js b/dist/block/node.js index 0fb02b8..01d8afe 100644 --- a/dist/block/node.js +++ b/dist/block/node.js @@ -21,6 +21,7 @@ var BlockNode = class { tight = true; alignments = []; headerLine = ""; + footnoteLabel = ""; constructor(kind, startLine) { this.kind = kind; this.startLine = startLine; @@ -51,6 +52,7 @@ var BlockNode = class { }; function canContain(parent, child) { switch (parent) { + case "footnoteDefinition": return child !== "listItem" && child !== "footnoteDefinition"; case "document": case "blockquote": case "listItem": return child !== "listItem"; diff --git a/dist/block/table.d.cts b/dist/block/table.d.cts index 3c9a2ed..59bfe0a 100644 --- a/dist/block/table.d.cts +++ b/dist/block/table.d.cts @@ -1,4 +1,4 @@ -import { j as MarkdownTableAlignment } from "../ast-DbjiuYr8.cjs"; +import { N as MarkdownTableAlignment } from "../ast-8XCbjRQT.cjs"; //#region src/block/table.d.ts declare function splitTableRow(line: string): string[]; declare function parseTableDelimiterRow(line: string): MarkdownTableAlignment[] | undefined; diff --git a/dist/block/table.d.ts b/dist/block/table.d.ts index cfbfd5d..ebdf9bd 100644 --- a/dist/block/table.d.ts +++ b/dist/block/table.d.ts @@ -1,4 +1,4 @@ -import { j as MarkdownTableAlignment } from "../ast-DbjiuYr8.js"; +import { N as MarkdownTableAlignment } from "../ast-8XCbjRQT.js"; //#region src/block/table.d.ts declare function splitTableRow(line: string): string[]; declare function parseTableDelimiterRow(line: string): MarkdownTableAlignment[] | undefined; diff --git a/dist/diagnostics-B72W0P_E.d.cts b/dist/diagnostics-BuO5-SW1.d.cts similarity index 77% rename from dist/diagnostics-B72W0P_E.d.cts rename to dist/diagnostics-BuO5-SW1.d.cts index 377d5e9..69e6c26 100644 --- a/dist/diagnostics-B72W0P_E.d.cts +++ b/dist/diagnostics-BuO5-SW1.d.cts @@ -14,6 +14,7 @@ declare const MarkdownDiagnosticCodes: { readonly UNTERMINATED_HTML_BLOCK: "md/unterminated-html-block"; readonly TABLE_CELL_COUNT_MISMATCH: "md/table-cell-count-mismatch"; readonly DUPLICATE_LINK_REFERENCE: "md/duplicate-link-reference"; + readonly DUPLICATE_FOOTNOTE_DEFINITION: "md/duplicate-footnote-definition"; readonly LIST_MARKER_TYPE_CONFLICT: "md/list-marker-type-conflict"; readonly INVENTED_PAGE_GEOMETRY: "md/invented-page-geometry"; readonly NESTED_EMPHASIS_FLATTENED: "md/nested-emphasis-flattened"; @@ -28,6 +29,9 @@ declare const MarkdownDiagnosticCodes: { readonly MATH_BLOCK_PRESERVED_AS_TEXT: "md/math-block-preserved-as-text"; readonly MATH_INLINE_PRESERVED_AS_TEXT: "md/math-inline-preserved-as-text"; readonly FRONT_MATTER_KEY_UNMAPPED: "md/front-matter-key-unmapped"; + readonly FOOTNOTE_REFERENCE_PRESERVED_AS_TEXT: "md/footnote-reference-preserved-as-text"; + readonly FOOTNOTE_BODY_HEADING_FLATTENED: "md/footnote-body-heading-flattened"; + readonly CONSTRUCT_UNREPRESENTED: "md/construct-unrepresented"; readonly HEADING_LEVEL_CLAMPED: "md/heading-level-clamped"; readonly ADJACENT_LINKS_MERGED: "md/adjacent-links-merged"; readonly CODE_SPAN_AS_MONOSPACE_RUN: "md/code-span-as-monospace-run"; @@ -56,9 +60,14 @@ declare class MarkdownWriteError extends Error { readonly code: string; constructor(code: string, message: string); } +declare class MarkdownUnbalancedConstructMarkersError extends MarkdownWriteError { + readonly imbalanceKind: 'unmatchedEnd' | 'unclosedStart'; + readonly blockIndex: number; + constructor(imbalanceKind: 'unmatchedEnd' | 'unclosedStart', blockIndex: number); +} declare class MarkdownUnsupportedDocumentKindError extends MarkdownWriteError { readonly kind: string; constructor(kind: string); } //#endregion -export { MarkdownInputTooLargeError as a, MarkdownParseError as c, NOOP_MARKDOWN_DIAGNOSTIC_SINK as d, MarkdownDiagnosticSink as i, MarkdownUnsupportedDocumentKindError as l, MarkdownDiagnosticCodes as n, MarkdownInvalidUtf8Error as o, MarkdownDiagnosticSeverity as r, MarkdownNestingLimitExceededError as s, MarkdownDiagnostic as t, MarkdownWriteError as u }; \ No newline at end of file +export { MarkdownInputTooLargeError as a, MarkdownParseError as c, MarkdownWriteError as d, NOOP_MARKDOWN_DIAGNOSTIC_SINK as f, MarkdownDiagnosticSink as i, MarkdownUnbalancedConstructMarkersError as l, MarkdownDiagnosticCodes as n, MarkdownInvalidUtf8Error as o, MarkdownDiagnosticSeverity as r, MarkdownNestingLimitExceededError as s, MarkdownDiagnostic as t, MarkdownUnsupportedDocumentKindError as u }; \ No newline at end of file diff --git a/dist/diagnostics-B72W0P_E.d.ts b/dist/diagnostics-BuO5-SW1.d.ts similarity index 77% rename from dist/diagnostics-B72W0P_E.d.ts rename to dist/diagnostics-BuO5-SW1.d.ts index 377d5e9..69e6c26 100644 --- a/dist/diagnostics-B72W0P_E.d.ts +++ b/dist/diagnostics-BuO5-SW1.d.ts @@ -14,6 +14,7 @@ declare const MarkdownDiagnosticCodes: { readonly UNTERMINATED_HTML_BLOCK: "md/unterminated-html-block"; readonly TABLE_CELL_COUNT_MISMATCH: "md/table-cell-count-mismatch"; readonly DUPLICATE_LINK_REFERENCE: "md/duplicate-link-reference"; + readonly DUPLICATE_FOOTNOTE_DEFINITION: "md/duplicate-footnote-definition"; readonly LIST_MARKER_TYPE_CONFLICT: "md/list-marker-type-conflict"; readonly INVENTED_PAGE_GEOMETRY: "md/invented-page-geometry"; readonly NESTED_EMPHASIS_FLATTENED: "md/nested-emphasis-flattened"; @@ -28,6 +29,9 @@ declare const MarkdownDiagnosticCodes: { readonly MATH_BLOCK_PRESERVED_AS_TEXT: "md/math-block-preserved-as-text"; readonly MATH_INLINE_PRESERVED_AS_TEXT: "md/math-inline-preserved-as-text"; readonly FRONT_MATTER_KEY_UNMAPPED: "md/front-matter-key-unmapped"; + readonly FOOTNOTE_REFERENCE_PRESERVED_AS_TEXT: "md/footnote-reference-preserved-as-text"; + readonly FOOTNOTE_BODY_HEADING_FLATTENED: "md/footnote-body-heading-flattened"; + readonly CONSTRUCT_UNREPRESENTED: "md/construct-unrepresented"; readonly HEADING_LEVEL_CLAMPED: "md/heading-level-clamped"; readonly ADJACENT_LINKS_MERGED: "md/adjacent-links-merged"; readonly CODE_SPAN_AS_MONOSPACE_RUN: "md/code-span-as-monospace-run"; @@ -56,9 +60,14 @@ declare class MarkdownWriteError extends Error { readonly code: string; constructor(code: string, message: string); } +declare class MarkdownUnbalancedConstructMarkersError extends MarkdownWriteError { + readonly imbalanceKind: 'unmatchedEnd' | 'unclosedStart'; + readonly blockIndex: number; + constructor(imbalanceKind: 'unmatchedEnd' | 'unclosedStart', blockIndex: number); +} declare class MarkdownUnsupportedDocumentKindError extends MarkdownWriteError { readonly kind: string; constructor(kind: string); } //#endregion -export { MarkdownInputTooLargeError as a, MarkdownParseError as c, NOOP_MARKDOWN_DIAGNOSTIC_SINK as d, MarkdownDiagnosticSink as i, MarkdownUnsupportedDocumentKindError as l, MarkdownDiagnosticCodes as n, MarkdownInvalidUtf8Error as o, MarkdownDiagnosticSeverity as r, MarkdownNestingLimitExceededError as s, MarkdownDiagnostic as t, MarkdownWriteError as u }; \ No newline at end of file +export { MarkdownInputTooLargeError as a, MarkdownParseError as c, MarkdownWriteError as d, NOOP_MARKDOWN_DIAGNOSTIC_SINK as f, MarkdownDiagnosticSink as i, MarkdownUnbalancedConstructMarkersError as l, MarkdownDiagnosticCodes as n, MarkdownInvalidUtf8Error as o, MarkdownDiagnosticSeverity as r, MarkdownNestingLimitExceededError as s, MarkdownDiagnostic as t, MarkdownUnsupportedDocumentKindError as u }; \ No newline at end of file diff --git a/dist/diagnostics/diagnostics.cjs b/dist/diagnostics/diagnostics.cjs index d477ccc..9c8f841 100644 --- a/dist/diagnostics/diagnostics.cjs +++ b/dist/diagnostics/diagnostics.cjs @@ -7,6 +7,7 @@ const MarkdownDiagnosticCodes = { UNTERMINATED_HTML_BLOCK: "md/unterminated-html-block", TABLE_CELL_COUNT_MISMATCH: "md/table-cell-count-mismatch", DUPLICATE_LINK_REFERENCE: "md/duplicate-link-reference", + DUPLICATE_FOOTNOTE_DEFINITION: "md/duplicate-footnote-definition", LIST_MARKER_TYPE_CONFLICT: "md/list-marker-type-conflict", INVENTED_PAGE_GEOMETRY: "md/invented-page-geometry", NESTED_EMPHASIS_FLATTENED: "md/nested-emphasis-flattened", @@ -21,6 +22,9 @@ const MarkdownDiagnosticCodes = { MATH_BLOCK_PRESERVED_AS_TEXT: "md/math-block-preserved-as-text", MATH_INLINE_PRESERVED_AS_TEXT: "md/math-inline-preserved-as-text", FRONT_MATTER_KEY_UNMAPPED: "md/front-matter-key-unmapped", + FOOTNOTE_REFERENCE_PRESERVED_AS_TEXT: "md/footnote-reference-preserved-as-text", + FOOTNOTE_BODY_HEADING_FLATTENED: "md/footnote-body-heading-flattened", + CONSTRUCT_UNREPRESENTED: "md/construct-unrepresented", HEADING_LEVEL_CLAMPED: "md/heading-level-clamped", ADJACENT_LINKS_MERGED: "md/adjacent-links-merged", CODE_SPAN_AS_MONOSPACE_RUN: "md/code-span-as-monospace-run", @@ -69,6 +73,16 @@ var MarkdownWriteError = class extends Error { this.code = code; } }; +var MarkdownUnbalancedConstructMarkersError = class extends MarkdownWriteError { + imbalanceKind; + blockIndex; + constructor(imbalanceKind, blockIndex) { + super("md/unbalanced-construct-markers", `${imbalanceKind === "unmatchedEnd" ? "a constructEnd marker closes no open construct" : "a constructStart marker is never closed"} at block index ${String(blockIndex)}; a block list's construct boundary markers must pair as balanced brackets`); + this.name = "MarkdownUnbalancedConstructMarkersError"; + this.imbalanceKind = imbalanceKind; + this.blockIndex = blockIndex; + } +}; var MarkdownUnsupportedDocumentKindError = class extends MarkdownWriteError { kind; constructor(kind) { @@ -83,6 +97,7 @@ exports.MarkdownInputTooLargeError = MarkdownInputTooLargeError; exports.MarkdownInvalidUtf8Error = MarkdownInvalidUtf8Error; exports.MarkdownNestingLimitExceededError = MarkdownNestingLimitExceededError; exports.MarkdownParseError = MarkdownParseError; +exports.MarkdownUnbalancedConstructMarkersError = MarkdownUnbalancedConstructMarkersError; exports.MarkdownUnsupportedDocumentKindError = MarkdownUnsupportedDocumentKindError; exports.MarkdownWriteError = MarkdownWriteError; exports.NOOP_MARKDOWN_DIAGNOSTIC_SINK = NOOP_MARKDOWN_DIAGNOSTIC_SINK; diff --git a/dist/diagnostics/diagnostics.d.cts b/dist/diagnostics/diagnostics.d.cts index 8c2dd8b..798854c 100644 --- a/dist/diagnostics/diagnostics.d.cts +++ b/dist/diagnostics/diagnostics.d.cts @@ -1,2 +1,2 @@ -import { a as MarkdownInputTooLargeError, c as MarkdownParseError, d as NOOP_MARKDOWN_DIAGNOSTIC_SINK, i as MarkdownDiagnosticSink, l as MarkdownUnsupportedDocumentKindError, n as MarkdownDiagnosticCodes, o as MarkdownInvalidUtf8Error, r as MarkdownDiagnosticSeverity, s as MarkdownNestingLimitExceededError, t as MarkdownDiagnostic, u as MarkdownWriteError } from "../diagnostics-B72W0P_E.cjs"; -export { MarkdownDiagnostic, MarkdownDiagnosticCodes, MarkdownDiagnosticSeverity, MarkdownDiagnosticSink, MarkdownInputTooLargeError, MarkdownInvalidUtf8Error, MarkdownNestingLimitExceededError, MarkdownParseError, MarkdownUnsupportedDocumentKindError, MarkdownWriteError, NOOP_MARKDOWN_DIAGNOSTIC_SINK }; \ No newline at end of file +import { a as MarkdownInputTooLargeError, c as MarkdownParseError, d as MarkdownWriteError, f as NOOP_MARKDOWN_DIAGNOSTIC_SINK, i as MarkdownDiagnosticSink, l as MarkdownUnbalancedConstructMarkersError, n as MarkdownDiagnosticCodes, o as MarkdownInvalidUtf8Error, r as MarkdownDiagnosticSeverity, s as MarkdownNestingLimitExceededError, t as MarkdownDiagnostic, u as MarkdownUnsupportedDocumentKindError } from "../diagnostics-BuO5-SW1.cjs"; +export { MarkdownDiagnostic, MarkdownDiagnosticCodes, MarkdownDiagnosticSeverity, MarkdownDiagnosticSink, MarkdownInputTooLargeError, MarkdownInvalidUtf8Error, MarkdownNestingLimitExceededError, MarkdownParseError, MarkdownUnbalancedConstructMarkersError, MarkdownUnsupportedDocumentKindError, MarkdownWriteError, NOOP_MARKDOWN_DIAGNOSTIC_SINK }; \ No newline at end of file diff --git a/dist/diagnostics/diagnostics.d.ts b/dist/diagnostics/diagnostics.d.ts index 3b7ce16..395245d 100644 --- a/dist/diagnostics/diagnostics.d.ts +++ b/dist/diagnostics/diagnostics.d.ts @@ -1,2 +1,2 @@ -import { a as MarkdownInputTooLargeError, c as MarkdownParseError, d as NOOP_MARKDOWN_DIAGNOSTIC_SINK, i as MarkdownDiagnosticSink, l as MarkdownUnsupportedDocumentKindError, n as MarkdownDiagnosticCodes, o as MarkdownInvalidUtf8Error, r as MarkdownDiagnosticSeverity, s as MarkdownNestingLimitExceededError, t as MarkdownDiagnostic, u as MarkdownWriteError } from "../diagnostics-B72W0P_E.js"; -export { MarkdownDiagnostic, MarkdownDiagnosticCodes, MarkdownDiagnosticSeverity, MarkdownDiagnosticSink, MarkdownInputTooLargeError, MarkdownInvalidUtf8Error, MarkdownNestingLimitExceededError, MarkdownParseError, MarkdownUnsupportedDocumentKindError, MarkdownWriteError, NOOP_MARKDOWN_DIAGNOSTIC_SINK }; \ No newline at end of file +import { a as MarkdownInputTooLargeError, c as MarkdownParseError, d as MarkdownWriteError, f as NOOP_MARKDOWN_DIAGNOSTIC_SINK, i as MarkdownDiagnosticSink, l as MarkdownUnbalancedConstructMarkersError, n as MarkdownDiagnosticCodes, o as MarkdownInvalidUtf8Error, r as MarkdownDiagnosticSeverity, s as MarkdownNestingLimitExceededError, t as MarkdownDiagnostic, u as MarkdownUnsupportedDocumentKindError } from "../diagnostics-BuO5-SW1.js"; +export { MarkdownDiagnostic, MarkdownDiagnosticCodes, MarkdownDiagnosticSeverity, MarkdownDiagnosticSink, MarkdownInputTooLargeError, MarkdownInvalidUtf8Error, MarkdownNestingLimitExceededError, MarkdownParseError, MarkdownUnbalancedConstructMarkersError, MarkdownUnsupportedDocumentKindError, MarkdownWriteError, NOOP_MARKDOWN_DIAGNOSTIC_SINK }; \ No newline at end of file diff --git a/dist/diagnostics/diagnostics.js b/dist/diagnostics/diagnostics.js index dbcabf3..5a8cb19 100644 --- a/dist/diagnostics/diagnostics.js +++ b/dist/diagnostics/diagnostics.js @@ -6,6 +6,7 @@ const MarkdownDiagnosticCodes = { UNTERMINATED_HTML_BLOCK: "md/unterminated-html-block", TABLE_CELL_COUNT_MISMATCH: "md/table-cell-count-mismatch", DUPLICATE_LINK_REFERENCE: "md/duplicate-link-reference", + DUPLICATE_FOOTNOTE_DEFINITION: "md/duplicate-footnote-definition", LIST_MARKER_TYPE_CONFLICT: "md/list-marker-type-conflict", INVENTED_PAGE_GEOMETRY: "md/invented-page-geometry", NESTED_EMPHASIS_FLATTENED: "md/nested-emphasis-flattened", @@ -20,6 +21,9 @@ const MarkdownDiagnosticCodes = { MATH_BLOCK_PRESERVED_AS_TEXT: "md/math-block-preserved-as-text", MATH_INLINE_PRESERVED_AS_TEXT: "md/math-inline-preserved-as-text", FRONT_MATTER_KEY_UNMAPPED: "md/front-matter-key-unmapped", + FOOTNOTE_REFERENCE_PRESERVED_AS_TEXT: "md/footnote-reference-preserved-as-text", + FOOTNOTE_BODY_HEADING_FLATTENED: "md/footnote-body-heading-flattened", + CONSTRUCT_UNREPRESENTED: "md/construct-unrepresented", HEADING_LEVEL_CLAMPED: "md/heading-level-clamped", ADJACENT_LINKS_MERGED: "md/adjacent-links-merged", CODE_SPAN_AS_MONOSPACE_RUN: "md/code-span-as-monospace-run", @@ -68,6 +72,16 @@ var MarkdownWriteError = class extends Error { this.code = code; } }; +var MarkdownUnbalancedConstructMarkersError = class extends MarkdownWriteError { + imbalanceKind; + blockIndex; + constructor(imbalanceKind, blockIndex) { + super("md/unbalanced-construct-markers", `${imbalanceKind === "unmatchedEnd" ? "a constructEnd marker closes no open construct" : "a constructStart marker is never closed"} at block index ${String(blockIndex)}; a block list's construct boundary markers must pair as balanced brackets`); + this.name = "MarkdownUnbalancedConstructMarkersError"; + this.imbalanceKind = imbalanceKind; + this.blockIndex = blockIndex; + } +}; var MarkdownUnsupportedDocumentKindError = class extends MarkdownWriteError { kind; constructor(kind) { @@ -77,4 +91,4 @@ var MarkdownUnsupportedDocumentKindError = class extends MarkdownWriteError { } }; //#endregion -export { MarkdownDiagnosticCodes, MarkdownInputTooLargeError, MarkdownInvalidUtf8Error, MarkdownNestingLimitExceededError, MarkdownParseError, MarkdownUnsupportedDocumentKindError, MarkdownWriteError, NOOP_MARKDOWN_DIAGNOSTIC_SINK }; +export { MarkdownDiagnosticCodes, MarkdownInputTooLargeError, MarkdownInvalidUtf8Error, MarkdownNestingLimitExceededError, MarkdownParseError, MarkdownUnbalancedConstructMarkersError, MarkdownUnsupportedDocumentKindError, MarkdownWriteError, NOOP_MARKDOWN_DIAGNOSTIC_SINK }; diff --git a/dist/emit/emit.cjs b/dist/emit/emit.cjs index c445fac..c7ba97a 100644 --- a/dist/emit/emit.cjs +++ b/dist/emit/emit.cjs @@ -1,4 +1,5 @@ Object.defineProperty(exports, Symbol.toStringTag, { value: "Module" }); +const require_inline_footnote = require("../inline/footnote.cjs"); require("../defaults/defaults.cjs"); const require_diagnostics_diagnostics = require("../diagnostics/diagnostics.cjs"); const require_shared_list_id = require("../shared/list-id.cjs"); @@ -193,29 +194,98 @@ function renderListRegion(items, context) { } return out; } -function emitBlocks(blocks, context) { - const parts = []; - let index = 0; +function isConstructItem(item) { + return "descriptor" in item; +} +function groupConstructItems(blocks, start) { + const items = []; + let index = start; while (index < blocks.length) { const block = blocks[index]; if (block === void 0) break; - if (block.kind === "paragraph" && block.list !== void 0) { + index += 1; + if (block.kind === "constructEnd") return { + items, + next: index + }; + if (block.kind === "constructStart") { + const nested = groupConstructItems(blocks, index); + items.push({ + descriptor: block.descriptor, + children: nested.items + }); + index = nested.next; + continue; + } + items.push({ block }); + } + return { + items, + next: index + }; +} +const FOOTNOTE_CONTINUATION_INDENT = 4; +function renderFootnoteDefinition(name, body) { + const marker = `[^${name}]:`; + if (body.length === 0) return marker; + const indent = " ".repeat(FOOTNOTE_CONTINUATION_INDENT); + const [firstLine = "", ...restLines] = body.split("\n"); + return [`${marker} ${firstLine}`, ...restLines.map((line) => line.length === 0 ? line : `${indent}${line}`)].join("\n"); +} +function renderConstruct(item, context) { + const body = renderItems(item.children, context); + const { descriptor } = item; + if (descriptor.kind === "anchor" && descriptor.anchorType === "footnote") { + if (require_inline_footnote.isValidFootnoteLabel(descriptor.name)) return renderFootnoteDefinition(descriptor.name, body); + context.sink({ + code: require_diagnostics_diagnostics.MarkdownDiagnosticCodes.CONSTRUCT_UNREPRESENTED, + severity: "info", + message: `a footnote anchor's own name "${descriptor.name}" cannot be spelled as a "[^label]:" marker (whitespace or "]" would reparse as something else); its own extent still renders in place, but the construct itself is not represented` + }); + return body; + } + const detail = descriptor.kind === "anchor" ? `${descriptor.kind} (${descriptor.anchorType})` : descriptor.kind; + context.sink({ + code: require_diagnostics_diagnostics.MarkdownDiagnosticCodes.CONSTRUCT_UNREPRESENTED, + severity: "info", + message: `a "${detail}" construct has no markdown syntax; its own extent still renders in place, but the construct itself is not represented` + }); + return body; +} +function renderItems(items, context) { + const parts = []; + let index = 0; + while (index < items.length) { + const item = items[index]; + if (item === void 0) break; + if (isConstructItem(item)) { + const rendered = renderConstruct(item, context); + if (rendered.length > 0) parts.push(rendered); + index += 1; + continue; + } + if (item.block.kind === "paragraph" && item.block.list !== void 0) { const region = []; let end = index; - for (let candidate = blocks[end]; candidate?.kind === "paragraph" && candidate.list !== void 0; candidate = blocks[end]) { - region.push(candidate); + for (let candidate = items[end]; candidate !== void 0 && !isConstructItem(candidate) && candidate.block.kind === "paragraph" && candidate.block.list !== void 0; candidate = items[end]) { + region.push(candidate.block); end += 1; } parts.push(renderListRegion(region, context)); index = end; continue; } - const rendered = renderTopLevelBlock(block, context); + const rendered = renderTopLevelBlock(item.block, context); if (rendered.length > 0) parts.push(rendered); index += 1; } return parts.join("\n\n"); } +function emitBlocks(blocks, context) { + const imbalance = (0, document_schema_js.findConstructMarkerImbalance)(blocks); + if (imbalance !== void 0) throw new require_diagnostics_diagnostics.MarkdownUnbalancedConstructMarkersError(imbalance.kind, imbalance.index); + return renderItems(groupConstructItems(blocks, 0).items, context); +} function emitMarkdown(document, options = {}) { if (document.kind !== "wordprocessing") throw new require_diagnostics_diagnostics.MarkdownUnsupportedDocumentKindError(document.kind); const context = { diff --git a/dist/emit/emit.js b/dist/emit/emit.js index ab23cf5..b220418 100644 --- a/dist/emit/emit.js +++ b/dist/emit/emit.js @@ -1,12 +1,13 @@ +import { isValidFootnoteLabel } from "../inline/footnote.js"; import "../defaults/defaults.js"; -import { MarkdownDiagnosticCodes, MarkdownUnsupportedDocumentKindError, NOOP_MARKDOWN_DIAGNOSTIC_SINK } from "../diagnostics/diagnostics.js"; +import { MarkdownDiagnosticCodes, MarkdownUnbalancedConstructMarkersError, MarkdownUnsupportedDocumentKindError, NOOP_MARKDOWN_DIAGNOSTIC_SINK } from "../diagnostics/diagnostics.js"; import { parseListNumId } from "../shared/list-id.js"; import { CODE_BLOCK_STYLE_ID, HORIZONTAL_RULE_STYLE_ID, HTML_PREFORMATTED_STYLE_ID, MATH_BLOCK_STYLE_ID, QUOTE_STYLE_ID, parseHeadingStyleId } from "../shared/style-constants.js"; import { emitFrontMatter } from "./front-matter.js"; import { emitRuns } from "./inline.js"; import { emitImage } from "./image.js"; import { emitTable } from "./table.js"; -import { clampHeadingLevel } from "document-schema.js"; +import { clampHeadingLevel, findConstructMarkerImbalance } from "document-schema.js"; //#region src/emit/emit.ts const MAX_SETEXT_LEVEL = 2; const SETEXT_LEVEL_1_CHAR = "="; @@ -192,29 +193,98 @@ function renderListRegion(items, context) { } return out; } -function emitBlocks(blocks, context) { - const parts = []; - let index = 0; +function isConstructItem(item) { + return "descriptor" in item; +} +function groupConstructItems(blocks, start) { + const items = []; + let index = start; while (index < blocks.length) { const block = blocks[index]; if (block === void 0) break; - if (block.kind === "paragraph" && block.list !== void 0) { + index += 1; + if (block.kind === "constructEnd") return { + items, + next: index + }; + if (block.kind === "constructStart") { + const nested = groupConstructItems(blocks, index); + items.push({ + descriptor: block.descriptor, + children: nested.items + }); + index = nested.next; + continue; + } + items.push({ block }); + } + return { + items, + next: index + }; +} +const FOOTNOTE_CONTINUATION_INDENT = 4; +function renderFootnoteDefinition(name, body) { + const marker = `[^${name}]:`; + if (body.length === 0) return marker; + const indent = " ".repeat(FOOTNOTE_CONTINUATION_INDENT); + const [firstLine = "", ...restLines] = body.split("\n"); + return [`${marker} ${firstLine}`, ...restLines.map((line) => line.length === 0 ? line : `${indent}${line}`)].join("\n"); +} +function renderConstruct(item, context) { + const body = renderItems(item.children, context); + const { descriptor } = item; + if (descriptor.kind === "anchor" && descriptor.anchorType === "footnote") { + if (isValidFootnoteLabel(descriptor.name)) return renderFootnoteDefinition(descriptor.name, body); + context.sink({ + code: MarkdownDiagnosticCodes.CONSTRUCT_UNREPRESENTED, + severity: "info", + message: `a footnote anchor's own name "${descriptor.name}" cannot be spelled as a "[^label]:" marker (whitespace or "]" would reparse as something else); its own extent still renders in place, but the construct itself is not represented` + }); + return body; + } + const detail = descriptor.kind === "anchor" ? `${descriptor.kind} (${descriptor.anchorType})` : descriptor.kind; + context.sink({ + code: MarkdownDiagnosticCodes.CONSTRUCT_UNREPRESENTED, + severity: "info", + message: `a "${detail}" construct has no markdown syntax; its own extent still renders in place, but the construct itself is not represented` + }); + return body; +} +function renderItems(items, context) { + const parts = []; + let index = 0; + while (index < items.length) { + const item = items[index]; + if (item === void 0) break; + if (isConstructItem(item)) { + const rendered = renderConstruct(item, context); + if (rendered.length > 0) parts.push(rendered); + index += 1; + continue; + } + if (item.block.kind === "paragraph" && item.block.list !== void 0) { const region = []; let end = index; - for (let candidate = blocks[end]; candidate?.kind === "paragraph" && candidate.list !== void 0; candidate = blocks[end]) { - region.push(candidate); + for (let candidate = items[end]; candidate !== void 0 && !isConstructItem(candidate) && candidate.block.kind === "paragraph" && candidate.block.list !== void 0; candidate = items[end]) { + region.push(candidate.block); end += 1; } parts.push(renderListRegion(region, context)); index = end; continue; } - const rendered = renderTopLevelBlock(block, context); + const rendered = renderTopLevelBlock(item.block, context); if (rendered.length > 0) parts.push(rendered); index += 1; } return parts.join("\n\n"); } +function emitBlocks(blocks, context) { + const imbalance = findConstructMarkerImbalance(blocks); + if (imbalance !== void 0) throw new MarkdownUnbalancedConstructMarkersError(imbalance.kind, imbalance.index); + return renderItems(groupConstructItems(blocks, 0).items, context); +} function emitMarkdown(document, options = {}) { if (document.kind !== "wordprocessing") throw new MarkdownUnsupportedDocumentKindError(document.kind); const context = { diff --git a/dist/emit/inline.cjs b/dist/emit/inline.cjs index ed8d29e..d8dbc44 100644 --- a/dist/emit/inline.cjs +++ b/dist/emit/inline.cjs @@ -85,6 +85,7 @@ function renderLeaf(run, context) { }); return renderCodeSpan(run.text); } + if (run.fontFamily === "Footnote Reference") return run.text; if (run.fontFamily === "Cambria Math") return `\\(${run.text}\\)`; return escapeMarkdownText(run.text); } @@ -135,7 +136,7 @@ function renderNestedStyles(runs, depth, context) { return out; } function isPlainAutolink(run) { - if (run.hyperlink === void 0 || run.hyperlink.length === 0 || run.bold === true || run.italic === true || run.strike === true || run.fontFamily === "Courier New" || run.fontFamily === "Cambria Math") return false; + if (run.hyperlink === void 0 || run.hyperlink.length === 0 || run.bold === true || run.italic === true || run.strike === true || run.fontFamily === "Courier New" || run.fontFamily === "Cambria Math" || run.fontFamily === "Footnote Reference") return false; return run.text === run.hyperlink || run.hyperlink === `mailto:${run.text}`; } function escapeLinkDestination(destination) { diff --git a/dist/emit/inline.d.cts b/dist/emit/inline.d.cts index 70eb62c..79a4519 100644 --- a/dist/emit/inline.d.cts +++ b/dist/emit/inline.d.cts @@ -1,4 +1,4 @@ -import { i as MarkdownDiagnosticSink } from "../diagnostics-B72W0P_E.cjs"; +import { i as MarkdownDiagnosticSink } from "../diagnostics-BuO5-SW1.cjs"; import { ContentRun } from "document-schema.js"; //#region src/emit/inline.d.ts interface InlineEmitContext { diff --git a/dist/emit/inline.d.ts b/dist/emit/inline.d.ts index 1e80a71..246da09 100644 --- a/dist/emit/inline.d.ts +++ b/dist/emit/inline.d.ts @@ -1,4 +1,4 @@ -import { i as MarkdownDiagnosticSink } from "../diagnostics-B72W0P_E.js"; +import { i as MarkdownDiagnosticSink } from "../diagnostics-BuO5-SW1.js"; import { ContentRun } from "document-schema.js"; //#region src/emit/inline.d.ts interface InlineEmitContext { diff --git a/dist/emit/inline.js b/dist/emit/inline.js index 197e596..f57322c 100644 --- a/dist/emit/inline.js +++ b/dist/emit/inline.js @@ -84,6 +84,7 @@ function renderLeaf(run, context) { }); return renderCodeSpan(run.text); } + if (run.fontFamily === "Footnote Reference") return run.text; if (run.fontFamily === "Cambria Math") return `\\(${run.text}\\)`; return escapeMarkdownText(run.text); } @@ -134,7 +135,7 @@ function renderNestedStyles(runs, depth, context) { return out; } function isPlainAutolink(run) { - if (run.hyperlink === void 0 || run.hyperlink.length === 0 || run.bold === true || run.italic === true || run.strike === true || run.fontFamily === "Courier New" || run.fontFamily === "Cambria Math") return false; + if (run.hyperlink === void 0 || run.hyperlink.length === 0 || run.bold === true || run.italic === true || run.strike === true || run.fontFamily === "Courier New" || run.fontFamily === "Cambria Math" || run.fontFamily === "Footnote Reference") return false; return run.text === run.hyperlink || run.hyperlink === `mailto:${run.text}`; } function escapeLinkDestination(destination) { diff --git a/dist/emit/table.d.cts b/dist/emit/table.d.cts index 92bf106..870d31f 100644 --- a/dist/emit/table.d.cts +++ b/dist/emit/table.d.cts @@ -1,4 +1,4 @@ -import { i as MarkdownDiagnosticSink } from "../diagnostics-B72W0P_E.cjs"; +import { i as MarkdownDiagnosticSink } from "../diagnostics-BuO5-SW1.cjs"; import { InlineEmitContext } from "./inline.cjs"; import { ContentTable } from "document-schema.js"; //#region src/emit/table.d.ts diff --git a/dist/emit/table.d.ts b/dist/emit/table.d.ts index f55e39c..783c592 100644 --- a/dist/emit/table.d.ts +++ b/dist/emit/table.d.ts @@ -1,4 +1,4 @@ -import { i as MarkdownDiagnosticSink } from "../diagnostics-B72W0P_E.js"; +import { i as MarkdownDiagnosticSink } from "../diagnostics-BuO5-SW1.js"; import { InlineEmitContext } from "./inline.js"; import { ContentTable } from "document-schema.js"; //#region src/emit/table.d.ts diff --git a/dist/footnote-CKk4JbLk.d.cts b/dist/footnote-CKk4JbLk.d.cts new file mode 100644 index 0000000..b698f29 --- /dev/null +++ b/dist/footnote-CKk4JbLk.d.cts @@ -0,0 +1,15 @@ +//#region src/inline/footnote.d.ts +interface FootnoteLabelMatch { + readonly label: string; + readonly end: number; +} +declare function matchFootnoteLabel(text: string, start: number): FootnoteLabelMatch | undefined; +interface FootnoteDefinitionMatch { + readonly label: string; + readonly markerLength: number; +} +declare function matchFootnoteDefinitionMarker(lineText: string): FootnoteDefinitionMatch | undefined; +type FootnoteLabelSet = ReadonlySet; +declare function isValidFootnoteLabel(label: string): boolean; +//#endregion +export { matchFootnoteDefinitionMarker as a, isValidFootnoteLabel as i, FootnoteLabelMatch as n, matchFootnoteLabel as o, FootnoteLabelSet as r, FootnoteDefinitionMatch as t }; \ No newline at end of file diff --git a/dist/footnote-CKk4JbLk.d.ts b/dist/footnote-CKk4JbLk.d.ts new file mode 100644 index 0000000..b698f29 --- /dev/null +++ b/dist/footnote-CKk4JbLk.d.ts @@ -0,0 +1,15 @@ +//#region src/inline/footnote.d.ts +interface FootnoteLabelMatch { + readonly label: string; + readonly end: number; +} +declare function matchFootnoteLabel(text: string, start: number): FootnoteLabelMatch | undefined; +interface FootnoteDefinitionMatch { + readonly label: string; + readonly markerLength: number; +} +declare function matchFootnoteDefinitionMarker(lineText: string): FootnoteDefinitionMatch | undefined; +type FootnoteLabelSet = ReadonlySet; +declare function isValidFootnoteLabel(label: string): boolean; +//#endregion +export { matchFootnoteDefinitionMarker as a, isValidFootnoteLabel as i, FootnoteLabelMatch as n, matchFootnoteLabel as o, FootnoteLabelSet as r, FootnoteDefinitionMatch as t }; \ No newline at end of file diff --git a/dist/html/render.cjs b/dist/html/render.cjs index 95202f4..48af7e4 100644 --- a/dist/html/render.cjs +++ b/dist/html/render.cjs @@ -79,6 +79,7 @@ function renderInline(node) { case "rawHtml": return node.literal; case "hardBreak": return "
\n"; case "softBreak": return "\n"; + case "footnoteReference": return escapeHtml(`[^${node.label}]`); case "mathInline": return `\\(${escapeHtml(node.literal)}\\)`; } } @@ -140,6 +141,12 @@ var HtmlRenderer = class { this.out += `$$\n${escapeHtml(node.literal)}\n$$\n`; this.cr(); return; + case "footnoteDefinition": + this.cr(); + this.out += `${escapeHtml(`[^${node.label}]:`)}\n`; + this.render(node.children, false); + this.cr(); + return; case "document": case "listItem": case "tableRow": diff --git a/dist/html/render.d.cts b/dist/html/render.d.cts index a0b7dcc..06916f4 100644 --- a/dist/html/render.d.cts +++ b/dist/html/render.d.cts @@ -1,4 +1,4 @@ -import { g as MarkdownInlineNode, s as MarkdownDocumentNode } from "../ast-DbjiuYr8.cjs"; +import { s as MarkdownDocumentNode, v as MarkdownInlineNode } from "../ast-8XCbjRQT.cjs"; //#region src/html/render.d.ts declare function escapeHtml(text: string): string; declare function escapeHref(href: string): string; diff --git a/dist/html/render.d.ts b/dist/html/render.d.ts index d00d889..ef799e4 100644 --- a/dist/html/render.d.ts +++ b/dist/html/render.d.ts @@ -1,4 +1,4 @@ -import { g as MarkdownInlineNode, s as MarkdownDocumentNode } from "../ast-DbjiuYr8.js"; +import { s as MarkdownDocumentNode, v as MarkdownInlineNode } from "../ast-8XCbjRQT.js"; //#region src/html/render.d.ts declare function escapeHtml(text: string): string; declare function escapeHref(href: string): string; diff --git a/dist/html/render.js b/dist/html/render.js index 0efc238..6854bf2 100644 --- a/dist/html/render.js +++ b/dist/html/render.js @@ -78,6 +78,7 @@ function renderInline(node) { case "rawHtml": return node.literal; case "hardBreak": return "
\n"; case "softBreak": return "\n"; + case "footnoteReference": return escapeHtml(`[^${node.label}]`); case "mathInline": return `\\(${escapeHtml(node.literal)}\\)`; } } @@ -139,6 +140,12 @@ var HtmlRenderer = class { this.out += `$$\n${escapeHtml(node.literal)}\n$$\n`; this.cr(); return; + case "footnoteDefinition": + this.cr(); + this.out += `${escapeHtml(`[^${node.label}]:`)}\n`; + this.render(node.children, false); + this.cr(); + return; case "document": case "listItem": case "tableRow": diff --git a/dist/index.cjs b/dist/index.cjs index c473d77..18396e4 100644 --- a/dist/index.cjs +++ b/dist/index.cjs @@ -6,6 +6,7 @@ const require_read = require("./read.cjs"); const require_write = require("./write.cjs"); const require_codec = require("./codec.cjs"); exports.CODE_BLOCK_STYLE_ID = require_shared_style_constants.CODE_BLOCK_STYLE_ID; +exports.FOOTNOTE_REFERENCE_FONT_MARKER = require_shared_style_constants.FOOTNOTE_REFERENCE_FONT_MARKER; exports.HORIZONTAL_RULE_STYLE_ID = require_shared_style_constants.HORIZONTAL_RULE_STYLE_ID; exports.HTML_PREFORMATTED_STYLE_ID = require_shared_style_constants.HTML_PREFORMATTED_STYLE_ID; exports.MAX_HEADING_STYLE_LEVEL = require_shared_style_constants.MAX_HEADING_STYLE_LEVEL; @@ -16,6 +17,7 @@ exports.MarkdownInputTooLargeError = require_diagnostics_diagnostics.MarkdownInp exports.MarkdownInvalidUtf8Error = require_diagnostics_diagnostics.MarkdownInvalidUtf8Error; exports.MarkdownNestingLimitExceededError = require_diagnostics_diagnostics.MarkdownNestingLimitExceededError; exports.MarkdownParseError = require_diagnostics_diagnostics.MarkdownParseError; +exports.MarkdownUnbalancedConstructMarkersError = require_diagnostics_diagnostics.MarkdownUnbalancedConstructMarkersError; exports.MarkdownUnsupportedDocumentKindError = require_diagnostics_diagnostics.MarkdownUnsupportedDocumentKindError; exports.MarkdownWriteError = require_diagnostics_diagnostics.MarkdownWriteError; exports.NOOP_MARKDOWN_DIAGNOSTIC_SINK = require_diagnostics_diagnostics.NOOP_MARKDOWN_DIAGNOSTIC_SINK; diff --git a/dist/index.d.cts b/dist/index.d.cts index 962b449..fa3dc42 100644 --- a/dist/index.d.cts +++ b/dist/index.d.cts @@ -1,9 +1,9 @@ -import { a as MarkdownInputTooLargeError, c as MarkdownParseError, d as NOOP_MARKDOWN_DIAGNOSTIC_SINK, i as MarkdownDiagnosticSink, l as MarkdownUnsupportedDocumentKindError, n as MarkdownDiagnosticCodes, o as MarkdownInvalidUtf8Error, r as MarkdownDiagnosticSeverity, s as MarkdownNestingLimitExceededError, t as MarkdownDiagnostic, u as MarkdownWriteError } from "./diagnostics-B72W0P_E.cjs"; +import { a as MarkdownInputTooLargeError, c as MarkdownParseError, d as MarkdownWriteError, f as NOOP_MARKDOWN_DIAGNOSTIC_SINK, i as MarkdownDiagnosticSink, l as MarkdownUnbalancedConstructMarkersError, n as MarkdownDiagnosticCodes, o as MarkdownInvalidUtf8Error, r as MarkdownDiagnosticSeverity, s as MarkdownNestingLimitExceededError, t as MarkdownDiagnostic, u as MarkdownUnsupportedDocumentKindError } from "./diagnostics-BuO5-SW1.cjs"; import { MarkdownBytesSchema, markdownCodec } from "./codec.cjs"; import { n as MarkdownImageResolver, r as MarkdownResolvedImageBytes, t as MarkdownImageResolveContext } from "./image-C4KYmz_L.cjs"; import { MarkdownBulletListMarker, MarkdownCodeFenceChar, MarkdownEmphasisMarker, MarkdownHeadingStyle, MarkdownLineEnding, MarkdownOrderedListDelimiter, MarkdownThematicBreakChar, ReadMarkdownOptions, WriteMarkdownOptions, WriteMarkdownStyleOptions } from "./options/options.cjs"; import { ReadMarkdownResult, readMarkdown } from "./read.cjs"; import { writeMarkdown } from "./write.cjs"; import { ListNumIdInfo, ListNumIdMintOptions, NumIdMintState, createNumIdMintState, mintListNumId, mintedListType, parseListNumId } from "./shared/list-id.cjs"; -import { CODE_BLOCK_STYLE_ID, HORIZONTAL_RULE_STYLE_ID, HTML_PREFORMATTED_STYLE_ID, MAX_HEADING_STYLE_LEVEL, MONOSPACE_FONT_FAMILY, QUOTE_INDENT_PT, QUOTE_STYLE_ID, headingStyleId, parseHeadingStyleId } from "./shared/style-constants.cjs"; -export { CODE_BLOCK_STYLE_ID, HORIZONTAL_RULE_STYLE_ID, HTML_PREFORMATTED_STYLE_ID, type ListNumIdInfo, type ListNumIdMintOptions, MAX_HEADING_STYLE_LEVEL, MONOSPACE_FONT_FAMILY, type MarkdownBulletListMarker, MarkdownBytesSchema, type MarkdownCodeFenceChar, type MarkdownDiagnostic, MarkdownDiagnosticCodes, type MarkdownDiagnosticSeverity, type MarkdownDiagnosticSink, type MarkdownEmphasisMarker, type MarkdownHeadingStyle, type MarkdownImageResolveContext, type MarkdownImageResolver, MarkdownInputTooLargeError, MarkdownInvalidUtf8Error, type MarkdownLineEnding, MarkdownNestingLimitExceededError, type MarkdownOrderedListDelimiter, MarkdownParseError, type MarkdownResolvedImageBytes, type MarkdownThematicBreakChar, MarkdownUnsupportedDocumentKindError, MarkdownWriteError, NOOP_MARKDOWN_DIAGNOSTIC_SINK, type NumIdMintState, QUOTE_INDENT_PT, QUOTE_STYLE_ID, type ReadMarkdownOptions, type ReadMarkdownResult, type WriteMarkdownOptions, type WriteMarkdownStyleOptions, createNumIdMintState, headingStyleId, markdownCodec, mintListNumId, mintedListType, parseHeadingStyleId, parseListNumId, readMarkdown, writeMarkdown }; \ No newline at end of file +import { CODE_BLOCK_STYLE_ID, FOOTNOTE_REFERENCE_FONT_MARKER, HORIZONTAL_RULE_STYLE_ID, HTML_PREFORMATTED_STYLE_ID, MAX_HEADING_STYLE_LEVEL, MONOSPACE_FONT_FAMILY, QUOTE_INDENT_PT, QUOTE_STYLE_ID, headingStyleId, parseHeadingStyleId } from "./shared/style-constants.cjs"; +export { CODE_BLOCK_STYLE_ID, FOOTNOTE_REFERENCE_FONT_MARKER, HORIZONTAL_RULE_STYLE_ID, HTML_PREFORMATTED_STYLE_ID, type ListNumIdInfo, type ListNumIdMintOptions, MAX_HEADING_STYLE_LEVEL, MONOSPACE_FONT_FAMILY, type MarkdownBulletListMarker, MarkdownBytesSchema, type MarkdownCodeFenceChar, type MarkdownDiagnostic, MarkdownDiagnosticCodes, type MarkdownDiagnosticSeverity, type MarkdownDiagnosticSink, type MarkdownEmphasisMarker, type MarkdownHeadingStyle, type MarkdownImageResolveContext, type MarkdownImageResolver, MarkdownInputTooLargeError, MarkdownInvalidUtf8Error, type MarkdownLineEnding, MarkdownNestingLimitExceededError, type MarkdownOrderedListDelimiter, MarkdownParseError, type MarkdownResolvedImageBytes, type MarkdownThematicBreakChar, MarkdownUnbalancedConstructMarkersError, MarkdownUnsupportedDocumentKindError, MarkdownWriteError, NOOP_MARKDOWN_DIAGNOSTIC_SINK, type NumIdMintState, QUOTE_INDENT_PT, QUOTE_STYLE_ID, type ReadMarkdownOptions, type ReadMarkdownResult, type WriteMarkdownOptions, type WriteMarkdownStyleOptions, createNumIdMintState, headingStyleId, markdownCodec, mintListNumId, mintedListType, parseHeadingStyleId, parseListNumId, readMarkdown, writeMarkdown }; \ No newline at end of file diff --git a/dist/index.d.ts b/dist/index.d.ts index b0c5846..59617d1 100644 --- a/dist/index.d.ts +++ b/dist/index.d.ts @@ -1,9 +1,9 @@ -import { a as MarkdownInputTooLargeError, c as MarkdownParseError, d as NOOP_MARKDOWN_DIAGNOSTIC_SINK, i as MarkdownDiagnosticSink, l as MarkdownUnsupportedDocumentKindError, n as MarkdownDiagnosticCodes, o as MarkdownInvalidUtf8Error, r as MarkdownDiagnosticSeverity, s as MarkdownNestingLimitExceededError, t as MarkdownDiagnostic, u as MarkdownWriteError } from "./diagnostics-B72W0P_E.js"; +import { a as MarkdownInputTooLargeError, c as MarkdownParseError, d as MarkdownWriteError, f as NOOP_MARKDOWN_DIAGNOSTIC_SINK, i as MarkdownDiagnosticSink, l as MarkdownUnbalancedConstructMarkersError, n as MarkdownDiagnosticCodes, o as MarkdownInvalidUtf8Error, r as MarkdownDiagnosticSeverity, s as MarkdownNestingLimitExceededError, t as MarkdownDiagnostic, u as MarkdownUnsupportedDocumentKindError } from "./diagnostics-BuO5-SW1.js"; import { MarkdownBytesSchema, markdownCodec } from "./codec.js"; import { n as MarkdownImageResolver, r as MarkdownResolvedImageBytes, t as MarkdownImageResolveContext } from "./image-Cm3hT5PS.js"; import { MarkdownBulletListMarker, MarkdownCodeFenceChar, MarkdownEmphasisMarker, MarkdownHeadingStyle, MarkdownLineEnding, MarkdownOrderedListDelimiter, MarkdownThematicBreakChar, ReadMarkdownOptions, WriteMarkdownOptions, WriteMarkdownStyleOptions } from "./options/options.js"; import { ReadMarkdownResult, readMarkdown } from "./read.js"; import { writeMarkdown } from "./write.js"; import { ListNumIdInfo, ListNumIdMintOptions, NumIdMintState, createNumIdMintState, mintListNumId, mintedListType, parseListNumId } from "./shared/list-id.js"; -import { CODE_BLOCK_STYLE_ID, HORIZONTAL_RULE_STYLE_ID, HTML_PREFORMATTED_STYLE_ID, MAX_HEADING_STYLE_LEVEL, MONOSPACE_FONT_FAMILY, QUOTE_INDENT_PT, QUOTE_STYLE_ID, headingStyleId, parseHeadingStyleId } from "./shared/style-constants.js"; -export { CODE_BLOCK_STYLE_ID, HORIZONTAL_RULE_STYLE_ID, HTML_PREFORMATTED_STYLE_ID, type ListNumIdInfo, type ListNumIdMintOptions, MAX_HEADING_STYLE_LEVEL, MONOSPACE_FONT_FAMILY, type MarkdownBulletListMarker, MarkdownBytesSchema, type MarkdownCodeFenceChar, type MarkdownDiagnostic, MarkdownDiagnosticCodes, type MarkdownDiagnosticSeverity, type MarkdownDiagnosticSink, type MarkdownEmphasisMarker, type MarkdownHeadingStyle, type MarkdownImageResolveContext, type MarkdownImageResolver, MarkdownInputTooLargeError, MarkdownInvalidUtf8Error, type MarkdownLineEnding, MarkdownNestingLimitExceededError, type MarkdownOrderedListDelimiter, MarkdownParseError, type MarkdownResolvedImageBytes, type MarkdownThematicBreakChar, MarkdownUnsupportedDocumentKindError, MarkdownWriteError, NOOP_MARKDOWN_DIAGNOSTIC_SINK, type NumIdMintState, QUOTE_INDENT_PT, QUOTE_STYLE_ID, type ReadMarkdownOptions, type ReadMarkdownResult, type WriteMarkdownOptions, type WriteMarkdownStyleOptions, createNumIdMintState, headingStyleId, markdownCodec, mintListNumId, mintedListType, parseHeadingStyleId, parseListNumId, readMarkdown, writeMarkdown }; \ No newline at end of file +import { CODE_BLOCK_STYLE_ID, FOOTNOTE_REFERENCE_FONT_MARKER, HORIZONTAL_RULE_STYLE_ID, HTML_PREFORMATTED_STYLE_ID, MAX_HEADING_STYLE_LEVEL, MONOSPACE_FONT_FAMILY, QUOTE_INDENT_PT, QUOTE_STYLE_ID, headingStyleId, parseHeadingStyleId } from "./shared/style-constants.js"; +export { CODE_BLOCK_STYLE_ID, FOOTNOTE_REFERENCE_FONT_MARKER, HORIZONTAL_RULE_STYLE_ID, HTML_PREFORMATTED_STYLE_ID, type ListNumIdInfo, type ListNumIdMintOptions, MAX_HEADING_STYLE_LEVEL, MONOSPACE_FONT_FAMILY, type MarkdownBulletListMarker, MarkdownBytesSchema, type MarkdownCodeFenceChar, type MarkdownDiagnostic, MarkdownDiagnosticCodes, type MarkdownDiagnosticSeverity, type MarkdownDiagnosticSink, type MarkdownEmphasisMarker, type MarkdownHeadingStyle, type MarkdownImageResolveContext, type MarkdownImageResolver, MarkdownInputTooLargeError, MarkdownInvalidUtf8Error, type MarkdownLineEnding, MarkdownNestingLimitExceededError, type MarkdownOrderedListDelimiter, MarkdownParseError, type MarkdownResolvedImageBytes, type MarkdownThematicBreakChar, MarkdownUnbalancedConstructMarkersError, MarkdownUnsupportedDocumentKindError, MarkdownWriteError, NOOP_MARKDOWN_DIAGNOSTIC_SINK, type NumIdMintState, QUOTE_INDENT_PT, QUOTE_STYLE_ID, type ReadMarkdownOptions, type ReadMarkdownResult, type WriteMarkdownOptions, type WriteMarkdownStyleOptions, createNumIdMintState, headingStyleId, markdownCodec, mintListNumId, mintedListType, parseHeadingStyleId, parseListNumId, readMarkdown, writeMarkdown }; \ No newline at end of file diff --git a/dist/index.js b/dist/index.js index 2a45756..fd3290a 100644 --- a/dist/index.js +++ b/dist/index.js @@ -1,7 +1,7 @@ -import { MarkdownDiagnosticCodes, MarkdownInputTooLargeError, MarkdownInvalidUtf8Error, MarkdownNestingLimitExceededError, MarkdownParseError, MarkdownUnsupportedDocumentKindError, MarkdownWriteError, NOOP_MARKDOWN_DIAGNOSTIC_SINK } from "./diagnostics/diagnostics.js"; +import { MarkdownDiagnosticCodes, MarkdownInputTooLargeError, MarkdownInvalidUtf8Error, MarkdownNestingLimitExceededError, MarkdownParseError, MarkdownUnbalancedConstructMarkersError, MarkdownUnsupportedDocumentKindError, MarkdownWriteError, NOOP_MARKDOWN_DIAGNOSTIC_SINK } from "./diagnostics/diagnostics.js"; import { createNumIdMintState, mintListNumId, mintedListType, parseListNumId } from "./shared/list-id.js"; -import { CODE_BLOCK_STYLE_ID, HORIZONTAL_RULE_STYLE_ID, HTML_PREFORMATTED_STYLE_ID, MAX_HEADING_STYLE_LEVEL, MONOSPACE_FONT_FAMILY, QUOTE_INDENT_PT, QUOTE_STYLE_ID, headingStyleId, parseHeadingStyleId } from "./shared/style-constants.js"; +import { CODE_BLOCK_STYLE_ID, FOOTNOTE_REFERENCE_FONT_MARKER, HORIZONTAL_RULE_STYLE_ID, HTML_PREFORMATTED_STYLE_ID, MAX_HEADING_STYLE_LEVEL, MONOSPACE_FONT_FAMILY, QUOTE_INDENT_PT, QUOTE_STYLE_ID, headingStyleId, parseHeadingStyleId } from "./shared/style-constants.js"; import { readMarkdown } from "./read.js"; import { writeMarkdown } from "./write.js"; import { MarkdownBytesSchema, markdownCodec } from "./codec.js"; -export { CODE_BLOCK_STYLE_ID, HORIZONTAL_RULE_STYLE_ID, HTML_PREFORMATTED_STYLE_ID, MAX_HEADING_STYLE_LEVEL, MONOSPACE_FONT_FAMILY, MarkdownBytesSchema, MarkdownDiagnosticCodes, MarkdownInputTooLargeError, MarkdownInvalidUtf8Error, MarkdownNestingLimitExceededError, MarkdownParseError, MarkdownUnsupportedDocumentKindError, MarkdownWriteError, NOOP_MARKDOWN_DIAGNOSTIC_SINK, QUOTE_INDENT_PT, QUOTE_STYLE_ID, createNumIdMintState, headingStyleId, markdownCodec, mintListNumId, mintedListType, parseHeadingStyleId, parseListNumId, readMarkdown, writeMarkdown }; +export { CODE_BLOCK_STYLE_ID, FOOTNOTE_REFERENCE_FONT_MARKER, HORIZONTAL_RULE_STYLE_ID, HTML_PREFORMATTED_STYLE_ID, MAX_HEADING_STYLE_LEVEL, MONOSPACE_FONT_FAMILY, MarkdownBytesSchema, MarkdownDiagnosticCodes, MarkdownInputTooLargeError, MarkdownInvalidUtf8Error, MarkdownNestingLimitExceededError, MarkdownParseError, MarkdownUnbalancedConstructMarkersError, MarkdownUnsupportedDocumentKindError, MarkdownWriteError, NOOP_MARKDOWN_DIAGNOSTIC_SINK, QUOTE_INDENT_PT, QUOTE_STYLE_ID, createNumIdMintState, headingStyleId, markdownCodec, mintListNumId, mintedListType, parseHeadingStyleId, parseListNumId, readMarkdown, writeMarkdown }; diff --git a/dist/inline-TuBQ2TUr.d.ts b/dist/inline-B_V7bs5j.d.ts similarity index 60% rename from dist/inline-TuBQ2TUr.d.ts rename to dist/inline-B_V7bs5j.d.ts index 4c489f8..5e18583 100644 --- a/dist/inline-TuBQ2TUr.d.ts +++ b/dist/inline-B_V7bs5j.d.ts @@ -1,10 +1,11 @@ -import { g as MarkdownInlineNode } from "./ast-DbjiuYr8.js"; +import { v as MarkdownInlineNode } from "./ast-8XCbjRQT.js"; +import { r as FootnoteLabelSet } from "./footnote-CKk4JbLk.js"; import { n as LinkReferenceMap } from "./link-Dv4kxVjk.js"; //#region src/inline/inline.d.ts interface InlineParseOptions { readonly gfmAutolinks?: boolean; readonly gfmStrikethrough?: boolean; } -declare function parseInlines(content: string, references: LinkReferenceMap, options?: InlineParseOptions): MarkdownInlineNode[]; +declare function parseInlines(content: string, references: LinkReferenceMap, footnotes: FootnoteLabelSet, options?: InlineParseOptions): MarkdownInlineNode[]; //#endregion export { parseInlines as n, InlineParseOptions as t }; \ No newline at end of file diff --git a/dist/inline-uVHJ5xzT.d.cts b/dist/inline-CXVQWQnW.d.cts similarity index 59% rename from dist/inline-uVHJ5xzT.d.cts rename to dist/inline-CXVQWQnW.d.cts index 7cfda50..14b9162 100644 --- a/dist/inline-uVHJ5xzT.d.cts +++ b/dist/inline-CXVQWQnW.d.cts @@ -1,10 +1,11 @@ -import { g as MarkdownInlineNode } from "./ast-DbjiuYr8.cjs"; +import { v as MarkdownInlineNode } from "./ast-8XCbjRQT.cjs"; +import { r as FootnoteLabelSet } from "./footnote-CKk4JbLk.cjs"; import { n as LinkReferenceMap } from "./link-Dv4kxVjk.cjs"; //#region src/inline/inline.d.ts interface InlineParseOptions { readonly gfmAutolinks?: boolean; readonly gfmStrikethrough?: boolean; } -declare function parseInlines(content: string, references: LinkReferenceMap, options?: InlineParseOptions): MarkdownInlineNode[]; +declare function parseInlines(content: string, references: LinkReferenceMap, footnotes: FootnoteLabelSet, options?: InlineParseOptions): MarkdownInlineNode[]; //#endregion export { parseInlines as n, InlineParseOptions as t }; \ No newline at end of file diff --git a/dist/inline/footnote.cjs b/dist/inline/footnote.cjs new file mode 100644 index 0000000..724f893 --- /dev/null +++ b/dist/inline/footnote.cjs @@ -0,0 +1,30 @@ +Object.defineProperty(exports, Symbol.toStringTag, { value: "Module" }); +//#region src/inline/footnote.ts +const FOOTNOTE_LABEL_PATTERN = /\[\^([^\s[\]]+)\]/y; +const FOOTNOTE_LABEL_ONLY_PATTERN = /^[^\s[\]]+$/; +function matchFootnoteLabel(text, start) { + FOOTNOTE_LABEL_PATTERN.lastIndex = start; + const match = FOOTNOTE_LABEL_PATTERN.exec(text); + if (match === null) return; + const label = match[1]; + if (label === void 0) return; + return { + label, + end: start + match[0].length + }; +} +function matchFootnoteDefinitionMarker(lineText) { + const match = matchFootnoteLabel(lineText, 0); + if (match === void 0 || lineText.charAt(match.end) !== ":") return; + return { + label: match.label, + markerLength: match.end + 1 + }; +} +function isValidFootnoteLabel(label) { + return FOOTNOTE_LABEL_ONLY_PATTERN.test(label); +} +//#endregion +exports.isValidFootnoteLabel = isValidFootnoteLabel; +exports.matchFootnoteDefinitionMarker = matchFootnoteDefinitionMarker; +exports.matchFootnoteLabel = matchFootnoteLabel; diff --git a/dist/inline/footnote.d.cts b/dist/inline/footnote.d.cts new file mode 100644 index 0000000..156fbe5 --- /dev/null +++ b/dist/inline/footnote.d.cts @@ -0,0 +1,2 @@ +import { a as matchFootnoteDefinitionMarker, i as isValidFootnoteLabel, n as FootnoteLabelMatch, o as matchFootnoteLabel, r as FootnoteLabelSet, t as FootnoteDefinitionMatch } from "../footnote-CKk4JbLk.cjs"; +export { FootnoteDefinitionMatch, FootnoteLabelMatch, FootnoteLabelSet, isValidFootnoteLabel, matchFootnoteDefinitionMarker, matchFootnoteLabel }; \ No newline at end of file diff --git a/dist/inline/footnote.d.ts b/dist/inline/footnote.d.ts new file mode 100644 index 0000000..efe556a --- /dev/null +++ b/dist/inline/footnote.d.ts @@ -0,0 +1,2 @@ +import { a as matchFootnoteDefinitionMarker, i as isValidFootnoteLabel, n as FootnoteLabelMatch, o as matchFootnoteLabel, r as FootnoteLabelSet, t as FootnoteDefinitionMatch } from "../footnote-CKk4JbLk.js"; +export { FootnoteDefinitionMatch, FootnoteLabelMatch, FootnoteLabelSet, isValidFootnoteLabel, matchFootnoteDefinitionMarker, matchFootnoteLabel }; \ No newline at end of file diff --git a/dist/inline/footnote.js b/dist/inline/footnote.js new file mode 100644 index 0000000..38a7a5c --- /dev/null +++ b/dist/inline/footnote.js @@ -0,0 +1,27 @@ +//#region src/inline/footnote.ts +const FOOTNOTE_LABEL_PATTERN = /\[\^([^\s[\]]+)\]/y; +const FOOTNOTE_LABEL_ONLY_PATTERN = /^[^\s[\]]+$/; +function matchFootnoteLabel(text, start) { + FOOTNOTE_LABEL_PATTERN.lastIndex = start; + const match = FOOTNOTE_LABEL_PATTERN.exec(text); + if (match === null) return; + const label = match[1]; + if (label === void 0) return; + return { + label, + end: start + match[0].length + }; +} +function matchFootnoteDefinitionMarker(lineText) { + const match = matchFootnoteLabel(lineText, 0); + if (match === void 0 || lineText.charAt(match.end) !== ":") return; + return { + label: match.label, + markerLength: match.end + 1 + }; +} +function isValidFootnoteLabel(label) { + return FOOTNOTE_LABEL_ONLY_PATTERN.test(label); +} +//#endregion +export { isValidFootnoteLabel, matchFootnoteDefinitionMarker, matchFootnoteLabel }; diff --git a/dist/inline/inline.cjs b/dist/inline/inline.cjs index 862f7ea..a6c5605 100644 --- a/dist/inline/inline.cjs +++ b/dist/inline/inline.cjs @@ -1,4 +1,5 @@ Object.defineProperty(exports, Symbol.toStringTag, { value: "Module" }); +const require_inline_footnote = require("./footnote.cjs"); const require_html_html = require("../html/html.cjs"); const require_inline_chars = require("./chars.cjs"); const require_inline_entity = require("./entity.cjs"); @@ -25,14 +26,16 @@ function createWrapper(kind, marker) { var InlineParser = class { text; references; + footnotes; gfmStrikethrough; container = new require_inline_node.InlineNode("container"); delimiters = new require_inline_delimiter.DelimiterStack(); brackets; pos = 0; - constructor(text, references, options) { + constructor(text, references, footnotes, options) { this.text = text; this.references = references; + this.footnotes = footnotes; this.gfmStrikethrough = options.gfmStrikethrough ?? true; } parse() { @@ -238,9 +241,22 @@ var InlineParser = class { } parseOpenBracket() { const start = this.pos; + const footnote = this.matchFootnoteReference(); + if (footnote !== void 0) { + const node = new require_inline_node.InlineNode("footnoteReference"); + node.label = footnote.label; + this.container.appendChild(node); + this.pos = footnote.end; + return; + } this.pos += 1; this.pushBracket(this.appendText("["), start, false); } + matchFootnoteReference() { + const match = require_inline_footnote.matchFootnoteLabel(this.text, this.pos); + if (match === void 0 || !this.footnotes.has(match.label)) return; + return match; + } parseBang() { const start = this.pos; this.pos += 1; @@ -248,6 +264,10 @@ var InlineParser = class { this.appendText("!"); return; } + if (this.matchFootnoteReference() !== void 0) { + this.appendText("!"); + return; + } this.pos += 1; this.pushBracket(this.appendText("!["), start + 1, true); } @@ -363,6 +383,7 @@ function flattenToPlainText(node) { case "autolink": return node.destination; case "softBreak": case "hardBreak": return " "; + case "footnoteReference": return `[^${node.label}]`; default: { let result = ""; let child = node.firstChild; @@ -458,11 +479,15 @@ function toAstNode(node) { type: "mathInline", literal: node.literal }; + case "footnoteReference": return { + type: "footnoteReference", + label: node.label + }; case "container": return; } } -function parseInlines(content, references, options = {}) { - const root = new InlineParser(content, references, options).parse(); +function parseInlines(content, references, footnotes, options = {}) { + const root = new InlineParser(content, references, footnotes, options).parse(); if (options.gfmAutolinks ?? true) { require_inline_gfm_autolink.applyGfmAutolinks(root); mergeAdjacentText(root); diff --git a/dist/inline/inline.d.cts b/dist/inline/inline.d.cts index 8e667b1..1e0b232 100644 --- a/dist/inline/inline.d.cts +++ b/dist/inline/inline.d.cts @@ -1,2 +1,2 @@ -import { n as parseInlines, t as InlineParseOptions } from "../inline-uVHJ5xzT.cjs"; +import { n as parseInlines, t as InlineParseOptions } from "../inline-CXVQWQnW.cjs"; export { InlineParseOptions, parseInlines }; \ No newline at end of file diff --git a/dist/inline/inline.d.ts b/dist/inline/inline.d.ts index e00e8d3..e8b15f9 100644 --- a/dist/inline/inline.d.ts +++ b/dist/inline/inline.d.ts @@ -1,2 +1,2 @@ -import { n as parseInlines, t as InlineParseOptions } from "../inline-TuBQ2TUr.js"; +import { n as parseInlines, t as InlineParseOptions } from "../inline-B_V7bs5j.js"; export { InlineParseOptions, parseInlines }; \ No newline at end of file diff --git a/dist/inline/inline.js b/dist/inline/inline.js index d74e3af..f781d21 100644 --- a/dist/inline/inline.js +++ b/dist/inline/inline.js @@ -1,3 +1,4 @@ +import { matchFootnoteLabel } from "./footnote.js"; import { matchHtmlTag } from "../html/html.js"; import { containsAsciiControlOrSpace, isAsciiPunctuation } from "./chars.js"; import { matchEntity } from "./entity.js"; @@ -24,14 +25,16 @@ function createWrapper(kind, marker) { var InlineParser = class { text; references; + footnotes; gfmStrikethrough; container = new InlineNode("container"); delimiters = new DelimiterStack(); brackets; pos = 0; - constructor(text, references, options) { + constructor(text, references, footnotes, options) { this.text = text; this.references = references; + this.footnotes = footnotes; this.gfmStrikethrough = options.gfmStrikethrough ?? true; } parse() { @@ -237,9 +240,22 @@ var InlineParser = class { } parseOpenBracket() { const start = this.pos; + const footnote = this.matchFootnoteReference(); + if (footnote !== void 0) { + const node = new InlineNode("footnoteReference"); + node.label = footnote.label; + this.container.appendChild(node); + this.pos = footnote.end; + return; + } this.pos += 1; this.pushBracket(this.appendText("["), start, false); } + matchFootnoteReference() { + const match = matchFootnoteLabel(this.text, this.pos); + if (match === void 0 || !this.footnotes.has(match.label)) return; + return match; + } parseBang() { const start = this.pos; this.pos += 1; @@ -247,6 +263,10 @@ var InlineParser = class { this.appendText("!"); return; } + if (this.matchFootnoteReference() !== void 0) { + this.appendText("!"); + return; + } this.pos += 1; this.pushBracket(this.appendText("!["), start + 1, true); } @@ -362,6 +382,7 @@ function flattenToPlainText(node) { case "autolink": return node.destination; case "softBreak": case "hardBreak": return " "; + case "footnoteReference": return `[^${node.label}]`; default: { let result = ""; let child = node.firstChild; @@ -457,11 +478,15 @@ function toAstNode(node) { type: "mathInline", literal: node.literal }; + case "footnoteReference": return { + type: "footnoteReference", + label: node.label + }; case "container": return; } } -function parseInlines(content, references, options = {}) { - const root = new InlineParser(content, references, options).parse(); +function parseInlines(content, references, footnotes, options = {}) { + const root = new InlineParser(content, references, footnotes, options).parse(); if (options.gfmAutolinks ?? true) { applyGfmAutolinks(root); mergeAdjacentText(root); diff --git a/dist/inline/node.cjs b/dist/inline/node.cjs index e51ad63..23c3719 100644 --- a/dist/inline/node.cjs +++ b/dist/inline/node.cjs @@ -8,6 +8,7 @@ var InlineNode = class { email = false; marker = "*"; raw = ""; + label = ""; parent; firstChild; lastChild; diff --git a/dist/inline/node.d.cts b/dist/inline/node.d.cts index 5c99ac3..b4b6d5f 100644 --- a/dist/inline/node.d.cts +++ b/dist/inline/node.d.cts @@ -1,5 +1,5 @@ //#region src/inline/node.d.ts -type InlineNodeKind = 'text' | 'emphasis' | 'strong' | 'strikethrough' | 'codeSpan' | 'link' | 'image' | 'autolink' | 'hardBreak' | 'softBreak' | 'rawHtml' | 'entity' | 'mathInline' | 'container'; +type InlineNodeKind = 'text' | 'emphasis' | 'strong' | 'strikethrough' | 'codeSpan' | 'link' | 'image' | 'autolink' | 'hardBreak' | 'softBreak' | 'rawHtml' | 'entity' | 'mathInline' | 'footnoteReference' | 'container'; declare class InlineNode { readonly kind: InlineNodeKind; literal: string; @@ -8,6 +8,7 @@ declare class InlineNode { email: boolean; marker: '_' | '*'; raw: string; + label: string; parent: InlineNode | undefined; firstChild: InlineNode | undefined; lastChild: InlineNode | undefined; diff --git a/dist/inline/node.d.ts b/dist/inline/node.d.ts index 5c99ac3..b4b6d5f 100644 --- a/dist/inline/node.d.ts +++ b/dist/inline/node.d.ts @@ -1,5 +1,5 @@ //#region src/inline/node.d.ts -type InlineNodeKind = 'text' | 'emphasis' | 'strong' | 'strikethrough' | 'codeSpan' | 'link' | 'image' | 'autolink' | 'hardBreak' | 'softBreak' | 'rawHtml' | 'entity' | 'mathInline' | 'container'; +type InlineNodeKind = 'text' | 'emphasis' | 'strong' | 'strikethrough' | 'codeSpan' | 'link' | 'image' | 'autolink' | 'hardBreak' | 'softBreak' | 'rawHtml' | 'entity' | 'mathInline' | 'footnoteReference' | 'container'; declare class InlineNode { readonly kind: InlineNodeKind; literal: string; @@ -8,6 +8,7 @@ declare class InlineNode { email: boolean; marker: '_' | '*'; raw: string; + label: string; parent: InlineNode | undefined; firstChild: InlineNode | undefined; lastChild: InlineNode | undefined; diff --git a/dist/inline/node.js b/dist/inline/node.js index 5f0a09d..01cc0b0 100644 --- a/dist/inline/node.js +++ b/dist/inline/node.js @@ -7,6 +7,7 @@ var InlineNode = class { email = false; marker = "*"; raw = ""; + label = ""; parent; firstChild; lastChild; diff --git a/dist/lower/front-matter.d.cts b/dist/lower/front-matter.d.cts index 447f7fe..40e20d0 100644 --- a/dist/lower/front-matter.d.cts +++ b/dist/lower/front-matter.d.cts @@ -1,4 +1,4 @@ -import { i as MarkdownDiagnosticSink } from "../diagnostics-B72W0P_E.cjs"; +import { i as MarkdownDiagnosticSink } from "../diagnostics-BuO5-SW1.cjs"; import { LayoutMetadata } from "document-schema.js"; //#region src/lower/front-matter.d.ts interface FrontMatterResult { diff --git a/dist/lower/front-matter.d.ts b/dist/lower/front-matter.d.ts index 38be999..e344283 100644 --- a/dist/lower/front-matter.d.ts +++ b/dist/lower/front-matter.d.ts @@ -1,4 +1,4 @@ -import { i as MarkdownDiagnosticSink } from "../diagnostics-B72W0P_E.js"; +import { i as MarkdownDiagnosticSink } from "../diagnostics-BuO5-SW1.js"; import { LayoutMetadata } from "document-schema.js"; //#region src/lower/front-matter.d.ts interface FrontMatterResult { diff --git a/dist/lower/inline.cjs b/dist/lower/inline.cjs index 9a5d7ad..1e1292e 100644 --- a/dist/lower/inline.cjs +++ b/dist/lower/inline.cjs @@ -53,6 +53,13 @@ function lowerInlineNode(node, style, context) { message: "inline math (\\( \\)) was preserved as literal raw LaTeX text; it is not parsed as LaTeX or converted to MathML by this package" }); return [buildRun(node.literal, style, require_shared_style_constants.MATH_INLINE_FONT_MARKER)]; + case "footnoteReference": + context.sink({ + code: require_diagnostics_diagnostics.MarkdownDiagnosticCodes.FOOTNOTE_REFERENCE_PRESERVED_AS_TEXT, + severity: "info", + message: `footnote reference "[^${node.label}]" is preserved as a marked text run rather than an anchor construct: a construct's extent is block-scoped, and a reference site sits between two runs inside a paragraph, which no block-level boundary marker can bracket` + }); + return [buildRun(`[^${node.label}]`, style, require_shared_style_constants.FOOTNOTE_REFERENCE_FONT_MARKER)]; case "autolink": { const destination = node.email ? `mailto:${node.destination}` : node.destination; return [buildRun(node.destination, { diff --git a/dist/lower/inline.d.cts b/dist/lower/inline.d.cts index 7caa8ef..1218a47 100644 --- a/dist/lower/inline.d.cts +++ b/dist/lower/inline.d.cts @@ -1,5 +1,5 @@ -import { g as MarkdownInlineNode } from "../ast-DbjiuYr8.cjs"; -import { i as MarkdownDiagnosticSink } from "../diagnostics-B72W0P_E.cjs"; +import { v as MarkdownInlineNode } from "../ast-8XCbjRQT.cjs"; +import { i as MarkdownDiagnosticSink } from "../diagnostics-BuO5-SW1.cjs"; import { ContentRun } from "document-schema.js"; //#region src/lower/inline.d.ts interface InlineLowerContext { diff --git a/dist/lower/inline.d.ts b/dist/lower/inline.d.ts index ccaf816..f52c917 100644 --- a/dist/lower/inline.d.ts +++ b/dist/lower/inline.d.ts @@ -1,5 +1,5 @@ -import { g as MarkdownInlineNode } from "../ast-DbjiuYr8.js"; -import { i as MarkdownDiagnosticSink } from "../diagnostics-B72W0P_E.js"; +import { v as MarkdownInlineNode } from "../ast-8XCbjRQT.js"; +import { i as MarkdownDiagnosticSink } from "../diagnostics-BuO5-SW1.js"; import { ContentRun } from "document-schema.js"; //#region src/lower/inline.d.ts interface InlineLowerContext { diff --git a/dist/lower/inline.js b/dist/lower/inline.js index ccc1f8e..0007dec 100644 --- a/dist/lower/inline.js +++ b/dist/lower/inline.js @@ -1,5 +1,5 @@ import { MarkdownDiagnosticCodes } from "../diagnostics/diagnostics.js"; -import { MATH_INLINE_FONT_MARKER, MONOSPACE_FONT_FAMILY } from "../shared/style-constants.js"; +import { FOOTNOTE_REFERENCE_FONT_MARKER, MATH_INLINE_FONT_MARKER, MONOSPACE_FONT_FAMILY } from "../shared/style-constants.js"; //#region src/lower/inline.ts function buildRun(text, style, fontFamily) { return { @@ -52,6 +52,13 @@ function lowerInlineNode(node, style, context) { message: "inline math (\\( \\)) was preserved as literal raw LaTeX text; it is not parsed as LaTeX or converted to MathML by this package" }); return [buildRun(node.literal, style, MATH_INLINE_FONT_MARKER)]; + case "footnoteReference": + context.sink({ + code: MarkdownDiagnosticCodes.FOOTNOTE_REFERENCE_PRESERVED_AS_TEXT, + severity: "info", + message: `footnote reference "[^${node.label}]" is preserved as a marked text run rather than an anchor construct: a construct's extent is block-scoped, and a reference site sits between two runs inside a paragraph, which no block-level boundary marker can bracket` + }); + return [buildRun(`[^${node.label}]`, style, FOOTNOTE_REFERENCE_FONT_MARKER)]; case "autolink": { const destination = node.email ? `mailto:${node.destination}` : node.destination; return [buildRun(node.destination, { diff --git a/dist/lower/lower.cjs b/dist/lower/lower.cjs index 64b4608..8569608 100644 --- a/dist/lower/lower.cjs +++ b/dist/lower/lower.cjs @@ -230,6 +230,55 @@ function lowerList(node, ancestorNumId, level, context, contentWidthPt) { } return node.children.flatMap((item) => lowerListItem(item, numId, level, context, contentWidthPt)); } +function flattenFootnoteBodyHeadings(node, context) { + switch (node.type) { + case "heading": + context.sink({ + code: require_diagnostics_diagnostics.MarkdownDiagnosticCodes.FOOTNOTE_BODY_HEADING_FLATTENED, + severity: "info", + message: `a level-${String(node.level)} heading inside a footnote definition's body is carried as literal ATX text: a construct boundary marker's extent may not contain a block that opens or closes a heading scope, so the heading cannot stay a heading inside the anchor construct the definition lowers to` + }); + return { + type: "paragraph", + children: [{ + type: "text", + value: `${"#".repeat(node.level)} ` + }, ...node.children] + }; + case "blockquote": return { + type: "blockquote", + children: node.children.map((child) => flattenFootnoteBodyHeadings(child, context)) + }; + case "list": return { + ...node, + children: node.children.map((item) => flattenFootnoteBodyHeadingsInItem(item, context)) + }; + case "listItem": return flattenFootnoteBodyHeadingsInItem(node, context); + default: return node; + } +} +function flattenFootnoteBodyHeadingsInItem(item, context) { + return { + ...item, + children: item.children.map((child) => flattenFootnoteBodyHeadings(child, context)) + }; +} +function lowerFootnoteDefinition(node, context, contentWidthPt) { + const descriptor = { + kind: "anchor", + anchorType: "footnote", + name: node.label + }; + const body = node.children.flatMap((child) => lowerBlock(flattenFootnoteBodyHeadings(child, context), context, contentWidthPt)); + return [ + { + kind: "constructStart", + descriptor + }, + ...body, + { kind: "constructEnd" } + ]; +} function lowerBlock(node, context, contentWidthPt) { switch (node.type) { case "paragraph": return lowerParagraph(node, context); @@ -240,6 +289,7 @@ function lowerBlock(node, context, contentWidthPt) { case "thematicBreak": return lowerThematicBreak(context); case "htmlBlock": return lowerHtmlBlock(node, context); case "mathBlock": return lowerMathBlock(node, context); + case "footnoteDefinition": return lowerFootnoteDefinition(node, context, contentWidthPt); case "table": if (context.list !== void 0) context.sink({ code: require_diagnostics_diagnostics.MarkdownDiagnosticCodes.LIST_ITEM_BLOCK_UNLISTED, @@ -296,6 +346,7 @@ function lowerMarkdown(source, options = {}) { gfmAutolinks: options.gfmAutolinks, gfmStrikethrough: options.gfmStrikethrough, gfmTaskLists: options.gfmTaskLists, + footnotes: options.footnotes, maxNesting: options.maxBlockNesting, sink }; diff --git a/dist/lower/lower.js b/dist/lower/lower.js index ce8379a..a1fd3ea 100644 --- a/dist/lower/lower.js +++ b/dist/lower/lower.js @@ -229,6 +229,55 @@ function lowerList(node, ancestorNumId, level, context, contentWidthPt) { } return node.children.flatMap((item) => lowerListItem(item, numId, level, context, contentWidthPt)); } +function flattenFootnoteBodyHeadings(node, context) { + switch (node.type) { + case "heading": + context.sink({ + code: MarkdownDiagnosticCodes.FOOTNOTE_BODY_HEADING_FLATTENED, + severity: "info", + message: `a level-${String(node.level)} heading inside a footnote definition's body is carried as literal ATX text: a construct boundary marker's extent may not contain a block that opens or closes a heading scope, so the heading cannot stay a heading inside the anchor construct the definition lowers to` + }); + return { + type: "paragraph", + children: [{ + type: "text", + value: `${"#".repeat(node.level)} ` + }, ...node.children] + }; + case "blockquote": return { + type: "blockquote", + children: node.children.map((child) => flattenFootnoteBodyHeadings(child, context)) + }; + case "list": return { + ...node, + children: node.children.map((item) => flattenFootnoteBodyHeadingsInItem(item, context)) + }; + case "listItem": return flattenFootnoteBodyHeadingsInItem(node, context); + default: return node; + } +} +function flattenFootnoteBodyHeadingsInItem(item, context) { + return { + ...item, + children: item.children.map((child) => flattenFootnoteBodyHeadings(child, context)) + }; +} +function lowerFootnoteDefinition(node, context, contentWidthPt) { + const descriptor = { + kind: "anchor", + anchorType: "footnote", + name: node.label + }; + const body = node.children.flatMap((child) => lowerBlock(flattenFootnoteBodyHeadings(child, context), context, contentWidthPt)); + return [ + { + kind: "constructStart", + descriptor + }, + ...body, + { kind: "constructEnd" } + ]; +} function lowerBlock(node, context, contentWidthPt) { switch (node.type) { case "paragraph": return lowerParagraph(node, context); @@ -239,6 +288,7 @@ function lowerBlock(node, context, contentWidthPt) { case "thematicBreak": return lowerThematicBreak(context); case "htmlBlock": return lowerHtmlBlock(node, context); case "mathBlock": return lowerMathBlock(node, context); + case "footnoteDefinition": return lowerFootnoteDefinition(node, context, contentWidthPt); case "table": if (context.list !== void 0) context.sink({ code: MarkdownDiagnosticCodes.LIST_ITEM_BLOCK_UNLISTED, @@ -295,6 +345,7 @@ function lowerMarkdown(source, options = {}) { gfmAutolinks: options.gfmAutolinks, gfmStrikethrough: options.gfmStrikethrough, gfmTaskLists: options.gfmTaskLists, + footnotes: options.footnotes, maxNesting: options.maxBlockNesting, sink }; diff --git a/dist/lower/table.d.cts b/dist/lower/table.d.cts index 93750ab..77194dd 100644 --- a/dist/lower/table.d.cts +++ b/dist/lower/table.d.cts @@ -1,4 +1,4 @@ -import { N as MarkdownTableNode } from "../ast-DbjiuYr8.cjs"; +import { F as MarkdownTableNode } from "../ast-8XCbjRQT.cjs"; import { InlineLowerContext } from "./inline.cjs"; import { ContentTable } from "document-schema.js"; //#region src/lower/table.d.ts diff --git a/dist/lower/table.d.ts b/dist/lower/table.d.ts index b4f8fb8..9a6c580 100644 --- a/dist/lower/table.d.ts +++ b/dist/lower/table.d.ts @@ -1,4 +1,4 @@ -import { N as MarkdownTableNode } from "../ast-DbjiuYr8.js"; +import { F as MarkdownTableNode } from "../ast-8XCbjRQT.js"; import { InlineLowerContext } from "./inline.js"; import { ContentTable } from "document-schema.js"; //#region src/lower/table.d.ts diff --git a/dist/options/options.d.cts b/dist/options/options.d.cts index babfd18..eb4c11d 100644 --- a/dist/options/options.d.cts +++ b/dist/options/options.d.cts @@ -1,4 +1,4 @@ -import { i as MarkdownDiagnosticSink } from "../diagnostics-B72W0P_E.cjs"; +import { i as MarkdownDiagnosticSink } from "../diagnostics-BuO5-SW1.cjs"; import { n as MarkdownImageResolver } from "../image-C4KYmz_L.cjs"; import { Margins, PageSize } from "document-schema.js"; //#region src/options/options.d.ts @@ -14,6 +14,7 @@ interface ReadMarkdownOptions { readonly gfmAutolinks?: boolean; readonly gfmStrikethrough?: boolean; readonly gfmTaskLists?: boolean; + readonly footnotes?: boolean; readonly maxInputBytes?: number; readonly maxBlockNesting?: number; } diff --git a/dist/options/options.d.ts b/dist/options/options.d.ts index 038ad52..24979d1 100644 --- a/dist/options/options.d.ts +++ b/dist/options/options.d.ts @@ -1,4 +1,4 @@ -import { i as MarkdownDiagnosticSink } from "../diagnostics-B72W0P_E.js"; +import { i as MarkdownDiagnosticSink } from "../diagnostics-BuO5-SW1.js"; import { n as MarkdownImageResolver } from "../image-Cm3hT5PS.js"; import { Margins, PageSize } from "document-schema.js"; //#region src/options/options.d.ts @@ -14,6 +14,7 @@ interface ReadMarkdownOptions { readonly gfmAutolinks?: boolean; readonly gfmStrikethrough?: boolean; readonly gfmTaskLists?: boolean; + readonly footnotes?: boolean; readonly maxInputBytes?: number; readonly maxBlockNesting?: number; } diff --git a/dist/read.d.cts b/dist/read.d.cts index 224715e..b26cc6d 100644 --- a/dist/read.d.cts +++ b/dist/read.d.cts @@ -1,4 +1,4 @@ -import { t as MarkdownDiagnostic } from "./diagnostics-B72W0P_E.cjs"; +import { t as MarkdownDiagnostic } from "./diagnostics-BuO5-SW1.cjs"; import { ReadMarkdownOptions } from "./options/options.cjs"; import { ContentDocument } from "document-schema.js"; //#region src/read.d.ts diff --git a/dist/read.d.ts b/dist/read.d.ts index 79e4280..85e1c2b 100644 --- a/dist/read.d.ts +++ b/dist/read.d.ts @@ -1,4 +1,4 @@ -import { t as MarkdownDiagnostic } from "./diagnostics-B72W0P_E.js"; +import { t as MarkdownDiagnostic } from "./diagnostics-BuO5-SW1.js"; import { ReadMarkdownOptions } from "./options/options.js"; import { ContentDocument } from "document-schema.js"; //#region src/read.d.ts diff --git a/dist/shared/style-constants.cjs b/dist/shared/style-constants.cjs index 936a19b..cda47c8 100644 --- a/dist/shared/style-constants.cjs +++ b/dist/shared/style-constants.cjs @@ -19,11 +19,13 @@ const HTML_PREFORMATTED_STYLE_ID = "HTMLPreformatted"; const MATH_BLOCK_STYLE_ID = "MathBlock"; const MONOSPACE_FONT_FAMILY = "Courier New"; const MATH_INLINE_FONT_MARKER = "Cambria Math"; +const FOOTNOTE_REFERENCE_FONT_MARKER = "Footnote Reference"; const QUOTE_INDENT_PT = 36; const TASK_CHECKBOX_UNCHECKED = "☐"; const TASK_CHECKBOX_CHECKED = "☒"; //#endregion exports.CODE_BLOCK_STYLE_ID = CODE_BLOCK_STYLE_ID; +exports.FOOTNOTE_REFERENCE_FONT_MARKER = FOOTNOTE_REFERENCE_FONT_MARKER; exports.HORIZONTAL_RULE_STYLE_ID = HORIZONTAL_RULE_STYLE_ID; exports.HTML_PREFORMATTED_STYLE_ID = HTML_PREFORMATTED_STYLE_ID; exports.MATH_BLOCK_STYLE_ID = MATH_BLOCK_STYLE_ID; diff --git a/dist/shared/style-constants.d.cts b/dist/shared/style-constants.d.cts index 8ce1b0a..8a586de 100644 --- a/dist/shared/style-constants.d.cts +++ b/dist/shared/style-constants.d.cts @@ -9,8 +9,9 @@ declare const HTML_PREFORMATTED_STYLE_ID = "HTMLPreformatted"; declare const MATH_BLOCK_STYLE_ID = "MathBlock"; declare const MONOSPACE_FONT_FAMILY = "Courier New"; declare const MATH_INLINE_FONT_MARKER = "Cambria Math"; +declare const FOOTNOTE_REFERENCE_FONT_MARKER = "Footnote Reference"; declare const QUOTE_INDENT_PT = 36; declare const TASK_CHECKBOX_UNCHECKED = "☐"; declare const TASK_CHECKBOX_CHECKED = "☒"; //#endregion -export { CODE_BLOCK_STYLE_ID, HORIZONTAL_RULE_STYLE_ID, HTML_PREFORMATTED_STYLE_ID, MATH_BLOCK_STYLE_ID, MATH_INLINE_FONT_MARKER, MAX_HEADING_STYLE_LEVEL, MONOSPACE_FONT_FAMILY, QUOTE_INDENT_PT, QUOTE_STYLE_ID, TASK_CHECKBOX_CHECKED, TASK_CHECKBOX_UNCHECKED, headingStyleId, parseHeadingStyleId }; \ No newline at end of file +export { CODE_BLOCK_STYLE_ID, FOOTNOTE_REFERENCE_FONT_MARKER, HORIZONTAL_RULE_STYLE_ID, HTML_PREFORMATTED_STYLE_ID, MATH_BLOCK_STYLE_ID, MATH_INLINE_FONT_MARKER, MAX_HEADING_STYLE_LEVEL, MONOSPACE_FONT_FAMILY, QUOTE_INDENT_PT, QUOTE_STYLE_ID, TASK_CHECKBOX_CHECKED, TASK_CHECKBOX_UNCHECKED, headingStyleId, parseHeadingStyleId }; \ No newline at end of file diff --git a/dist/shared/style-constants.d.ts b/dist/shared/style-constants.d.ts index 8ce1b0a..8a586de 100644 --- a/dist/shared/style-constants.d.ts +++ b/dist/shared/style-constants.d.ts @@ -9,8 +9,9 @@ declare const HTML_PREFORMATTED_STYLE_ID = "HTMLPreformatted"; declare const MATH_BLOCK_STYLE_ID = "MathBlock"; declare const MONOSPACE_FONT_FAMILY = "Courier New"; declare const MATH_INLINE_FONT_MARKER = "Cambria Math"; +declare const FOOTNOTE_REFERENCE_FONT_MARKER = "Footnote Reference"; declare const QUOTE_INDENT_PT = 36; declare const TASK_CHECKBOX_UNCHECKED = "☐"; declare const TASK_CHECKBOX_CHECKED = "☒"; //#endregion -export { CODE_BLOCK_STYLE_ID, HORIZONTAL_RULE_STYLE_ID, HTML_PREFORMATTED_STYLE_ID, MATH_BLOCK_STYLE_ID, MATH_INLINE_FONT_MARKER, MAX_HEADING_STYLE_LEVEL, MONOSPACE_FONT_FAMILY, QUOTE_INDENT_PT, QUOTE_STYLE_ID, TASK_CHECKBOX_CHECKED, TASK_CHECKBOX_UNCHECKED, headingStyleId, parseHeadingStyleId }; \ No newline at end of file +export { CODE_BLOCK_STYLE_ID, FOOTNOTE_REFERENCE_FONT_MARKER, HORIZONTAL_RULE_STYLE_ID, HTML_PREFORMATTED_STYLE_ID, MATH_BLOCK_STYLE_ID, MATH_INLINE_FONT_MARKER, MAX_HEADING_STYLE_LEVEL, MONOSPACE_FONT_FAMILY, QUOTE_INDENT_PT, QUOTE_STYLE_ID, TASK_CHECKBOX_CHECKED, TASK_CHECKBOX_UNCHECKED, headingStyleId, parseHeadingStyleId }; \ No newline at end of file diff --git a/dist/shared/style-constants.js b/dist/shared/style-constants.js index 8e2645f..4a3c071 100644 --- a/dist/shared/style-constants.js +++ b/dist/shared/style-constants.js @@ -18,8 +18,9 @@ const HTML_PREFORMATTED_STYLE_ID = "HTMLPreformatted"; const MATH_BLOCK_STYLE_ID = "MathBlock"; const MONOSPACE_FONT_FAMILY = "Courier New"; const MATH_INLINE_FONT_MARKER = "Cambria Math"; +const FOOTNOTE_REFERENCE_FONT_MARKER = "Footnote Reference"; const QUOTE_INDENT_PT = 36; const TASK_CHECKBOX_UNCHECKED = "☐"; const TASK_CHECKBOX_CHECKED = "☒"; //#endregion -export { CODE_BLOCK_STYLE_ID, HORIZONTAL_RULE_STYLE_ID, HTML_PREFORMATTED_STYLE_ID, MATH_BLOCK_STYLE_ID, MATH_INLINE_FONT_MARKER, MAX_HEADING_STYLE_LEVEL, MONOSPACE_FONT_FAMILY, QUOTE_INDENT_PT, QUOTE_STYLE_ID, TASK_CHECKBOX_CHECKED, TASK_CHECKBOX_UNCHECKED, headingStyleId, parseHeadingStyleId }; +export { CODE_BLOCK_STYLE_ID, FOOTNOTE_REFERENCE_FONT_MARKER, HORIZONTAL_RULE_STYLE_ID, HTML_PREFORMATTED_STYLE_ID, MATH_BLOCK_STYLE_ID, MATH_INLINE_FONT_MARKER, MAX_HEADING_STYLE_LEVEL, MONOSPACE_FONT_FAMILY, QUOTE_INDENT_PT, QUOTE_STYLE_ID, TASK_CHECKBOX_CHECKED, TASK_CHECKBOX_UNCHECKED, headingStyleId, parseHeadingStyleId }; diff --git a/package.json b/package.json index 9a6a8d7..391c49e 100644 --- a/package.json +++ b/package.json @@ -79,7 +79,7 @@ "license": "MIT", "packageManager": "pnpm@11.6.0", "dependencies": { - "document-schema.js": "^4.1.0", + "document-schema.js": "^4.2.0", "zod": "^4.4.3" }, "devDependencies": { diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 7e8a285..95d3894 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -9,8 +9,8 @@ importers: .: dependencies: document-schema.js: - specifier: ^4.1.0 - version: 4.1.0 + specifier: ^4.2.0 + version: 4.2.0 zod: specifier: ^4.4.3 version: 4.4.3 @@ -1603,8 +1603,8 @@ packages: resolution: {integrity: sha512-WkrWp9GR4KXfKGYzOLmTuGVi1UWFfws377n9cc55/tb6DuqyF6pcQ5AbiHEshaDpY9v6oaSr2XCDidGmMwdzIA==} engines: {node: '>=8'} - document-schema.js@4.1.0: - resolution: {integrity: sha512-sRnhIHGb7UvmtDX9DeJY6/uEULoQ3JZn4whH+gyyLAcP26zzr/u8EXEIZYDRIfabHEZHksBZOUQgCD1cacH3cA==} + document-schema.js@4.2.0: + resolution: {integrity: sha512-ckdp0yIItnQKPK23ciINygPbKPQanvodeLT9uJIj1EtQpjIlaMqHT/8iKdoql2zrgOol0rUguJAsEWCHEr44Gw==} engines: {node: '>=20'} dot-prop@5.3.0: @@ -4522,7 +4522,7 @@ snapshots: dependencies: path-type: 4.0.0 - document-schema.js@4.1.0: + document-schema.js@4.2.0: dependencies: zod: 4.4.3 diff --git a/src/ast/ast.ts b/src/ast/ast.ts index 05eace7..85e49e9 100644 --- a/src/ast/ast.ts +++ b/src/ast/ast.ts @@ -27,7 +27,8 @@ export type MarkdownBlockNode = | MarkdownTableNode | MarkdownTableRowNode | MarkdownTableCellNode - | MarkdownMathBlockNode; + | MarkdownMathBlockNode + | MarkdownFootnoteDefinitionNode; export interface MarkdownDocumentNode { readonly type: 'document'; @@ -138,6 +139,15 @@ export interface MarkdownMathBlockNode { readonly position?: MarkdownPosition; } +// A footnote definition's own tail-of-document block (`[^label]: body`, ExaDev/markdown-codec#66) -- a CONTAINER, not a leaf: its body is ordinary block content (further paragraphs, code blocks, tables, quotes, nested lists), continued by four columns of indentation exactly as a list item's own body is. That containment is what the definition's ContentDocument mapping needs: src/lower/lower.ts lowers this node to an `anchor` construct's boundary-marker pair (document-schema.js 4.2.0) bracketing the lowered body blocks, and AnchorDescriptor's own `definition` field names a package-level definitions-table key that a flat ContentDocument has no root to carry -- so the body rides the construct's own extent rather than a table entry. +export interface MarkdownFootnoteDefinitionNode { + readonly type: 'footnoteDefinition'; + // The identifier between `[^` and `]`, verbatim -- matched against a reference's own label exactly, with no case folding (see src/inline/footnote.ts). + readonly label: string; + readonly children: MarkdownBlockNode[]; + readonly position?: MarkdownPosition; +} + // --- Inline nodes --- export type MarkdownInlineNode = @@ -153,7 +163,8 @@ export type MarkdownInlineNode = | MarkdownSoftBreakNode | MarkdownRawHtmlNode | MarkdownEntityNode - | MarkdownMathInlineNode; + | MarkdownMathInlineNode + | MarkdownFootnoteReferenceNode; export type MarkdownEmphasisMarker = '_' | '*'; @@ -249,6 +260,15 @@ export interface MarkdownMathInlineNode { readonly position?: MarkdownPosition; } +// A footnote REFERENCE site (`[^label]` in running text, ExaDev/markdown-codec#66). Produced only when the document also carries a definition under that exact label -- an unmatched `[^label]` is ordinary text, which is GitHub's own reading and the only one that keeps a bracketed aside from silently becoming a dangling note. +// +// Deliberately a leaf carrying its label and nothing else: unlike the definition above, a reference has no extent. That is also why it is the one half of the footnote pair src/lower/lower.ts CANNOT map onto an `anchor` construct: a construct's extent is block-scoped by document-schema.js's own definition, and a reference sits between two runs INSIDE a paragraph, which no block-level boundary marker can bracket without splitting the paragraph in two. See src/lower/inline.ts's own footnoteReference case for the degrade that carries it instead, and MarkdownDiagnosticCodes.FOOTNOTE_REFERENCE_PRESERVED_AS_TEXT for the gap it reports. +export interface MarkdownFootnoteReferenceNode { + readonly type: 'footnoteReference'; + readonly label: string; + readonly position?: MarkdownPosition; +} + const BLOCK_NODE_TYPES: ReadonlySet = new Set([ 'document', 'paragraph', @@ -263,6 +283,7 @@ const BLOCK_NODE_TYPES: ReadonlySet = new Set 'tableRow', 'tableCell', 'mathBlock', + 'footnoteDefinition', ]); export function isMarkdownBlockNode(node: MarkdownNode): node is MarkdownBlockNode { diff --git a/src/block/block.ts b/src/block/block.ts index 28edf71..a1bea68 100644 --- a/src/block/block.ts +++ b/src/block/block.ts @@ -28,6 +28,8 @@ import type { MarkdownTableNode, MarkdownTableRowNode, } from '../ast/ast'; +import type { FootnoteLabelSet } from '../inline/footnote'; +import { matchFootnoteDefinitionMarker } from '../inline/footnote'; import type { MarkdownDiagnosticSink } from '../diagnostics/diagnostics'; import { DEFAULT_MAX_BLOCK_NESTING } from '../defaults/defaults'; import { MarkdownDiagnosticCodes, MarkdownNestingLimitExceededError, NOOP_MARKDOWN_DIAGNOSTIC_SINK } from '../diagnostics/diagnostics'; @@ -52,8 +54,8 @@ const NUL_PATTERN = /\0/g; const LINE_ENDING_PATTERN = /\r\n|\n|\r/; -// A cheap first filter before the block-start list is tried at all: no block start, and no paragraph promotion, can begin with any other character. `|` and `:` are here for the GFM table delimiter row (`| --- |`, `:-: | ---:`), the only construct in this package that can start with either. `$` is here for a $$ math block's own opening line (ExaDev/markdown-codec#53). -const MAYBE_SPECIAL_PATTERN = /^[#$`~*+_=<>0-9|:-]/; +// A cheap first filter before the block-start list is tried at all: no block start, and no paragraph promotion, can begin with any other character. `|` and `:` are here for the GFM table delimiter row (`| --- |`, `:-: | ---:`), the only construct in this package that can start with either. `$` is here for a $$ math block's own opening line (ExaDev/markdown-codec#53), and `[` for a footnote definition's own `[^label]:` marker (ExaDev/markdown-codec#66). +const MAYBE_SPECIAL_PATTERN = /^[#$`~*+_=<>[0-9|:-]/; // spec 0.31.2, "ATX headings": one to six `#` characters, followed by spaces/tabs or the end of the line. const ATX_MARKER_PATTERN = /^#{1,6}(?:[ \t]+|$)/; @@ -96,6 +98,8 @@ export interface MarkdownParseOptions extends InlineParseOptions { readonly gfmTables?: boolean; // GFM's task-list-item extension (`- [ ] foo` / `- [x] bar`). Enabled by default for the same reason; with it off, a leading `[ ]`/`[x]` is ordinary paragraph text, matching CommonMark's own reading (task lists are not part of CommonMark proper). readonly gfmTaskLists?: boolean; + // GitHub's footnote extension (`[^label]` markers with `[^label]: body` definitions, ExaDev/markdown-codec#66). Enabled by default like the four above; with it off, both spellings are ordinary text, which is what CommonMark and the GFM spec document itself both say (neither defines footnotes at all -- see src/inline/footnote.ts). + readonly footnotes?: boolean; // Throws MarkdownNestingLimitExceededError (src/diagnostics) rather than opening a block past this many levels deep in the open-block stack -- defaults to DEFAULT_MAX_BLOCK_NESTING (src/defaults), matching cmark's own reference-implementation guard against pathological/adversarial nesting. readonly maxNesting?: number; readonly sink?: MarkdownDiagnosticSink; @@ -105,6 +109,8 @@ export interface ParsedMarkdown { readonly document: MarkdownDocumentNode; // The document-global link-reference-definition table, complete before any inline was parsed against it. readonly references: LinkReferenceMap; + // The document-global set of footnote labels a definition was found for, complete before any inline was parsed against it -- the same forward-visibility guarantee `references` carries, for the same structural reason. + readonly footnotes: FootnoteLabelSet; } function isBlankContent(content: string): boolean { @@ -130,8 +136,10 @@ function headingLevelOf(hashes: number): BlockHeadingLevel { class BlockParser { readonly references = new Map(); + readonly footnotes = new Set(); private readonly document = new BlockNode('document', 1); private readonly tables: boolean; + private readonly footnotesEnabled: boolean; private readonly sink: MarkdownDiagnosticSink; private readonly maxNesting: number; private tip: BlockNode = this.document; @@ -146,6 +154,7 @@ class BlockParser { constructor(options: MarkdownParseOptions) { this.tables = options.gfmTables ?? true; + this.footnotesEnabled = options.footnotes ?? true; this.sink = options.sink ?? NOOP_MARKDOWN_DIAGNOSTIC_SINK; this.maxNesting = options.maxNesting ?? DEFAULT_MAX_BLOCK_NESTING; } @@ -227,6 +236,8 @@ class BlockParser { return this.continueBlockquote(); case 'listItem': return this.continueListItem(node); + case 'footnoteDefinition': + return this.continueFootnoteDefinition(node); case 'codeBlock': return this.continueCodeBlock(node); case 'mathBlock': @@ -285,6 +296,22 @@ class BlockParser { return 'not-matched'; } + // A definition's body continues on any line indented at least four columns -- the same continuation indent Pandoc and GitHub both use for a multi-block footnote, and the same one src/emit/emit.ts writes back out. A blank line continues it too (a definition may hold several paragraphs), except when the definition still has no content at all, mirroring the "a list item can begin with at most one blank line" rule one function up: `[^1]:` on a line of its own followed by a blank line is an empty definition, not the opening of one that swallows the rest of the document. + private continueFootnoteDefinition(node: BlockNode): ContinueResult { + if (this.line.blank) { + if (node.children.length === 0) { + return 'not-matched'; + } + this.line.advanceToNextNonspace(); + return 'matched'; + } + if (this.line.indent >= CODE_INDENT_COLUMNS) { + this.line.advance(CODE_INDENT_COLUMNS); + return 'matched'; + } + return 'not-matched'; + } + private continueCodeBlock(node: BlockNode): ContinueResult { if (!node.fenced) { if (this.line.indent >= CODE_INDENT_COLUMNS) { @@ -349,6 +376,7 @@ class BlockParser { () => this.tryAtxHeadingStart(), () => this.tryCodeFenceStart(), () => this.tryMathBlockStart(), + () => this.tryFootnoteDefinitionStart(container), () => this.tryHtmlBlockStart(container), () => this.tryPromoteParagraph(container), () => this.tryThematicBreakStart(), @@ -431,6 +459,41 @@ class BlockParser { return 'leaf'; } + // A footnote definition (ExaDev/markdown-codec#66) opens a CONTAINER, exactly as a list item does: the rest of the marker's own line, and every following line indented four columns, is its body. + // + // Two restrictions, both deliberate and both about what the ContentDocument mapping downstream can actually represent rather than about markdown's own grammar: + // + // - It may not interrupt a paragraph, matching a link reference definition (which is only ever recognised at the FRONT of a paragraph's accumulated content, src/block/definitions.ts) and matching Pandoc. A `[^1]: note` line directly under a line of prose is lazy paragraph continuation text. + // - It is recognised ONLY as a direct child of the document -- never inside a block quote or a list item. src/lower/lower.ts lowers a definition to a construct boundary-marker pair (document-schema.js 4.2.0) bracketing its own body, and that pair's extent may not cross a scope its enclosing container had already opened: a definition inside a list item would have to carry the item's own ContentListMembership on every body block, which a body table or image cannot carry at all, closing the item's list scope from INSIDE the pair -- precisely what the flat form's bracket-matching contract forbids a producer from emitting. A block quote is the same shape one level along: its own `> ` prefix is recovered on the way out from each paragraph's indentLeftPt, and a definition's label line has no paragraph of its own to carry it. Inside either container the `[^1]: ...` text stays an ordinary paragraph, exactly as it did before footnotes existed here. + private tryFootnoteDefinitionStart(container: BlockNode): BlockStartResult { + if (!this.footnotesEnabled || this.line.indented || !this.footnoteDefinitionMayOpenIn(container)) { + return 'none'; + } + const marker = matchFootnoteDefinitionMarker(this.line.restFromNextNonspace()); + if (marker === undefined) { + return 'none'; + } + if (this.footnotes.has(marker.label)) { + this.sink({ code: MarkdownDiagnosticCodes.DUPLICATE_FOOTNOTE_DEFINITION, severity: 'warning', message: `footnote "${marker.label}" was already defined earlier in the document; every reference resolves to the first definition, and both definitions are kept as written`, line: this.lineNumber }); + } + this.footnotes.add(marker.label); + this.line.advanceToNextNonspace(); + this.closeUnmatchedBlocks(); + const node = this.addChild('footnoteDefinition'); + node.footnoteLabel = marker.label; + this.line.advance(marker.markerLength); + return 'container'; + } + + // Whether `container` -- the deepest block the current line matched in step 1 -- sits at the document's own top level, walking up through any still-open `list` ancestors first. `continueBlock` treats a `list` node as unconditionally continued no matter what the line is (a list only actually closes when something tries to become its child and can't), so a line right after a top-level list's last item reports its matched container as that LIST, not the document, even though the list itself is about to close. Skipping over `list` ancestors here mirrors what `addChild` does a few lines below once a definition is actually opened: it walks up finalising whatever the tip can't contain, which closes a bare top-level list the same way any other block start does. A list nested inside a block quote or a list item still walks up to THAT container rather than the document, so the existing restriction on those two holds. + private footnoteDefinitionMayOpenIn(container: BlockNode): boolean { + let node: BlockNode | undefined = container; + while (node?.kind === 'list') { + node = node.parent; + } + return node?.kind === 'document'; + } + private tryHtmlBlockStart(container: BlockNode): BlockStartResult { if (this.line.indented || this.line.peekNextNonspace() !== '<') { return 'none'; @@ -672,13 +735,20 @@ class BlockParser { } } -function toInlineChildren(content: string, references: LinkReferenceMap, options: MarkdownParseOptions): MarkdownInlineNode[] { +// The two document-global tables the inline phase resolves against, plus the parse options, threaded through the AST conversion as one value rather than as three parallel parameters on every function below. +interface AstConversionContext { + readonly references: LinkReferenceMap; + readonly footnotes: FootnoteLabelSet; + readonly options: MarkdownParseOptions; +} + +function toInlineChildren(content: string, context: AstConversionContext): MarkdownInlineNode[] { // A leaf block's accumulated content keeps the line endings that separated its source lines but not the whitespace around the block itself: leading indentation was stripped as each line was added, and trailing whitespace at the very end of the block is not a hard line break. - return parseInlines(content.trim(), references, options); + return parseInlines(content.trim(), context.references, context.footnotes, context.options); } -function toHeadingNode(node: BlockNode, references: LinkReferenceMap, options: MarkdownParseOptions): MarkdownHeadingNode { - return { type: 'heading', level: node.level, style: node.setext ? 'setext' : 'atx', children: toInlineChildren(node.content, references, options) }; +function toHeadingNode(node: BlockNode, context: AstConversionContext): MarkdownHeadingNode { + return { type: 'heading', level: node.level, style: node.setext ? 'setext' : 'atx', children: toInlineChildren(node.content, context) }; } // Extracts a task-list-item marker from the FIRST child of a list item, mutating that child's own raw content in place to strip the marker (so the paragraph's own inline content, parsed afterwards, never sees it). Returns undefined -- never a false/absent sentinel -- when the item is not a task item at all, matching MarkdownListItemNode.checked's own "absent, not false" convention. @@ -695,16 +765,16 @@ function extractTaskListMarker(itemChildren: readonly BlockNode[]): boolean | un return match[1] !== ' '; } -function toListItemNode(item: BlockNode, references: LinkReferenceMap, options: MarkdownParseOptions): MarkdownListItemNode { - const taskLists = options.gfmTaskLists ?? true; +function toListItemNode(item: BlockNode, context: AstConversionContext): MarkdownListItemNode { + const taskLists = context.options.gfmTaskLists ?? true; const checked = taskLists ? extractTaskListMarker(item.children) : undefined; return checked === undefined - ? { type: 'listItem', children: toAstBlocks(item.children, references, options) } - : { type: 'listItem', checked, children: toAstBlocks(item.children, references, options) }; + ? { type: 'listItem', children: toAstBlocks(item.children, context) } + : { type: 'listItem', checked, children: toAstBlocks(item.children, context) }; } -function toListNode(node: BlockNode, references: LinkReferenceMap, options: MarkdownParseOptions): MarkdownListNode { - const children: MarkdownListItemNode[] = node.children.map((item) => toListItemNode(item, references, options)); +function toListNode(node: BlockNode, context: AstConversionContext): MarkdownListNode { + const children: MarkdownListItemNode[] = node.children.map((item) => toListItemNode(item, context)); const data = node.listData; if (data?.type === 'ordered') { return { type: 'list', markerType: 'ordered', orderedDelimiter: data.delimiter, start: data.start, tight: node.tight, children }; @@ -712,15 +782,15 @@ function toListNode(node: BlockNode, references: LinkReferenceMap, options: Mark return { type: 'list', markerType: 'bullet', bulletMarker: data?.bulletChar, tight: node.tight, children }; } -function toTableRow(cells: readonly string[], header: boolean, references: LinkReferenceMap, options: MarkdownParseOptions): MarkdownTableRowNode { - const children: MarkdownTableCellNode[] = cells.map((cell) => ({ type: 'tableCell', children: toInlineChildren(cell, references, options) })); +function toTableRow(cells: readonly string[], header: boolean, context: AstConversionContext): MarkdownTableRowNode { + const children: MarkdownTableCellNode[] = cells.map((cell) => ({ type: 'tableCell', children: toInlineChildren(cell, context) })); return { type: 'tableRow', header, children }; } -function toTableNode(node: BlockNode, references: LinkReferenceMap, options: MarkdownParseOptions): MarkdownTableNode { - const sink = options.sink ?? NOOP_MARKDOWN_DIAGNOSTIC_SINK; +function toTableNode(node: BlockNode, context: AstConversionContext): MarkdownTableNode { + const sink = context.options.sink ?? NOOP_MARKDOWN_DIAGNOSTIC_SINK; const columnCount = node.alignments.length; - const rows: MarkdownTableRowNode[] = [toTableRow(fitRowToColumns(splitTableRow(node.headerLine), columnCount), true, references, options)]; + const rows: MarkdownTableRowNode[] = [toTableRow(fitRowToColumns(splitTableRow(node.headerLine), columnCount), true, context)]; for (const rowLine of node.content.split('\n')) { if (rowLine.trim().length === 0) { continue; @@ -729,21 +799,23 @@ function toTableNode(node: BlockNode, references: LinkReferenceMap, options: Mar if (cells.length !== columnCount) { sink({ code: MarkdownDiagnosticCodes.TABLE_CELL_COUNT_MISMATCH, severity: 'warning', message: `table row has ${String(cells.length)} cell(s), but the header row declares ${String(columnCount)}; the row is padded with empty cells or truncated to fit`, line: node.startLine }); } - rows.push(toTableRow(fitRowToColumns(cells, columnCount), false, references, options)); + rows.push(toTableRow(fitRowToColumns(cells, columnCount), false, context)); } return { type: 'table', alignments: node.alignments, children: rows }; } -function toAstBlock(node: BlockNode, references: LinkReferenceMap, options: MarkdownParseOptions): MarkdownBlockNode | undefined { +function toAstBlock(node: BlockNode, context: AstConversionContext): MarkdownBlockNode | undefined { switch (node.kind) { case 'paragraph': - return { type: 'paragraph', children: toInlineChildren(node.content, references, options) }; + return { type: 'paragraph', children: toInlineChildren(node.content, context) }; case 'heading': - return toHeadingNode(node, references, options); + return toHeadingNode(node, context); case 'blockquote': - return { type: 'blockquote', children: toAstBlocks(node.children, references, options) }; + return { type: 'blockquote', children: toAstBlocks(node.children, context) }; case 'list': - return toListNode(node, references, options); + return toListNode(node, context); + case 'footnoteDefinition': + return { type: 'footnoteDefinition', label: node.footnoteLabel, children: toAstBlocks(node.children, context) }; case 'codeBlock': return node.fenced ? { type: 'codeBlock', fenced: true, fenceChar: node.fenceChar, infoString: node.infoString, literal: node.literal } @@ -755,7 +827,7 @@ function toAstBlock(node: BlockNode, references: LinkReferenceMap, options: Mark case 'mathBlock': return { type: 'mathBlock', literal: node.literal }; case 'table': - return toTableNode(node, references, options); + return toTableNode(node, context); case 'document': case 'listItem': // Neither can appear as a child of anything toAstBlocks walks: a document is the root, and a list item is only ever reached through its own list. @@ -763,10 +835,10 @@ function toAstBlock(node: BlockNode, references: LinkReferenceMap, options: Mark } } -function toAstBlocks(nodes: readonly BlockNode[], references: LinkReferenceMap, options: MarkdownParseOptions): MarkdownBlockNode[] { +function toAstBlocks(nodes: readonly BlockNode[], context: AstConversionContext): MarkdownBlockNode[] { const blocks: MarkdownBlockNode[] = []; for (const node of nodes) { - const converted = toAstBlock(node, references, options); + const converted = toAstBlock(node, context); if (converted !== undefined) { blocks.push(converted); } @@ -774,10 +846,10 @@ function toAstBlocks(nodes: readonly BlockNode[], references: LinkReferenceMap, return blocks; } -// Parses a whole markdown document: block structure first, to completion, then every leaf block's own inline content against the finished link-reference-definition table. See this module's own top-of-file note on why that ordering is structural rather than a matter of convenience. +// Parses a whole markdown document: block structure first, to completion, then every leaf block's own inline content against the finished link-reference-definition table and footnote-label set. See this module's own top-of-file note on why that ordering is structural rather than a matter of convenience. export function parseMarkdown(source: string, options: MarkdownParseOptions = {}): ParsedMarkdown { const parser = new BlockParser(options); const root = parser.parse(source); - const references: LinkReferenceMap = parser.references; - return { document: { type: 'document', children: toAstBlocks(root.children, references, options) }, references }; + const context: AstConversionContext = { references: parser.references, footnotes: parser.footnotes, options }; + return { document: { type: 'document', children: toAstBlocks(root.children, context) }, references: context.references, footnotes: context.footnotes }; } diff --git a/src/block/node.ts b/src/block/node.ts index 69ae30b..107f21e 100644 --- a/src/block/node.ts +++ b/src/block/node.ts @@ -16,7 +16,8 @@ export type BlockNodeKind = | 'htmlBlock' | 'thematicBreak' | 'table' - | 'mathBlock'; + | 'mathBlock' + | 'footnoteDefinition'; export type BlockHeadingLevel = 1 | 2 | 3 | 4 | 5 | 6; @@ -71,6 +72,9 @@ export class BlockNode { // The table's header row, as raw source text; the body rows arrive through `content` like any other line-accepting leaf block. headerLine = ''; + // A footnote definition's own label, from its `[^label]:` opening marker. + footnoteLabel = ''; + constructor(kind: BlockNodeKind, startLine: number) { this.kind = kind; this.startLine = startLine; @@ -115,8 +119,12 @@ export class BlockNode { } // Which block kinds may hold which children, per CommonMark's own container/leaf split. A list holds only items; an item (like a blockquote or the document) holds anything except a bare item; every leaf block holds no blocks at all. +// +// A footnote definition is a container of the same shape as a list item -- its body is ordinary block content, continued by indentation -- with one further restriction: it may not hold another footnote definition. Nesting one inside another has no meaning (a definition is addressed by a document-global label, not by its position), and src/block/block.ts's own start rule already refuses to open one anywhere but at the document's own top level, so this is the type-level statement of a rule the parser never reaches the other way round. export function canContain(parent: BlockNodeKind, child: BlockNodeKind): boolean { switch (parent) { + case 'footnoteDefinition': + return child !== 'listItem' && child !== 'footnoteDefinition'; case 'document': case 'blockquote': case 'listItem': diff --git a/src/conformance.test.ts b/src/conformance.test.ts index 35cfe14..a4c0440 100644 --- a/src/conformance.test.ts +++ b/src/conformance.test.ts @@ -2,7 +2,7 @@ // // Each example is run end to end through the real PUBLIC readMarkdown/writeMarkdown surface, not just the bare parser: readMarkdown (src/read.ts, itself src/block/block.ts's parseMarkdown plus src/lower/lower.ts's lowering to a ContentDocument) produces the document-schema.js pivot; writeMarkdown (src/write.ts, src/emit/emit.ts) renders that pivot back to markdown text; parseMarkdown reads that rewritten text a second time, under the identical CommonMark-only options, back to this package's own internal AST; and src/html/render.ts (the real CommonMark-HTML conformance oracle) renders that AST to HTML, compared byte for byte against the corpus's own `html` field. This is deliberately a stricter bar than measuring the bare parser alone: a round trip through the ContentDocument pivot has to survive src/lower's own semantic mapping AND src/emit's own inverse rendering with no loss the reparse can detect, which is exactly the wiring this test exists to prove now that read/write/codec are assembled -- see src/lower/ and src/emit/'s own top-of-file comments for what each stage is documented to gain or lose. // -// GFM's own extensions are switched OFF for both the read and the reparse: a bare `http://example.com` in paragraph text is plain text under CommonMark and a link under GFM, a `~~x~~` is literal tildes, a delimiter row is ordinary paragraph text, and a leading `[ ]`/`[x]` is ordinary paragraph text rather than a task-list marker -- this suite measures CommonMark, and src/gfm-conformance.test.ts measures the extensions (through the identical read -> write -> reparse -> render path) against their own corpus. writeMarkdown itself has no GFM toggle of its own to match: it emits whatever markdown syntax a given ContentDocument construct needs (a ContentTable always becomes a GFM table, a strike run always becomes `~~x~~`), and with the extensions off on the read side no such construct is ever produced from a CommonMark-only example in the first place. +// GFM's own extensions, and GitHub's footnote extension alongside them, are switched OFF for both the read and the reparse: a bare `http://example.com` in paragraph text is plain text under CommonMark and a link under GFM, a `~~x~~` is literal tildes, a delimiter row is ordinary paragraph text, and a leading `[ ]`/`[x]` is ordinary paragraph text rather than a task-list marker -- this suite measures CommonMark, and src/gfm-conformance.test.ts measures the extensions (through the identical read -> write -> reparse -> render path) against their own corpus. writeMarkdown itself has no GFM toggle of its own to match: it emits whatever markdown syntax a given ContentDocument construct needs (a ContentTable always becomes a GFM table, a strike run always becomes `~~x~~`), and with the extensions off on the read side no such construct is ever produced from a CommonMark-only example in the first place. // // Anything not yet passing is named individually in src/test-support/conformance-exclusions.ts, with a test below asserting that every excluded example genuinely still fails -- see that file for why the list can only shrink. @@ -16,7 +16,7 @@ import { loadSpecExamples } from './test-support/spec-corpus'; import { writeMarkdown } from './write'; // CommonMark, not CommonMark+GFM -- see this file's own top-of-file note. -const COMMONMARK_ONLY = { gfmAutolinks: false, gfmStrikethrough: false, gfmTables: false, gfmTaskLists: false }; +const COMMONMARK_ONLY = { gfmAutolinks: false, gfmStrikethrough: false, gfmTables: false, gfmTaskLists: false, footnotes: false }; // read -> write -> reparse -> render, all through this package's real public surface -- see this file's own top-of-file note for why this is the bar now, not a direct parseMarkdown -> render measurement. function render(example: SpecExample): string { diff --git a/src/diagnostics/diagnostics.test.ts b/src/diagnostics/diagnostics.test.ts index cad663c..6671f01 100644 --- a/src/diagnostics/diagnostics.test.ts +++ b/src/diagnostics/diagnostics.test.ts @@ -204,6 +204,38 @@ describe('every MarkdownDiagnosticCodes entry is reachable from real input', () reached.add(MarkdownDiagnosticCodes.TABLE_CELL_MULTI_PARAGRAPH_JOINED); }); + it('DUPLICATE_FOOTNOTE_DEFINITION: two definitions sharing one label', () => { + const collector = createDiagnosticCollector(); + parseMarkdown('[^1]: first\n\n[^1]: second', { sink: collector.sink }); + expect(collector.has(MarkdownDiagnosticCodes.DUPLICATE_FOOTNOTE_DEFINITION)).toBe(true); + reached.add(MarkdownDiagnosticCodes.DUPLICATE_FOOTNOTE_DEFINITION); + }); + + it('FOOTNOTE_REFERENCE_PRESERVED_AS_TEXT: a reference resolving against a definition', () => { + const collector = createDiagnosticCollector(); + lowerMarkdown('see[^1]\n\n[^1]: note', { sink: collector.sink }); + expect(collector.has(MarkdownDiagnosticCodes.FOOTNOTE_REFERENCE_PRESERVED_AS_TEXT)).toBe(true); + reached.add(MarkdownDiagnosticCodes.FOOTNOTE_REFERENCE_PRESERVED_AS_TEXT); + }); + + it('FOOTNOTE_BODY_HEADING_FLATTENED: a heading inside a definition body', () => { + const collector = createDiagnosticCollector(); + lowerMarkdown('[^1]: intro\n\n # inner', { sink: collector.sink }); + expect(collector.has(MarkdownDiagnosticCodes.FOOTNOTE_BODY_HEADING_FLATTENED)).toBe(true); + reached.add(MarkdownDiagnosticCodes.FOOTNOTE_BODY_HEADING_FLATTENED); + }); + + it('CONSTRUCT_UNREPRESENTED: a construct kind markdown has no syntax for', () => { + const collector = createDiagnosticCollector(); + emitMarkdown(minimalDocument([ + { kind: 'constructStart', descriptor: { kind: 'division', name: 'chapter' } }, + { kind: 'paragraph', runs: [{ text: 'inside' }] }, + { kind: 'constructEnd' }, + ]), { sink: collector.sink }); + expect(collector.has(MarkdownDiagnosticCodes.CONSTRUCT_UNREPRESENTED)).toBe(true); + reached.add(MarkdownDiagnosticCodes.CONSTRUCT_UNREPRESENTED); + }); + it('has no dead code: every value in MarkdownDiagnosticCodes was proven reachable above', () => { expect(reached).toEqual(new Set(Object.values(MarkdownDiagnosticCodes))); }); diff --git a/src/diagnostics/diagnostics.ts b/src/diagnostics/diagnostics.ts index a2e98cf..f109564 100644 --- a/src/diagnostics/diagnostics.ts +++ b/src/diagnostics/diagnostics.ts @@ -30,6 +30,7 @@ export const MarkdownDiagnosticCodes = { UNTERMINATED_HTML_BLOCK: 'md/unterminated-html-block', TABLE_CELL_COUNT_MISMATCH: 'md/table-cell-count-mismatch', DUPLICATE_LINK_REFERENCE: 'md/duplicate-link-reference', + DUPLICATE_FOOTNOTE_DEFINITION: 'md/duplicate-footnote-definition', LIST_MARKER_TYPE_CONFLICT: 'md/list-marker-type-conflict', // src/lower (read side: markdown -> ContentDocument) INVENTED_PAGE_GEOMETRY: 'md/invented-page-geometry', @@ -45,7 +46,10 @@ export const MarkdownDiagnosticCodes = { MATH_BLOCK_PRESERVED_AS_TEXT: 'md/math-block-preserved-as-text', MATH_INLINE_PRESERVED_AS_TEXT: 'md/math-inline-preserved-as-text', FRONT_MATTER_KEY_UNMAPPED: 'md/front-matter-key-unmapped', + FOOTNOTE_REFERENCE_PRESERVED_AS_TEXT: 'md/footnote-reference-preserved-as-text', + FOOTNOTE_BODY_HEADING_FLATTENED: 'md/footnote-body-heading-flattened', // src/emit (write side: ContentDocument -> markdown) + CONSTRUCT_UNREPRESENTED: 'md/construct-unrepresented', HEADING_LEVEL_CLAMPED: 'md/heading-level-clamped', ADJACENT_LINKS_MERGED: 'md/adjacent-links-merged', CODE_SPAN_AS_MONOSPACE_RUN: 'md/code-span-as-monospace-run', @@ -109,6 +113,21 @@ export class MarkdownWriteError extends Error { } } +// Thrown by writeMarkdown when handed a ContentDocument whose block list's construct boundary markers (document-schema.js 4.2.0's ContentConstructStart/ContentConstructEnd) do not pair up as balanced brackets -- an end with no construct open at that point, or a start still open when the list ended. This is the throw tier rather than a degrade because an unbalanced list is malformed input rather than a shape to repair: the schema's own bracket-matching contract states outright that the blocks between a matched pair ARE the construct's extent, so with the pairing broken there is no extent to render and no correct guess about which blocks the producer meant to include. Detected via document-schema.js's own findConstructMarkerImbalance -- the one shared definition of the check, which every codec that emits a pair and documents.js's own decompose all have to agree on exactly. +export class MarkdownUnbalancedConstructMarkersError extends MarkdownWriteError { + // Which end of the pairing failed, and the offending block's own index in its section's block list -- carried verbatim from findConstructMarkerImbalance so a caller can locate it without re-running the walk. + readonly imbalanceKind: 'unmatchedEnd' | 'unclosedStart'; + readonly blockIndex: number; + + constructor(imbalanceKind: 'unmatchedEnd' | 'unclosedStart', blockIndex: number) { + const description = imbalanceKind === 'unmatchedEnd' ? 'a constructEnd marker closes no open construct' : 'a constructStart marker is never closed'; + super('md/unbalanced-construct-markers', `${description} at block index ${String(blockIndex)}; a block list's construct boundary markers must pair as balanced brackets`); + this.name = 'MarkdownUnbalancedConstructMarkersError'; + this.imbalanceKind = imbalanceKind; + this.blockIndex = blockIndex; + } +} + // Thrown by writeMarkdown when handed a ContentDocument whose kind is not 'wordprocessing' -- markdown has no presentation/spreadsheet/drawing equivalent to render, matching ooxml.js's buildXlsxPackage's own "throw outright for the wrong document kind" convention (see documents.js's src/ooxml/xlsx precedent) rather than accepting a value a caller would need to pre-check themselves. Extends MarkdownWriteError, not MarkdownParseError -- this is a write-time failure (writeMarkdown's own entry point), never reachable from readMarkdown at all. export class MarkdownUnsupportedDocumentKindError extends MarkdownWriteError { readonly kind: string; diff --git a/src/emit/emit.ts b/src/emit/emit.ts index e378ced..728d342 100644 --- a/src/emit/emit.ts +++ b/src/emit/emit.ts @@ -10,12 +10,13 @@ // // ContentPageBreak and ContentEmbeddedObjectBlock have no markdown representation of any kind (this package's own src/lower never produces either, but ContentDocument is a shared pivot a caller can construct directly) -- both are silently dropped, contributing no output at all; this is not one of this package's own named mapping gaps (there was never a markdown construct to lose fidelity from), so it carries no diagnostic code. -import type { ContentBlock, ContentDocument, ContentParagraph } from 'document-schema.js'; -import { clampHeadingLevel } from 'document-schema.js'; -import { MarkdownUnsupportedDocumentKindError } from '../diagnostics/diagnostics'; +import type { ConstructDescriptor, ContentBlock, ContentConstructEnd, ContentConstructStart, ContentDocument, ContentParagraph } from 'document-schema.js'; +import { clampHeadingLevel, findConstructMarkerImbalance } from 'document-schema.js'; +import { MarkdownUnbalancedConstructMarkersError, MarkdownUnsupportedDocumentKindError } from '../diagnostics/diagnostics'; import type { MarkdownDiagnosticSink } from '../diagnostics/diagnostics'; import { MarkdownDiagnosticCodes, NOOP_MARKDOWN_DIAGNOSTIC_SINK } from '../diagnostics/diagnostics'; import { DEFAULT_BULLET_LIST_MARKER, DEFAULT_CODE_FENCE_CHAR, DEFAULT_EMPHASIS_MARKER, DEFAULT_HEADING_STYLE, DEFAULT_LINE_ENDING, DEFAULT_ORDERED_LIST_DELIMITER, DEFAULT_THEMATIC_BREAK_CHAR } from '../defaults/defaults'; +import { isValidFootnoteLabel } from '../inline/footnote'; import type { MarkdownHeadingStyle, WriteMarkdownOptions } from '../options/options'; import type { ListNumIdInfo } from '../shared/list-id'; import { parseListNumId } from '../shared/list-id'; @@ -143,7 +144,10 @@ function renderParagraph(paragraph: ContentParagraph, context: EmitContext): str .join('\n'); } -function renderTopLevelBlock(block: ContentBlock, context: EmitContext): string { +// Every ContentBlock kind that renders as content in its own right -- the whole union MINUS the two construct boundary markers, which are structure rather than content and are consumed by groupConstructItems below before any block reaches here. Spelled as a type rather than as two unreachable switch arms so the compiler, not a comment, is what guarantees a marker never arrives. +type RenderableBlock = Exclude; + +function renderTopLevelBlock(block: RenderableBlock, context: EmitContext): string { switch (block.kind) { case 'paragraph': return renderParagraph(block, context); @@ -281,27 +285,105 @@ function renderListRegion(items: readonly ContentParagraph[], context: EmitConte return out; } -// A consecutive run of quoted top-level blocks at the SAME depth is genuinely ambiguous once lowered -- ContentParagraph.indentLeftPt has no field distinguishing "one blockquote containing several blocks" from "several independent blockquotes back to back at the same depth" (document-schema.js carries no ContentBlockquote container of its own; src/lower/lower.ts flattens both shapes identically). Joining every top-level block with a bare blank line, as below, resolves that ambiguity by always choosing the "independent blockquotes" reading -- the correctness-preserving default, since re-joining two ADJACENT SAME-depth quoted blocks into one blockquote (tried and reverted here) fixed no example this package's own soft-line-break handling (src/lower/inline.ts's own softBreak -> ' ' mapping, see src/test-support/conformance-exclusions.ts) did not already fail on for an unrelated reason, while genuinely breaking two real cases (two independent same-depth blockquotes with nothing between them) that this simpler join gets right. -function emitBlocks(blocks: readonly ContentBlock[], context: EmitContext): string { - const parts: string[] = []; - let index = 0; +// --- Construct boundary markers (document-schema.js 4.2.0): the flat form encodes a construct as a MATCHED PAIR of markers bracketing the blocks it spans, so the writer's first job over any block list is to recover that bracketing as a tree before rendering anything. --- + +// One item of a block list once the markers have been resolved: either an ordinary content block, or a construct with its own extent recovered as children (which may themselves contain further constructs, at any nesting depth). +type EmitItem = { readonly block: RenderableBlock } | ConstructItem; + +interface ConstructItem { + readonly descriptor: ConstructDescriptor; + readonly children: readonly EmitItem[]; +} + +function isConstructItem(item: EmitItem): item is ConstructItem { + return 'descriptor' in item; +} + +// Bracket matching, per document-schema.js's own contract: a constructEnd closes the nearest preceding still-open constructStart in the SAME block list, and the blocks between them are that construct's extent. emitMarkdown validates the whole list's balance up front (findConstructMarkerImbalance -- the one shared definition of that check, which this writer, every sibling codec, and documents.js's decompose all have to agree on exactly), so by the time this runs a closing marker for every open one is known to exist. +function groupConstructItems(blocks: readonly ContentBlock[], start: number): { readonly items: EmitItem[]; readonly next: number } { + const items: EmitItem[] = []; + let index = start; while (index < blocks.length) { const block = blocks[index]; if (block === undefined) { break; } - if (block.kind === 'paragraph' && block.list !== undefined) { + index += 1; + if (block.kind === 'constructEnd') { + return { items, next: index }; + } + if (block.kind === 'constructStart') { + const nested = groupConstructItems(blocks, index); + items.push({ descriptor: block.descriptor, children: nested.items }); + index = nested.next; + continue; + } + items.push({ block }); + } + return { items, next: index }; +} + +// Columns of indentation a footnote definition's own continuation lines carry -- the same four src/block/block.ts's continueFootnoteDefinition strips back off, and the same four Pandoc and GitHub both write. Deliberately NOT the rendered `[^label]: ` marker's own width (which varies with the label): a reader measures the continuation indent against a fixed column, not against whatever the marker happened to occupy. +const FOOTNOTE_CONTINUATION_INDENT = 4; + +// The write-side inverse of src/lower/lower.ts's lowerFootnoteDefinition: the anchor's own name becomes the `[^label]:` marker, and its extent becomes the definition's body, every line after the first indented to the continuation column. An empty extent (the point anchor a bodyless `[^1]:` lowers to) emits the bare marker rather than a marker followed by a trailing space. +function renderFootnoteDefinition(name: string, body: string): string { + const marker = `[^${name}]:`; + if (body.length === 0) { + return marker; + } + const indent = ' '.repeat(FOOTNOTE_CONTINUATION_INDENT); + const [firstLine = '', ...restLines] = body.split('\n'); + return [`${marker} ${firstLine}`, ...restLines.map((line) => (line.length === 0 ? line : `${indent}${line}`))].join('\n'); +} + +// A construct markdown has a syntax for renders as that syntax; one it does not is TRANSPARENT -- its extent still renders in place, and only the construct's own identity is lost. That is the correct degrade rather than dropping the extent: a ContentDocument reaching this writer from another codec (an odt division, a docx content control, a tracked-change wrapper) carries real content inside markers markdown cannot spell, and dropping the wrapper's content along with the wrapper would lose the document, not just the construct. +// +// `anchor` is the only descriptor kind with a markdown spelling at all, and only for its footnote arm: a bookmark, an endnote, and a comment have no CommonMark or GFM syntax, and neither does any of the other five descriptor kinds. +function renderConstruct(item: ConstructItem, context: EmitContext): string { + const body = renderItems(item.children, context); + const { descriptor } = item; + if (descriptor.kind === 'anchor' && descriptor.anchorType === 'footnote') { + if (isValidFootnoteLabel(descriptor.name)) { + return renderFootnoteDefinition(descriptor.name, body); + } + context.sink({ code: MarkdownDiagnosticCodes.CONSTRUCT_UNREPRESENTED, severity: 'info', message: `a footnote anchor's own name "${descriptor.name}" cannot be spelled as a "[^label]:" marker (whitespace or "]" would reparse as something else); its own extent still renders in place, but the construct itself is not represented` }); + return body; + } + const detail = descriptor.kind === 'anchor' ? `${descriptor.kind} (${descriptor.anchorType})` : descriptor.kind; + context.sink({ code: MarkdownDiagnosticCodes.CONSTRUCT_UNREPRESENTED, severity: 'info', message: `a "${detail}" construct has no markdown syntax; its own extent still renders in place, but the construct itself is not represented` }); + return body; +} + +// A consecutive run of quoted top-level blocks at the SAME depth is genuinely ambiguous once lowered -- ContentParagraph.indentLeftPt has no field distinguishing "one blockquote containing several blocks" from "several independent blockquotes back to back at the same depth" (document-schema.js carries no ContentBlockquote container of its own; src/lower/lower.ts flattens both shapes identically). Joining every top-level block with a bare blank line, as below, resolves that ambiguity by always choosing the "independent blockquotes" reading -- the correctness-preserving default, since re-joining two ADJACENT SAME-depth quoted blocks into one blockquote (tried and reverted here) fixed no example this package's own soft-line-break handling (src/lower/inline.ts's own softBreak -> ' ' mapping, see src/test-support/conformance-exclusions.ts) did not already fail on for an unrelated reason, while genuinely breaking two real cases (two independent same-depth blockquotes with nothing between them) that this simpler join gets right. +function renderItems(items: readonly EmitItem[], context: EmitContext): string { + const parts: string[] = []; + let index = 0; + while (index < items.length) { + const item = items[index]; + if (item === undefined) { + break; + } + if (isConstructItem(item)) { + const rendered = renderConstruct(item, context); + if (rendered.length > 0) { + parts.push(rendered); + } + index += 1; + continue; + } + if (item.block.kind === 'paragraph' && item.block.list !== undefined) { const region: ContentParagraph[] = []; let end = index; - for (let candidate = blocks[end]; candidate?.kind === 'paragraph' && candidate.list !== undefined; candidate = blocks[end]) { - region.push(candidate); + for (let candidate = items[end]; candidate !== undefined && !isConstructItem(candidate) && candidate.block.kind === 'paragraph' && candidate.block.list !== undefined; candidate = items[end]) { + region.push(candidate.block); end += 1; } parts.push(renderListRegion(region, context)); index = end; continue; } - const rendered = renderTopLevelBlock(block, context); + const rendered = renderTopLevelBlock(item.block, context); if (rendered.length > 0) { parts.push(rendered); } @@ -310,6 +392,14 @@ function emitBlocks(blocks: readonly ContentBlock[], context: EmitContext): stri return parts.join('\n\n'); } +function emitBlocks(blocks: readonly ContentBlock[], context: EmitContext): string { + const imbalance = findConstructMarkerImbalance(blocks); + if (imbalance !== undefined) { + throw new MarkdownUnbalancedConstructMarkersError(imbalance.kind, imbalance.index); + } + return renderItems(groupConstructItems(blocks, 0).items, context); +} + export function emitMarkdown(document: ContentDocument, options: WriteMarkdownOptions = {}): string { if (document.kind !== 'wordprocessing') { throw new MarkdownUnsupportedDocumentKindError(document.kind); diff --git a/src/emit/inline.ts b/src/emit/inline.ts index 5efbd11..aeb6f20 100644 --- a/src/emit/inline.ts +++ b/src/emit/inline.ts @@ -10,7 +10,7 @@ import type { ContentRun } from 'document-schema.js'; import type { MarkdownDiagnosticSink } from '../diagnostics/diagnostics'; import { MarkdownDiagnosticCodes } from '../diagnostics/diagnostics'; import { matchHtmlTag } from '../html/html'; -import { MATH_INLINE_FONT_MARKER, MONOSPACE_FONT_FAMILY } from '../shared/style-constants'; +import { FOOTNOTE_REFERENCE_FONT_MARKER, MATH_INLINE_FONT_MARKER, MONOSPACE_FONT_FAMILY } from '../shared/style-constants'; export interface InlineEmitContext { readonly sink: MarkdownDiagnosticSink; @@ -76,6 +76,10 @@ function renderLeaf(run: ContentRun, context: InlineEmitContext): string { context.sink({ code: MarkdownDiagnosticCodes.CODE_SPAN_AS_MONOSPACE_RUN, severity: 'info', message: 'a run styled with the Courier New font family is rendered as a code span; a genuinely monospace run from another format is indistinguishable from a real markdown code span on the way back out' }); return renderCodeSpan(run.text); } + if (run.fontFamily === FOOTNOTE_REFERENCE_FONT_MARKER) { + // The run's text already IS the reference's own `[^label]` spelling (src/lower/inline.ts), written out verbatim rather than escaped -- exactly the same reason the math case below skips escaping, and the reason the marker has to exist at all: escapeMarkdownText escapes `[`, `^`, and `]`, so a deliberately-escaped literal `\[^1\]` and a genuine reference are the same run text by the time they reach here, and only the marker separates them. + return run.text; + } if (run.fontFamily === MATH_INLINE_FONT_MARKER) { // The \( \) delimiters are regenerated fresh around the run's own (unescaped) text -- this run's text is never passed through escapeMarkdownText at all, since it is not "ordinary punctuation that happens to need escaping" but raw LaTeX carried verbatim (see this module's own top-of-file note on why a text-pattern-based recognition of an already-escaped '(...)' cannot distinguish this from ordinary parenthetical prose). return `\\(${run.text}\\)`; @@ -155,8 +159,8 @@ function renderNestedStyles(runs: readonly ContentRun[], depth: number, context: } function isPlainAutolink(run: ContentRun): boolean { - // An autolink's own <...> form can never be empty (CommonMark's own URI/email autolink grammar both require at least one character between the brackets) -- `<>` is not valid autolink syntax at all and would reparse as literal text, so an empty destination (only reachable via a `[](/url)`-shaped empty-text link whose text happens to equal its own empty destination) must fall through to the ordinary `[text](dest)` form instead. A monospace (code-span) or math-marked run is excluded the same way: both need their own dedicated renderLeaf rendering (a code span's backtick fence, math's own \( \) delimiters), never the bare <...> autolink form, however coincidentally their own text might equal the surrounding hyperlink. - if (run.hyperlink === undefined || run.hyperlink.length === 0 || run.bold === true || run.italic === true || run.strike === true || run.fontFamily === MONOSPACE_FONT_FAMILY || run.fontFamily === MATH_INLINE_FONT_MARKER) { + // An autolink's own <...> form can never be empty (CommonMark's own URI/email autolink grammar both require at least one character between the brackets) -- `<>` is not valid autolink syntax at all and would reparse as literal text, so an empty destination (only reachable via a `[](/url)`-shaped empty-text link whose text happens to equal its own empty destination) must fall through to the ordinary `[text](dest)` form instead. A monospace (code-span), math-marked, or footnote-reference-marked run is excluded the same way: each needs its own dedicated renderLeaf rendering (a code span's backtick fence, math's own \( \) delimiters, a reference's own unescaped `[^label]`), never the bare <...> autolink form, however coincidentally their own text might equal the surrounding hyperlink. + if (run.hyperlink === undefined || run.hyperlink.length === 0 || run.bold === true || run.italic === true || run.strike === true || run.fontFamily === MONOSPACE_FONT_FAMILY || run.fontFamily === MATH_INLINE_FONT_MARKER || run.fontFamily === FOOTNOTE_REFERENCE_FONT_MARKER) { return false; } return run.text === run.hyperlink || run.hyperlink === `mailto:${run.text}`; diff --git a/src/footnote.test.ts b/src/footnote.test.ts new file mode 100644 index 0000000..6167d32 --- /dev/null +++ b/src/footnote.test.ts @@ -0,0 +1,346 @@ +// GitHub footnotes end to end (ExaDev/markdown-codec#66): the block/inline phases that recognise `[^label]` and `[^label]: body`, the lowering that turns a definition into an `anchor` construct's boundary-marker pair (document-schema.js 4.2.0) and a reference into a marked run, and the writer that renders both back. Deliberately one file across all four stages rather than four scattered additions: the whole point of the feature is that the two halves of a footnote are carried by two DIFFERENT mechanisms and still have to reproduce each other, which no single-stage test can show. +// +// The round-trip assertion below is "read -> write -> read -> write reproduces the same text", not "write reproduces the source byte for byte". That is not a weaker bar chosen for convenience: this package normalises freely on the way out (it escapes ASCII punctuation, regenerates code fences, and picks its own bullet glyph), so byte equality with arbitrary source text is not a property `writeMarkdown` has for ANY construct. What must hold, and what is asserted, is that nothing about a footnote is lost on the way through -- the second pass produces the identical document and the identical text. + +import type { ContentBlock, ContentDocument } from 'document-schema.js'; +import { PAGE_SIZE_A4 } from 'document-schema.js'; +import { describe, expect, it } from 'vitest'; +import { parseMarkdown } from './block/block'; +import { MarkdownDiagnosticCodes, MarkdownUnbalancedConstructMarkersError } from './diagnostics/diagnostics'; +import { emitMarkdown } from './emit/emit'; +import { readMarkdown } from './read'; +import { FOOTNOTE_REFERENCE_FONT_MARKER } from './shared/style-constants'; +import { createDiagnosticCollector } from './test-support/diagnostics'; +import { writeMarkdown } from './write'; + +function blocksOf(document: ContentDocument): ContentBlock[] { + if (document.kind !== 'wordprocessing') { + throw new Error(`expected a wordprocessing document, got '${document.kind}'`); + } + return document.sections.flatMap((section) => section.blocks); +} + +function lowered(source: string): ContentBlock[] { + return blocksOf(readMarkdown(source).document); +} + +function minimalDocument(blocks: readonly ContentBlock[]): ContentDocument { + return { kind: 'wordprocessing', metadata: {}, sections: [{ pageSize: PAGE_SIZE_A4, margins: { topPt: 72, rightPt: 72, bottomPt: 72, leftPt: 72 }, blocks: [...blocks] }] }; +} + +// One full pass through the public surface and back, twice -- see this file's own top-of-file note on why the fixed point, rather than the source text, is what a round trip is measured against here. +function roundTrip(source: string): { readonly written: string; readonly rewritten: string; readonly document: ContentDocument; readonly reread: ContentDocument } { + const document = readMarkdown(source).document; + const written = writeMarkdown(document); + const reread = readMarkdown(written).document; + return { written, rewritten: writeMarkdown(reread), document, reread }; +} + +describe('reading footnote definitions', () => { + it('parses a definition as its own block node carrying its label and its body', () => { + expect(parseMarkdown('[^1]: The note.').document.children).toEqual([ + { type: 'footnoteDefinition', label: '1', children: [{ type: 'paragraph', children: [{ type: 'text', value: 'The note.' }] }] }, + ]); + }); + + it('collects every definition label into the document-global set, before any inline is parsed', () => { + expect([...parseMarkdown('[^a]: one\n\n[^b]: two').footnotes]).toEqual(['a', 'b']); + }); + + it('continues a definition body across further indented blocks', () => { + const [definition] = parseMarkdown('[^1]: First.\n\n Second.\n\n - item').document.children; + expect(definition).toEqual({ + type: 'footnoteDefinition', + label: '1', + children: [ + { type: 'paragraph', children: [{ type: 'text', value: 'First.' }] }, + { type: 'paragraph', children: [{ type: 'text', value: 'Second.' }] }, + { type: 'list', markerType: 'bullet', bulletMarker: '-', tight: true, children: [{ type: 'listItem', children: [{ type: 'paragraph', children: [{ type: 'text', value: 'item' }] }] }] }, + ], + }); + }); + + it('ends a definition at a blank line followed by unindented content', () => { + expect(parseMarkdown('[^1]: note\n\nafter').document.children).toEqual([ + { type: 'footnoteDefinition', label: '1', children: [{ type: 'paragraph', children: [{ type: 'text', value: 'note' }] }] }, + { type: 'paragraph', children: [{ type: 'text', value: 'after' }] }, + ]); + }); + + it('continues a definition\'s own paragraph lazily, exactly as a list item does', () => { + expect(parseMarkdown('[^1]: note\nsame paragraph').document.children).toEqual([ + { + type: 'footnoteDefinition', + label: '1', + children: [{ type: 'paragraph', children: [{ type: 'text', value: 'note' }, { type: 'softBreak' }, { type: 'text', value: 'same paragraph' }] }], + }, + ]); + }); + + it('accepts a definition with no body at all', () => { + expect(parseMarkdown('[^1]:').document.children).toEqual([{ type: 'footnoteDefinition', label: '1', children: [] }]); + }); + + it('does not let a definition interrupt a paragraph', () => { + expect(parseMarkdown('prose\n[^1]: not a definition').document.children).toEqual([ + { type: 'paragraph', children: [{ type: 'text', value: 'prose' }, { type: 'softBreak' }, { type: 'text', value: '[^1]: not a definition' }] }, + ]); + }); + + it('does not recognise a definition inside a block quote or a list item', () => { + // Both would put the construct pair's own extent inside a scope the enclosing container opened -- see src/block/block.ts's tryFootnoteDefinitionStart for why that is the one thing the marker contract forbids a producer from emitting. The text stays an ordinary paragraph there, exactly as it did before footnotes were recognised anywhere. + expect(parseMarkdown('> [^1]: quoted note text').document.children).toEqual([ + { type: 'blockquote', children: [{ type: 'paragraph', children: [{ type: 'text', value: '[^1]: quoted note text' }] }] }, + ]); + expect(parseMarkdown('- [^1]: listed note text').document.children).toEqual([ + { type: 'list', markerType: 'bullet', bulletMarker: '-', tight: true, children: [{ type: 'listItem', children: [{ type: 'paragraph', children: [{ type: 'text', value: '[^1]: listed note text' }] }] }] }, + ]); + }); + + it('recognises a definition that follows a list, closing the still-open list rather than folding the definition into a paragraph', () => { + // continueBlock (src/block/block.ts) reports a `list` node as continued unconditionally, so the container the block-start dispatch sees here is the list itself, not the document -- tryFootnoteDefinitionStart has to walk past that before its own document-only restriction applies. Without that walk this whole shape collapses: `[^1]: note` reads as an ordinary paragraph, extractDefinitions swallows it as a LINK reference definition instead, and the note body is gone. + expect(parseMarkdown('Body[^1].\n\n- a\n- b\n\n[^1]: note').document.children).toEqual([ + { type: 'paragraph', children: [{ type: 'text', value: 'Body' }, { type: 'footnoteReference', label: '1' }, { type: 'text', value: '.' }] }, + { + type: 'list', + markerType: 'bullet', + bulletMarker: '-', + tight: true, + children: [ + { type: 'listItem', children: [{ type: 'paragraph', children: [{ type: 'text', value: 'a' }] }] }, + { type: 'listItem', children: [{ type: 'paragraph', children: [{ type: 'text', value: 'b' }] }] }, + ], + }, + { type: 'footnoteDefinition', label: '1', children: [{ type: 'paragraph', children: [{ type: 'text', value: 'note' }] }] }, + ]); + }); + + it('still refuses a definition indented into a list item\'s own content, even though a bare definition after the same list is recognised', () => { + // "[^1]: note" here is indented enough to be the item's own second paragraph, so tryFootnoteDefinitionStart's guard correctly rejects it (matchedContainer is the listItem itself, not something that walks up to the document). What is left is an ordinary paragraph holding nothing but what extractDefinitions reads as a link reference definition, which leaves no block behind at all -- the item ends up with only its first paragraph. + expect(parseMarkdown('- a\n\n [^1]: note').document.children).toEqual([ + { + type: 'list', + markerType: 'bullet', + bulletMarker: '-', + tight: true, + children: [{ type: 'listItem', children: [{ type: 'paragraph', children: [{ type: 'text', value: 'a' }] }] }], + }, + ]); + }); + + it('reports a duplicate label and keeps both definitions as written', () => { + const collector = createDiagnosticCollector(); + const parsed = parseMarkdown('[^1]: first\n\n[^1]: second', { sink: collector.sink }); + expect(collector.has(MarkdownDiagnosticCodes.DUPLICATE_FOOTNOTE_DEFINITION)).toBe(true); + expect(parsed.document.children).toHaveLength(2); + }); + + it('leaves both spellings as ordinary text when footnotes are switched off', () => { + // A multi-word body deliberately, so the line cannot be read as a LINK reference definition either (`[^1]: note` alone is `[^1]` -> `note`, which is what this package did with the whole shape before footnotes existed). + expect(parseMarkdown('a[^1]\n\n[^1]: note text', { footnotes: false }).document.children).toEqual([ + { type: 'paragraph', children: [{ type: 'text', value: 'a[^1]' }] }, + { type: 'paragraph', children: [{ type: 'text', value: '[^1]: note text' }] }, + ]); + }); +}); + +describe('reading footnote references', () => { + it('parses a reference whose label has a definition somewhere in the document', () => { + const [paragraph] = parseMarkdown('see[^1] here\n\n[^1]: note').document.children; + expect(paragraph).toEqual({ + type: 'paragraph', + children: [{ type: 'text', value: 'see' }, { type: 'footnoteReference', label: '1' }, { type: 'text', value: ' here' }], + }); + }); + + it('resolves a reference against a definition that appears later in the document', () => { + const [paragraph] = parseMarkdown('forward[^late]\n\n[^late]: defined afterwards').document.children; + expect(paragraph).toEqual({ type: 'paragraph', children: [{ type: 'text', value: 'forward' }, { type: 'footnoteReference', label: 'late' }] }); + }); + + it('leaves a label with no definition as ordinary text', () => { + expect(parseMarkdown('see[^missing] here').document.children).toEqual([{ type: 'paragraph', children: [{ type: 'text', value: 'see[^missing] here' }] }]); + }); + + it('matches labels exactly, without case folding', () => { + const [paragraph] = parseMarkdown('a[^Note] b[^note]\n\n[^note]: only the lower-case one is defined').document.children; + expect(paragraph).toEqual({ + type: 'paragraph', + children: [{ type: 'text', value: 'a[^Note] b' }, { type: 'footnoteReference', label: 'note' }], + }); + }); + + it('reads `![^1]` as an exclamation mark followed by a reference, never an image', () => { + const [paragraph] = parseMarkdown('![^1]\n\n[^1]: note').document.children; + expect(paragraph).toEqual({ type: 'paragraph', children: [{ type: 'text', value: '!' }, { type: 'footnoteReference', label: '1' }] }); + }); +}); + +describe('lowering a footnote onto the schema', () => { + it('lowers a definition to an anchor construct bracketing its own body blocks', () => { + expect(lowered('[^1]: The note.')).toEqual([ + { kind: 'constructStart', descriptor: { kind: 'anchor', anchorType: 'footnote', name: '1' } }, + { kind: 'paragraph', runs: [{ text: 'The note.' }] }, + { kind: 'constructEnd' }, + ]); + }); + + it('carries a multi-block body inside the construct extent', () => { + expect(lowered('[^1]: One.\n\n Two.')).toEqual([ + { kind: 'constructStart', descriptor: { kind: 'anchor', anchorType: 'footnote', name: '1' } }, + { kind: 'paragraph', runs: [{ text: 'One.' }] }, + { kind: 'paragraph', runs: [{ text: 'Two.' }] }, + { kind: 'constructEnd' }, + ]); + }); + + it('lowers a bodyless definition to a point anchor -- a pair with nothing between it', () => { + expect(lowered('[^1]:')).toEqual([ + { kind: 'constructStart', descriptor: { kind: 'anchor', anchorType: 'footnote', name: '1' } }, + { kind: 'constructEnd' }, + ]); + }); + + it('lowers a reference to a marked run keeping its own source spelling', () => { + const collector = createDiagnosticCollector(); + const document = readMarkdown('see[^1]\n\n[^1]: note', { sink: collector.sink }).document; + expect(blocksOf(document)[0]).toEqual({ + kind: 'paragraph', + runs: [{ text: 'see' }, { text: '[^1]', fontFamily: FOOTNOTE_REFERENCE_FONT_MARKER }], + }); + expect(collector.has(MarkdownDiagnosticCodes.FOOTNOTE_REFERENCE_PRESERVED_AS_TEXT)).toBe(true); + }); + + it('carries a reference inside a link as one run of that link', () => { + expect(blocksOf(readMarkdown('[text[^1]](/u)\n\n[^1]: note').document)[0]).toEqual({ + kind: 'paragraph', + runs: [{ text: 'text', hyperlink: '/u' }, { text: '[^1]', hyperlink: '/u', fontFamily: FOOTNOTE_REFERENCE_FONT_MARKER }], + }); + }); + + it('flattens a heading inside a definition body to literal ATX text, and says so', () => { + const collector = createDiagnosticCollector(); + const document = readMarkdown('[^1]: intro\n\n ## inner', { sink: collector.sink }).document; + expect(blocksOf(document)).toEqual([ + { kind: 'constructStart', descriptor: { kind: 'anchor', anchorType: 'footnote', name: '1' } }, + { kind: 'paragraph', runs: [{ text: 'intro' }] }, + { kind: 'paragraph', runs: [{ text: '## ' }, { text: 'inner' }] }, + { kind: 'constructEnd' }, + ]); + expect(collector.has(MarkdownDiagnosticCodes.FOOTNOTE_BODY_HEADING_FLATTENED)).toBe(true); + }); + + it('leaves every construct marker pair balanced, which is what the schema requires of a producer', () => { + const blocks = lowered('a[^x]\n\n[^x]: one\n\n two\n\n[^y]: another'); + const opens = blocks.filter((block) => block.kind === 'constructStart').length; + const closes = blocks.filter((block) => block.kind === 'constructEnd').length; + expect(opens).toBe(closes); + }); +}); + +describe('writing footnotes back out', () => { + it('renders an anchor construct as a definition, and its marked run as a reference', () => { + expect(emitMarkdown(minimalDocument([ + { kind: 'paragraph', runs: [{ text: 'see' }, { text: '[^1]', fontFamily: FOOTNOTE_REFERENCE_FONT_MARKER }] }, + { kind: 'constructStart', descriptor: { kind: 'anchor', anchorType: 'footnote', name: '1' } }, + { kind: 'paragraph', runs: [{ text: 'note' }] }, + { kind: 'constructEnd' }, + ]))).toBe('see[^1]\n\n[^1]: note'); + }); + + it('indents a multi-block body to the continuation column a reader measures against', () => { + expect(emitMarkdown(minimalDocument([ + { kind: 'constructStart', descriptor: { kind: 'anchor', anchorType: 'footnote', name: 'long' } }, + { kind: 'paragraph', runs: [{ text: 'one' }] }, + { kind: 'paragraph', runs: [{ text: 'two' }] }, + { kind: 'constructEnd' }, + ]))).toBe('[^long]: one\n\n two'); + }); + + it('renders an empty extent as the bare marker, with no trailing space', () => { + expect(emitMarkdown(minimalDocument([ + { kind: 'constructStart', descriptor: { kind: 'anchor', anchorType: 'footnote', name: '1' } }, + { kind: 'constructEnd' }, + ]))).toBe('[^1]:'); + }); + + it('escapes an ordinary run that merely looks like a reference, so it reparses as text', () => { + const written = emitMarkdown(minimalDocument([{ kind: 'paragraph', runs: [{ text: 'literal [^1] here' }] }])); + expect(written).toBe('literal \\[\\^1\\] here'); + expect(parseMarkdown(`${written}\n\n[^1]: real`).document.children[0]).toEqual({ type: 'paragraph', children: [{ type: 'text', value: 'literal [^1] here' }] }); + }); + + it('renders a construct markdown has no syntax for transparently, keeping its extent', () => { + const collector = createDiagnosticCollector(); + const written = emitMarkdown(minimalDocument([ + { kind: 'constructStart', descriptor: { kind: 'division', name: 'chapter-one' } }, + { kind: 'paragraph', runs: [{ text: 'content inside a division' }] }, + { kind: 'constructEnd' }, + ]), { sink: collector.sink }); + expect(written).toBe('content inside a division'); + expect(collector.has(MarkdownDiagnosticCodes.CONSTRUCT_UNREPRESENTED)).toBe(true); + }); + + it('degrades a footnote anchor whose own name cannot be spelled as a "[^label]:" marker, rather than emitting markdown its own reader cannot parse back', () => { + // AnchorDescriptorSchema.name is a bare z.string() -- document-schema.js places no grammar constraint of its own on it, so a name from another codec sharing the same ContentDocument pivot may carry whitespace or a "]" this package's own [^label] grammar (src/inline/footnote.ts) cannot represent. Spelling either straight into a marker would emit text this package's own reader reparses as something else entirely (a link reference definition, or a plain paragraph), losing the construct with no diagnostic -- so both fall back to the same transparent degrade an unrepresentable construct kind already gets. + const whitespaceCollector = createDiagnosticCollector(); + const whitespaceWritten = emitMarkdown(minimalDocument([ + { kind: 'constructStart', descriptor: { kind: 'anchor', anchorType: 'footnote', name: 'My Note' } }, + { kind: 'paragraph', runs: [{ text: 'body' }] }, + { kind: 'constructEnd' }, + ]), { sink: whitespaceCollector.sink }); + expect(whitespaceWritten).toBe('body'); + expect(whitespaceCollector.has(MarkdownDiagnosticCodes.CONSTRUCT_UNREPRESENTED)).toBe(true); + expect(parseMarkdown(whitespaceWritten).document.children).toEqual([{ type: 'paragraph', children: [{ type: 'text', value: 'body' }] }]); + + const bracketCollector = createDiagnosticCollector(); + const bracketWritten = emitMarkdown(minimalDocument([ + { kind: 'constructStart', descriptor: { kind: 'anchor', anchorType: 'footnote', name: 'a]b' } }, + { kind: 'paragraph', runs: [{ text: 'body' }] }, + { kind: 'constructEnd' }, + ]), { sink: bracketCollector.sink }); + expect(bracketWritten).toBe('body'); + expect(bracketCollector.has(MarkdownDiagnosticCodes.CONSTRUCT_UNREPRESENTED)).toBe(true); + }); + + it('renders a nested construct inside a footnote body', () => { + expect(emitMarkdown(minimalDocument([ + { kind: 'constructStart', descriptor: { kind: 'anchor', anchorType: 'footnote', name: '1' } }, + { kind: 'constructStart', descriptor: { kind: 'anchor', anchorType: 'bookmark', name: 'mark' } }, + { kind: 'paragraph', runs: [{ text: 'bookmarked note' }] }, + { kind: 'constructEnd' }, + { kind: 'constructEnd' }, + ]))).toBe('[^1]: bookmarked note'); + }); + + it('throws rather than guessing when the markers do not pair up', () => { + expect(() => emitMarkdown(minimalDocument([{ kind: 'constructEnd' }]))).toThrow(MarkdownUnbalancedConstructMarkersError); + expect(() => emitMarkdown(minimalDocument([{ kind: 'constructStart', descriptor: { kind: 'anchor', anchorType: 'footnote', name: '1' } }]))).toThrow(MarkdownUnbalancedConstructMarkersError); + }); +}); + +describe('round trip', () => { + const sources = [ + 'Text with a note[^1] here.\n\n[^1]: The note body.', + 'A[^a] and B[^b].\n\n[^a]: First.\n\n[^b]: Second.', + 'See[^long].\n\n[^long]: First paragraph.\n\n Second paragraph.\n\n ```\n code\n ```', + '[^1]:', + '# Heading\n\nBody[^n].\n\n[^n]: note with *emphasis*, `code`, and a [link](/u).', + '[^1]: body\n\n - a\n - b', + 'Escaped \\[^1\\] stays literal.\n\n[^1]: while this one is real.', + 'Unmatched [^nope] stays literal text.', + 'Body[^1].\n\n- a\n- b\n\n[^1]: note', + ]; + + it.each(sources)('reaches a fixed point for %j', (source) => { + const { written, rewritten, document, reread } = roundTrip(source); + expect(rewritten).toBe(written); + expect(reread).toEqual(document); + }); + + it('keeps a definition body that a plain reparse would otherwise flatten into the surrounding flow', () => { + const { written } = roundTrip('intro[^1]\n\n[^1]: first\n\n second\n\nafter the note'); + expect(written).toBe('intro[^1]\n\n[^1]: first\n\n second\n\nafter the note'); + expect(blocksOf(readMarkdown(written).document).map((block) => block.kind)).toEqual(['paragraph', 'constructStart', 'paragraph', 'paragraph', 'constructEnd', 'paragraph']); + }); +}); diff --git a/src/html/render.ts b/src/html/render.ts index 1124bff..ddd6880 100644 --- a/src/html/render.ts +++ b/src/html/render.ts @@ -99,6 +99,9 @@ function renderInline(node: MarkdownInlineNode): string { return '
\n'; case 'softBreak': return '\n'; + case 'footnoteReference': + // Footnotes (ExaDev/markdown-codec#66) are a GitHub extension outside both CommonMark and the GFM spec document itself, so -- exactly as for the math case below -- neither vendored corpus this renderer exists to check against carries a footnote example, and there is no cmark-produced expected HTML to match. GitHub's own rendering (a superscripted `` plus a generated back-reference in a trailing notes section) is deliberately NOT reproduced: it is a whole-document transformation with its own id-minting rules, none of which any fixture here pins down. The reference's own source spelling is emitted instead, matching what the real write path (src/emit/inline.ts's renderLeaf) actually produces for the same node. + return escapeHtml(`[^${node.label}]`); case 'mathInline': // Math (ExaDev/markdown-codec#53) is a Pandoc/GFM extension outside CommonMark/GFM proper -- neither vendored corpus this renderer exists to check against (src/conformance.test.ts, src/gfm-conformance.test.ts) carries a math example, so there is no cmark-produced expected HTML to match here. The \( \) delimiters are reconstructed around the escaped literal -- matching what the real write path (src/emit/inline.ts's renderLeaf) actually produces -- so an EMPTY math span (a genuine, if unlikely, corpus edge case: two backslash escapes sitting directly adjacent, e.g. "\(\)") renders as "\(\)" here too rather than as nothing, keeping this internal oracle consistent with the real writer it exists to cross-check other constructs against. return `\\(${escapeHtml(node.literal)}\\)`; @@ -174,6 +177,13 @@ class HtmlRenderer { this.out += `$$\n${escapeHtml(node.literal)}\n$$\n`; this.cr(); return; + case 'footnoteDefinition': + // See renderInline's own footnoteReference case: no fixture pins GitHub's own notes-section markup down, so the definition's source spelling is reconstructed around its rendered body, matching src/emit/emit.ts's own renderFootnoteDefinition. The body renders as ordinary blocks -- a definition holding several paragraphs shows all of them. + this.cr(); + this.out += `${escapeHtml(`[^${node.label}]:`)}\n`; + this.render(node.children, false); + this.cr(); + return; case 'document': case 'listItem': case 'tableRow': diff --git a/src/index.ts b/src/index.ts index cec7159..a340a87 100644 --- a/src/index.ts +++ b/src/index.ts @@ -14,6 +14,7 @@ export { MarkdownInvalidUtf8Error, MarkdownNestingLimitExceededError, MarkdownParseError, + MarkdownUnbalancedConstructMarkersError, MarkdownUnsupportedDocumentKindError, MarkdownWriteError, NOOP_MARKDOWN_DIAGNOSTIC_SINK, @@ -39,6 +40,7 @@ export type { ListNumIdInfo, ListNumIdMintOptions, NumIdMintState } from './shar export { createNumIdMintState, mintedListType, mintListNumId, parseListNumId } from './shared/list-id'; export { CODE_BLOCK_STYLE_ID, + FOOTNOTE_REFERENCE_FONT_MARKER, headingStyleId, HORIZONTAL_RULE_STYLE_ID, HTML_PREFORMATTED_STYLE_ID, diff --git a/src/inline/footnote.ts b/src/inline/footnote.ts new file mode 100644 index 0000000..d5abc81 --- /dev/null +++ b/src/inline/footnote.ts @@ -0,0 +1,54 @@ +// Footnote-syntax primitives shared by the two phases that must agree exactly or a reference silently stops resolving against its own definition: the block phase's definition scanning (`[^label]: body`, src/block/block.ts's tryFootnoteDefinitionStart) and the inline phase's reference recognition (`[^label]`, src/inline/inline.ts). Lives beside src/inline/link.ts for the same reason that module does -- the label grammar itself is inline-shaped, while WHEN a definition is recognised is a block-structure question -- and the block phase imports from here exactly as src/block/definitions.ts already imports its own destination/title grammar from src/inline/link.ts. +// +// Footnotes are a GitHub extension BEYOND the GFM spec document itself: assets/gfm/spec.txt carries no footnote section at all (it is a snapshot of the four tagged extensions -- table, strikethrough, autolink, task list), and CommonMark 0.31.2 has no footnote concept either. The grammar below is therefore transcribed from what GitHub and Pandoc both actually accept for the marker-plus-tail-definition form, which is the shape both agree on: `[^` then an identifier carrying no whitespace and no further brackets, then `]`. +// +// Pandoc's inline note form (`^[note text here]`) is deliberately NOT recognised here: it has no GitHub analogue, and the ContentDocument mapping this package lowers onto (an anchor construct wrapping the note's own block extent, src/lower/lower.ts) has nowhere to put a note whose body sits inline in the middle of a paragraph -- exactly the run-level extent gap that already keeps the REFERENCE site from being an anchor construct of its own (see src/lower/inline.ts's own footnoteReference case). + +// The identifier between `[^` and `]`: at least one character, none of them whitespace and none of them a further square bracket. Both restrictions are load-bearing rather than tidiness -- a label containing whitespace is ambiguous with ordinary bracketed prose ("[^ see above]"), and one containing a bracket cannot be scanned without a nesting rule neither GitHub nor Pandoc defines. +const FOOTNOTE_LABEL_PATTERN = /\[\^([^\s[\]]+)\]/y; + +// The same identifier grammar as FOOTNOTE_LABEL_PATTERN's own capture group, anchored to the whole string rather than embedded in a `[^label]` match -- isValidFootnoteLabel below tests a candidate label in isolation, not a marker it was scanned out of. +const FOOTNOTE_LABEL_ONLY_PATTERN = /^[^\s[\]]+$/; + +export interface FootnoteLabelMatch { + readonly label: string; + // Source index one past the closing `]`. + readonly end: number; +} + +// Matches `[^label]` starting at `start` (which must be the `[`), or undefined when what follows is not a footnote label at all. +export function matchFootnoteLabel(text: string, start: number): FootnoteLabelMatch | undefined { + FOOTNOTE_LABEL_PATTERN.lastIndex = start; + const match = FOOTNOTE_LABEL_PATTERN.exec(text); + if (match === null) { + return undefined; + } + const label = match[1]; + if (label === undefined) { + return undefined; + } + return { label, end: start + match[0].length }; +} + +export interface FootnoteDefinitionMatch { + readonly label: string; + // How many characters of `lineText` the `[^label]:` marker itself occupies -- what the block phase advances its own line cursor past before the rest of the line becomes the definition's first line of content. Deliberately excludes any spaces after the colon: the block phase's own openNewBlocks skips those as ordinary leading whitespace, exactly as it does after a list item's marker. + readonly markerLength: number; +} + +// Matches a footnote DEFINITION's own opening marker (`[^label]:`) at the very start of `lineText`, which the block phase hands in already positioned at the line's first non-space character. +export function matchFootnoteDefinitionMarker(lineText: string): FootnoteDefinitionMatch | undefined { + const match = matchFootnoteLabel(lineText, 0); + if (match === undefined || lineText.charAt(match.end) !== ':') { + return undefined; + } + return { label: match.label, markerLength: match.end + 1 }; +} + +// Labels are matched EXACTLY, with no case folding and no whitespace collapsing -- unlike a link label (src/inline/link.ts's normalizeLinkLabel, which implements CommonMark's own normalisation rules verbatim). There is no spec text to transcribe here: footnotes are outside both CommonMark and GFM proper, so a normalisation rule would be this package's own invention, and an exact match is the one choice that cannot silently merge two labels an author meant to keep apart. `[^Note]` and `[^note]` are therefore two distinct footnotes, and each round-trips under its own spelling. +export type FootnoteLabelSet = ReadonlySet; + +// Whether `label` could itself appear, unchanged, between `[^` and `]` and be read back as the same footnote. src/emit/emit.ts calls this before spelling a footnote anchor's own name into a `[^label]:` marker: the name comes from document-schema.js's AnchorDescriptorSchema, a bare `z.string()` with no grammar constraint of its own, so it may arrive from a producer other than this package's own reader (ooxml.js, odf.js, or any other codec sharing the same ContentDocument pivot) carrying whitespace or a `]` this grammar cannot represent. Spelling such a name into a marker anyway would emit text this package's own reader -- and GitHub's -- cannot parse back as a footnote at all: whitespace reopens the "[^ see above]" ambiguity FOOTNOTE_LABEL_PATTERN's own comment describes, and a `]` ends the marker early, degrading the rest of the line into ordinary paragraph text or a link reference definition. The write side is expected to fall back to its transparent unrepresented-construct degrade when this returns false, rather than emit unparseable syntax. +export function isValidFootnoteLabel(label: string): boolean { + return FOOTNOTE_LABEL_ONLY_PATTERN.test(label); +} diff --git a/src/inline/inline.test.ts b/src/inline/inline.test.ts index a43066e..a7fb7ce 100644 --- a/src/inline/inline.test.ts +++ b/src/inline/inline.test.ts @@ -2,14 +2,16 @@ import { describe, expect, it } from 'vitest'; import type { MarkdownInlineNode } from '../ast/ast'; +import type { FootnoteLabelSet } from './footnote'; import { parseInlines } from './inline'; import type { LinkReferenceDefinition, LinkReferenceMap } from './link'; const NO_REFERENCES: LinkReferenceMap = new Map(); +const NO_FOOTNOTES: FootnoteLabelSet = new Set(); const COMMONMARK_ONLY = { gfmAutolinks: false, gfmStrikethrough: false }; function parse(source: string, references: LinkReferenceMap = NO_REFERENCES): MarkdownInlineNode[] { - return parseInlines(source, references, COMMONMARK_ONLY); + return parseInlines(source, references, NO_FOOTNOTES, COMMONMARK_ONLY); } describe('text, escapes, and character references', () => { @@ -187,16 +189,16 @@ describe('line breaks', () => { describe('GFM strikethrough', () => { it('matches one or two tildes through the shared delimiter stack', () => { - expect(parseInlines('~~a~~', NO_REFERENCES)).toEqual([{ type: 'strikethrough', children: [{ type: 'text', value: 'a' }] }]); - expect(parseInlines('~a~', NO_REFERENCES)).toEqual([{ type: 'strikethrough', children: [{ type: 'text', value: 'a' }] }]); + expect(parseInlines('~~a~~', NO_REFERENCES, NO_FOOTNOTES)).toEqual([{ type: 'strikethrough', children: [{ type: 'text', value: 'a' }] }]); + expect(parseInlines('~a~', NO_REFERENCES, NO_FOOTNOTES)).toEqual([{ type: 'strikethrough', children: [{ type: 'text', value: 'a' }] }]); }); it('requires the opening and closing runs to be the same length', () => { - expect(parseInlines('~~a~', NO_REFERENCES)).toEqual([{ type: 'text', value: '~~a~' }]); + expect(parseInlines('~~a~', NO_REFERENCES, NO_FOOTNOTES)).toEqual([{ type: 'text', value: '~~a~' }]); }); it('treats a run of three or more tildes as literal text', () => { - expect(parseInlines('~~~a~~~', NO_REFERENCES)).toEqual([{ type: 'text', value: '~~~a~~~' }]); + expect(parseInlines('~~~a~~~', NO_REFERENCES, NO_FOOTNOTES)).toEqual([{ type: 'text', value: '~~~a~~~' }]); }); it('leaves every tilde literal when the extension is disabled', () => { @@ -204,7 +206,7 @@ describe('GFM strikethrough', () => { }); it('resolves emphasis nested inside strikethrough', () => { - expect(parseInlines('~~*a*~~', NO_REFERENCES)).toEqual([ + expect(parseInlines('~~*a*~~', NO_REFERENCES, NO_FOOTNOTES)).toEqual([ { type: 'strikethrough', children: [{ type: 'emphasis', marker: '*', children: [{ type: 'text', value: 'a' }] }] }, ]); }); @@ -212,7 +214,7 @@ describe('GFM strikethrough', () => { describe('GFM extended autolinks', () => { it('links a www-prefixed run, prepending the scheme to the destination only', () => { - expect(parseInlines('see www.example.com now', NO_REFERENCES)).toEqual([ + expect(parseInlines('see www.example.com now', NO_REFERENCES, NO_FOOTNOTES)).toEqual([ { type: 'text', value: 'see ' }, { type: 'link', destination: 'http://www.example.com', children: [{ type: 'text', value: 'www.example.com' }] }, { type: 'text', value: ' now' }, @@ -220,13 +222,13 @@ describe('GFM extended autolinks', () => { }); it('links a bare http(s) run', () => { - expect(parseInlines('https://example.com/a', NO_REFERENCES)).toEqual([ + expect(parseInlines('https://example.com/a', NO_REFERENCES, NO_FOOTNOTES)).toEqual([ { type: 'link', destination: 'https://example.com/a', children: [{ type: 'text', value: 'https://example.com/a' }] }, ]); }); it('trims trailing punctuation and an unbalanced closing parenthesis', () => { - expect(parseInlines('(https://example.com/a).', NO_REFERENCES)).toEqual([ + expect(parseInlines('(https://example.com/a).', NO_REFERENCES, NO_FOOTNOTES)).toEqual([ { type: 'text', value: '(' }, { type: 'link', destination: 'https://example.com/a', children: [{ type: 'text', value: 'https://example.com/a' }] }, { type: 'text', value: ').' }, @@ -234,36 +236,36 @@ describe('GFM extended autolinks', () => { }); it('links a bare email address through a mailto: destination', () => { - expect(parseInlines('mail me@example.com', NO_REFERENCES)).toEqual([ + expect(parseInlines('mail me@example.com', NO_REFERENCES, NO_FOOTNOTES)).toEqual([ { type: 'text', value: 'mail ' }, { type: 'link', destination: 'mailto:me@example.com', children: [{ type: 'text', value: 'me@example.com' }] }, ]); }); it('rejects a domain with no dot', () => { - expect(parseInlines('www.example', NO_REFERENCES)).toEqual([{ type: 'text', value: 'www.example' }]); + expect(parseInlines('www.example', NO_REFERENCES, NO_FOOTNOTES)).toEqual([{ type: 'text', value: 'www.example' }]); }); it('links a bare ftp run, the third scheme GFM recognises alongside http and https', () => { - expect(parseInlines('ftp://foo.bar.baz', NO_REFERENCES)).toEqual([ + expect(parseInlines('ftp://foo.bar.baz', NO_REFERENCES, NO_FOOTNOTES)).toEqual([ { type: 'link', destination: 'ftp://foo.bar.baz', children: [{ type: 'text', value: 'ftp://foo.bar.baz' }] }, ]); }); it('drops a trailing dot from an email address but rejects one ending in a hyphen or underscore outright', () => { - expect(parseInlines('a.b-c_d@a.b.', NO_REFERENCES)).toEqual([ + expect(parseInlines('a.b-c_d@a.b.', NO_REFERENCES, NO_FOOTNOTES)).toEqual([ { type: 'link', destination: 'mailto:a.b-c_d@a.b', children: [{ type: 'text', value: 'a.b-c_d@a.b' }] }, { type: 'text', value: '.' }, ]); - expect(parseInlines('a.b-c_d@a.b-', NO_REFERENCES)).toEqual([{ type: 'text', value: 'a.b-c_d@a.b-' }]); - expect(parseInlines('a.b-c_d@a.b_', NO_REFERENCES)).toEqual([{ type: 'text', value: 'a.b-c_d@a.b_' }]); + expect(parseInlines('a.b-c_d@a.b-', NO_REFERENCES, NO_FOOTNOTES)).toEqual([{ type: 'text', value: 'a.b-c_d@a.b-' }]); + expect(parseInlines('a.b-c_d@a.b_', NO_REFERENCES, NO_FOOTNOTES)).toEqual([{ type: 'text', value: 'a.b-c_d@a.b_' }]); }); it('never creates an autolink inside an existing link or code span', () => { - expect(parseInlines('[www.example.com](/u)', NO_REFERENCES)).toEqual([ + expect(parseInlines('[www.example.com](/u)', NO_REFERENCES, NO_FOOTNOTES)).toEqual([ { type: 'link', destination: '/u', children: [{ type: 'text', value: 'www.example.com' }] }, ]); - expect(parseInlines('`www.example.com`', NO_REFERENCES)).toEqual([{ type: 'codeSpan', literal: 'www.example.com' }]); + expect(parseInlines('`www.example.com`', NO_REFERENCES, NO_FOOTNOTES)).toEqual([{ type: 'codeSpan', literal: 'www.example.com' }]); }); it('leaves a bare URL as plain text when the extension is disabled', () => { diff --git a/src/inline/inline.ts b/src/inline/inline.ts index 12a197f..7c6ef2f 100644 --- a/src/inline/inline.ts +++ b/src/inline/inline.ts @@ -16,6 +16,8 @@ import { containsAsciiControlOrSpace, isAsciiPunctuation } from './chars'; import type { Delimiter, DelimiterChar } from './delimiter'; import { DelimiterStack, isDelimiterChar, processEmphasis, scanDelimiterRun } from './delimiter'; import { matchEntity } from './entity'; +import type { FootnoteLabelSet } from './footnote'; +import { matchFootnoteLabel } from './footnote'; import { applyGfmAutolinks } from './gfm-autolink'; import type { LinkReferenceMap, ParsedSpan } from './link'; import { matchLinkLabel, normalizeLinkLabel, parseLinkDestination, parseLinkTitle, skipInlineWhitespace } from './link'; @@ -83,15 +85,17 @@ function createWrapper(kind: 'emphasis' | 'strong' | 'strikethrough', marker: De class InlineParser { private readonly text: string; private readonly references: LinkReferenceMap; + private readonly footnotes: FootnoteLabelSet; private readonly gfmStrikethrough: boolean; private readonly container = new InlineNode('container'); private readonly delimiters = new DelimiterStack(); private brackets: Bracket | undefined; private pos = 0; - constructor(text: string, references: LinkReferenceMap, options: InlineParseOptions) { + constructor(text: string, references: LinkReferenceMap, footnotes: FootnoteLabelSet, options: InlineParseOptions) { this.text = text; this.references = references; + this.footnotes = footnotes; this.gfmStrikethrough = options.gfmStrikethrough ?? true; } @@ -336,12 +340,31 @@ class InlineParser { this.brackets = { node, previous: this.brackets, previousDelimiter: this.delimiters.top, index, image, active: true, bracketAfter: false }; } + // A `[` opens a link/image bracket -- unless it opens a footnote reference instead. That check runs FIRST and consumes the whole `[^label]` outright rather than pushing a bracket, for the same reason a code span binds tighter than everything after it: a reference is a single indivisible token, and letting the `[` reach the bracket stack would leave the label's own `^` and text as ordinary inline content that emphasis resolution could reach into. + // + // A label with no matching DEFINITION in this document is deliberately not a reference at all -- GitHub's own reading, and the one that keeps ordinary bracketed prose ("[^2 is the exponent]" -- well, that one has whitespace, but "[^see]" in a document with no `[^see]:` line does not) from silently becoming a note pointing at nothing. This is exactly how a shortcut link reference already behaves one function down: no definition, no link. private parseOpenBracket(): void { const start = this.pos; + const footnote = this.matchFootnoteReference(); + if (footnote !== undefined) { + const node = new InlineNode('footnoteReference'); + node.label = footnote.label; + this.container.appendChild(node); + this.pos = footnote.end; + return; + } this.pos += 1; this.pushBracket(this.appendText('['), start, false); } + private matchFootnoteReference(): { readonly label: string; readonly end: number } | undefined { + const match = matchFootnoteLabel(this.text, this.pos); + if (match === undefined || !this.footnotes.has(match.label)) { + return undefined; + } + return match; + } + private parseBang(): void { const start = this.pos; this.pos += 1; @@ -349,6 +372,11 @@ class InlineParser { this.appendText('!'); return; } + // `![^label]` is an exclamation mark followed by a footnote reference, never an image whose description happens to start with a caret: the reference token is already complete before the image's own `](dest)` grammar could begin, so pushing an image bracket here would open one that can never close. Leaving the cursor on the `[` hands it straight to parseOpenBracket on the next step. + if (this.matchFootnoteReference() !== undefined) { + this.appendText('!'); + return; + } this.pos += 1; this.pushBracket(this.appendText('!['), start + 1, true); } @@ -490,6 +518,9 @@ function flattenToPlainText(node: InlineNode): string { case 'softBreak': case 'hardBreak': return ' '; + case 'footnoteReference': + // An alt attribute is plain text, so a reference inside an image description contributes its own source spelling -- the same thing every consumer that does not resolve footnotes shows for it. + return `[^${node.label}]`; default: { let result = ''; let child = node.firstChild; @@ -560,14 +591,16 @@ function toAstNode(node: InlineNode): MarkdownInlineNode | undefined { return { type: 'entity', raw: node.raw, value: node.literal }; case 'mathInline': return { type: 'mathInline', literal: node.literal }; + case 'footnoteReference': + return { type: 'footnoteReference', label: node.label }; case 'container': return undefined; } } -// Parses one block's raw inline content. `references` is the document-global link-reference-definition table the block phase built -- see this module's own top-of-file note on why it cannot be discovered here. -export function parseInlines(content: string, references: LinkReferenceMap, options: InlineParseOptions = {}): MarkdownInlineNode[] { - const root = new InlineParser(content, references, options).parse(); +// Parses one block's raw inline content. `references` is the document-global link-reference-definition table the block phase built, and `footnotes` the document-global set of footnote labels it collected alongside -- see this module's own top-of-file note on why neither can be discovered here. Both are forward-visible for the identical reason: a `[^1]` in the first paragraph resolves against a `[^1]:` definition on the last line. +export function parseInlines(content: string, references: LinkReferenceMap, footnotes: FootnoteLabelSet, options: InlineParseOptions = {}): MarkdownInlineNode[] { + const root = new InlineParser(content, references, footnotes, options).parse(); if (options.gfmAutolinks ?? true) { applyGfmAutolinks(root); mergeAdjacentText(root); diff --git a/src/inline/node.ts b/src/inline/node.ts index f7ab880..88155a7 100644 --- a/src/inline/node.ts +++ b/src/inline/node.ts @@ -18,6 +18,7 @@ export type InlineNodeKind = | 'rawHtml' | 'entity' | 'mathInline' + | 'footnoteReference' // The synthetic root every inline parse builds into -- never converted to an AST node itself, only its children are. | 'container'; @@ -33,6 +34,8 @@ export class InlineNode { marker: '_' | '*' = '*'; // An entity node's own literal source text (e.g. '&'), kept alongside `literal`'s decoded value. raw = ''; + // A footnote reference's own label -- kept in its own field rather than reusing `literal`, since a reference has no literal text of its own: `[^1]` is the label's SPELLING, reconstructed on the way out, not content the parser read. + label = ''; parent: InlineNode | undefined; firstChild: InlineNode | undefined; diff --git a/src/lower/inline.ts b/src/lower/inline.ts index fa64f6b..8e018f2 100644 --- a/src/lower/inline.ts +++ b/src/lower/inline.ts @@ -6,7 +6,7 @@ import type { ContentRun } from 'document-schema.js'; import type { MarkdownInlineNode } from '../ast/ast'; import type { MarkdownDiagnosticSink } from '../diagnostics/diagnostics'; import { MarkdownDiagnosticCodes } from '../diagnostics/diagnostics'; -import { MATH_INLINE_FONT_MARKER, MONOSPACE_FONT_FAMILY } from '../shared/style-constants'; +import { FOOTNOTE_REFERENCE_FONT_MARKER, MATH_INLINE_FONT_MARKER, MONOSPACE_FONT_FAMILY } from '../shared/style-constants'; export interface InlineLowerContext { readonly sink: MarkdownDiagnosticSink; @@ -65,6 +65,10 @@ export function lowerInlineNode(node: MarkdownInlineNode, style: RunStyle, conte // Marked with MATH_INLINE_FONT_MARKER, the same opportunistic-reuse trick a code span's own Courier New marker plays -- src/emit/inline.ts's renderLeaf reconstructs the \( \) delimiters around this run's own text (rather than escaping it as ordinary punctuation) specifically because it carries this marker, not because of anything about the text's own shape (see src/ast/ast.ts's own MarkdownMathInlineNode comment for why a text-pattern-based approach was tried and reverted). context.sink({ code: MarkdownDiagnosticCodes.MATH_INLINE_PRESERVED_AS_TEXT, severity: 'info', message: 'inline math (\\( \\)) was preserved as literal raw LaTeX text; it is not parsed as LaTeX or converted to MathML by this package' }); return [buildRun(node.literal, style, MATH_INLINE_FONT_MARKER)]; + case 'footnoteReference': + // The one half of a footnote this package cannot lower to the `anchor` construct document-schema.js defines for it: a construct's extent is block-scoped, and a reference sits inside a paragraph between two runs. The run keeps the reference's own source spelling as its text (so a consumer that knows nothing about footnotes still shows `[^1]` rather than nothing) and carries FOOTNOTE_REFERENCE_FONT_MARKER so src/emit/inline.ts's renderLeaf can tell it apart from a literal `\[^1\]` an author escaped deliberately -- see that constant's own note in src/shared/style-constants.ts for the full reasoning, and the DEFINITION half in src/lower/lower.ts for the anchor construct it does produce. + context.sink({ code: MarkdownDiagnosticCodes.FOOTNOTE_REFERENCE_PRESERVED_AS_TEXT, severity: 'info', message: `footnote reference "[^${node.label}]" is preserved as a marked text run rather than an anchor construct: a construct's extent is block-scoped, and a reference site sits between two runs inside a paragraph, which no block-level boundary marker can bracket` }); + return [buildRun(`[^${node.label}]`, style, FOOTNOTE_REFERENCE_FONT_MARKER)]; case 'autolink': { const destination = node.email ? `mailto:${node.destination}` : node.destination; return [buildRun(node.destination, { ...style, hyperlink: destination })]; diff --git a/src/lower/lower.ts b/src/lower/lower.ts index 6409f25..625a740 100644 --- a/src/lower/lower.ts +++ b/src/lower/lower.ts @@ -12,10 +12,11 @@ // - raw HTML -> preserved as literal text by default (styleId 'HTMLPreformatted' for block-level HTML), a rawHtml: 'drop' option available -- MarkdownDiagnosticCodes.RAW_HTML_PRESERVED_AS_TEXT / RAW_HTML_DROPPED. // - $$ display math / \( \) inline math (ExaDev/markdown-codec#53) -> preserved as literal raw LaTeX text (styleId 'MathBlock' for the block form; the inline form keeps its own \( \) delimiters in the run text so src/emit/inline.ts's escapeMarkdownText can recognise and pass it through unescaped -- see src/inline/math.ts) -- MarkdownDiagnosticCodes.MATH_BLOCK_PRESERVED_AS_TEXT / MATH_INLINE_PRESERVED_AS_TEXT. Never parsed as LaTeX or converted to MathML here -- that is a documents.js question (ExaDev/documents.js#563). // - front matter (src/lower/front-matter.ts) -> a flat-scalar-only LayoutMetadata subset -- MarkdownDiagnosticCodes.FRONT_MATTER_KEY_UNMAPPED. +// - footnote definition (ExaDev/markdown-codec#66) -> an `anchor` construct's boundary-marker pair (document-schema.js 4.2.0) bracketing its own lowered body blocks; the reference site is a marked run instead (src/lower/inline.ts) -- MarkdownDiagnosticCodes.FOOTNOTE_REFERENCE_PRESERVED_AS_TEXT, FOOTNOTE_BODY_HEADING_FLATTENED. See lowerFootnoteDefinition below for why the body rides the construct's extent rather than AnchorDescriptor's own `definition` field. -import type { ContentBlock, ContentDocument, ContentParagraph, ContentRun, LayoutMetadata } from 'document-schema.js'; +import type { AnchorDescriptor, ContentBlock, ContentDocument, ContentParagraph, ContentRun, LayoutMetadata } from 'document-schema.js'; import { PAGE_SIZE_A4 } from 'document-schema.js'; -import type { MarkdownBlockNode, MarkdownHeadingNode, MarkdownListItemNode, MarkdownListNode, MarkdownParagraphNode } from '../ast/ast'; +import type { MarkdownBlockNode, MarkdownFootnoteDefinitionNode, MarkdownHeadingNode, MarkdownListItemNode, MarkdownListNode, MarkdownParagraphNode } from '../ast/ast'; import type { MarkdownParseOptions, ParsedMarkdown } from '../block/block'; import { parseMarkdown } from '../block/block'; import { DEFAULT_FRONT_MATTER, DEFAULT_MARGINS, DEFAULT_RAW_HTML_MODE } from '../defaults/defaults'; @@ -222,6 +223,42 @@ function lowerList(node: MarkdownListNode, ancestorNumId: string | undefined, le return node.children.flatMap((item) => lowerListItem(item, numId, level, context, contentWidthPt)); } +// Rewrites every heading inside a footnote definition's own body into an ordinary paragraph carrying the heading's ATX spelling as leading literal text, recursively through the containers a body may hold. +// +// This is the one thing a footnote body cannot carry, and the reason is the construct boundary markers' own binding contract rather than anything about markdown: a marker pair's extent may not cross a heading-group scope boundary, and a heading INSIDE the extent both closes whatever heading scope was open outside it when the pair started (a `# H` in a footnote written under a `## Section`) and opens one that would still be standing at the closing marker. document-schema.js states that a producer must never emit such a pair, and that decompose rejects rather than repairs one -- so the choice here is between emitting a pair no consumer may accept and carrying the heading as text. The text form round-trips: `#` is escaped on the way out and unescaped identically on the way back in, so a second pass through this pipeline reproduces the same document. +// +// Deliberately unconditional rather than "only when a shallower heading is actually open outside": the level comparison would make one footnote's fidelity depend on which heading happens to precede it, so the same body would lower two different ways in two documents. A heading inside a footnote is a degenerate shape in the first place; a single, position-independent rule is the one a consumer can reason about. +function flattenFootnoteBodyHeadings(node: MarkdownBlockNode, context: BlockLowerContext): MarkdownBlockNode { + switch (node.type) { + case 'heading': + context.sink({ code: MarkdownDiagnosticCodes.FOOTNOTE_BODY_HEADING_FLATTENED, severity: 'info', message: `a level-${String(node.level)} heading inside a footnote definition's body is carried as literal ATX text: a construct boundary marker's extent may not contain a block that opens or closes a heading scope, so the heading cannot stay a heading inside the anchor construct the definition lowers to` }); + return { type: 'paragraph', children: [{ type: 'text', value: `${'#'.repeat(node.level)} ` }, ...node.children] }; + case 'blockquote': + return { type: 'blockquote', children: node.children.map((child) => flattenFootnoteBodyHeadings(child, context)) }; + case 'list': + return { ...node, children: node.children.map((item) => flattenFootnoteBodyHeadingsInItem(item, context)) }; + case 'listItem': + return flattenFootnoteBodyHeadingsInItem(node, context); + default: + return node; + } +} + +function flattenFootnoteBodyHeadingsInItem(item: MarkdownListItemNode, context: BlockLowerContext): MarkdownListItemNode { + return { ...item, children: item.children.map((child) => flattenFootnoteBodyHeadings(child, context)) }; +} + +// A footnote definition becomes an `anchor` construct: a constructStart carrying the descriptor, the definition's own lowered body blocks, and a constructEnd -- document-schema.js 4.2.0's flat-form encoding of the construct group its package tree already had. +// +// Why the body rides the construct's EXTENT rather than AnchorDescriptor's own `definition` field: that field is documented as "the definitions-table key holding this marker's body", and a definitions table is a DocumentPackage root field. A flat ContentDocument -- the only shape any codec in this family produces -- has no root to carry one, so there is no key to name and the field stays absent. The extent is not a workaround for that: a footnote body is genuinely block content (several paragraphs, a code block, a table), which a string field could not have held either way, and AnchorDescriptor's own note says outright that a ranged anchor "wraps the blocks it spans". A consumer that later factors these documents into a package is free to move the body into a definitions entry and populate `definition` then; nothing here has to be undone for it to. +// +// A definition with an empty body (`[^1]:` and nothing else) lowers to a pair with no blocks between the markers -- the point anchor the same descriptor note describes, not a special case. +function lowerFootnoteDefinition(node: MarkdownFootnoteDefinitionNode, context: BlockLowerContext, contentWidthPt: number): ContentBlock[] { + const descriptor: AnchorDescriptor = { kind: 'anchor', anchorType: 'footnote', name: node.label }; + const body = node.children.flatMap((child) => lowerBlock(flattenFootnoteBodyHeadings(child, context), context, contentWidthPt)); + return [{ kind: 'constructStart', descriptor }, ...body, { kind: 'constructEnd' }]; +} + function lowerBlock(node: MarkdownBlockNode, context: BlockLowerContext, contentWidthPt: number): ContentBlock[] { switch (node.type) { case 'paragraph': @@ -240,6 +277,8 @@ function lowerBlock(node: MarkdownBlockNode, context: BlockLowerContext, content return lowerHtmlBlock(node, context); case 'mathBlock': return lowerMathBlock(node, context); + case 'footnoteDefinition': + return lowerFootnoteDefinition(node, context, contentWidthPt); case 'table': { if (context.list !== undefined) { context.sink({ code: MarkdownDiagnosticCodes.LIST_ITEM_BLOCK_UNLISTED, severity: 'info', message: 'a table directly inside a list item has no ContentListMembership field of its own -- only ContentParagraph carries .list -- so its association with the enclosing list item is lost' }); @@ -297,6 +336,7 @@ export function lowerMarkdown(source: string, options: ReadMarkdownOptions = {}) gfmAutolinks: options.gfmAutolinks, gfmStrikethrough: options.gfmStrikethrough, gfmTaskLists: options.gfmTaskLists, + footnotes: options.footnotes, maxNesting: options.maxBlockNesting, sink, }; diff --git a/src/options/options.ts b/src/options/options.ts index 51b78a9..bce011e 100644 --- a/src/options/options.ts +++ b/src/options/options.ts @@ -23,6 +23,8 @@ export interface ReadMarkdownOptions { readonly gfmAutolinks?: boolean; readonly gfmStrikethrough?: boolean; readonly gfmTaskLists?: boolean; + // GitHub's footnote extension (`[^label]` markers with `[^label]: body` definitions). Defaults to true alongside the four toggles above; src/conformance.test.ts switches it off with them, since neither CommonMark nor the GFM spec document itself defines footnotes at all. + readonly footnotes?: boolean; // Throws MarkdownInputTooLargeError (src/diagnostics) rather than scanning input beyond this many bytes. readonly maxInputBytes?: number; // Throws MarkdownNestingLimitExceededError (src/diagnostics) rather than recursing past this many levels of block nesting (blockquote-in-list-in-blockquote, etc.). diff --git a/src/shared/style-constants.ts b/src/shared/style-constants.ts index 4b3c838..5586986 100644 --- a/src/shared/style-constants.ts +++ b/src/shared/style-constants.ts @@ -44,6 +44,13 @@ export const MONOSPACE_FONT_FAMILY = 'Courier New'; // An inline math run's own ContentRun.fontFamily marker (ExaDev/markdown-codec#53), the identical opportunistic-reuse trick MONOSPACE_FONT_FAMILY already plays for a code span -- a real Word/LibreOffice math font (the one Word itself uses for OOXML equation runs), not an invented sentinel, so a run genuinely styled this way for an unrelated reason degrades exactly as gracefully as a genuinely-monospace non-code-span run already does. Load-bearing, not cosmetic: escapeMarkdownText (src/emit/inline.ts) backslash-escapes every literal '(' and ')' in ORDINARY text, so an inline math run's own \( \) delimiters cannot be recovered by pattern-matching the written-out text after the fact (that was tried and reverted -- any ordinary parenthetical remark escapes to the identical \(...\) shape and would be misrecognised as math on reparse); this marker is what lets src/emit/inline.ts's own renderLeaf single out a genuine preserved-math run and skip escaping for that one run only. export const MATH_INLINE_FONT_MARKER = 'Cambria Math'; +// A footnote REFERENCE run's own ContentRun.fontFamily marker (ExaDev/markdown-codec#66) -- the third use of the same opportunistic-reuse trick MONOSPACE_FONT_FAMILY and MATH_INLINE_FONT_MARKER already play, and the one that needs its reasoning stated most carefully, because unlike those two it marks a SEMANTIC fact rather than a formatting one. +// +// Why a run marker at all rather than the `anchor` construct document-schema.js 4.2.0 defines for exactly this concept: a construct's extent is block-scoped by that schema's own definition ("a construct group wraps BLOCK-scoped extents... it does not wrap a sub-sequence of one paragraph's runs, because a run-level extent is not expressible without changing ContentParagraph's own shape"), and a footnote reference sits BETWEEN two runs inside a paragraph. The schema names this gap itself and parks the inline anchor case on a run-level extent mechanism it has not shipped: "the inline field/bookmark/tracked-change cases wait on a run-level extent mechanism rather than being forced into a block wrapper that would split the paragraph they sit inside". So the reference site is carried here, diagnosed as MarkdownDiagnosticCodes.FOOTNOTE_REFERENCE_PRESERVED_AS_TEXT, until that mechanism exists; the DEFINITION half, which is genuinely block-scoped, does become a real anchor construct (src/lower/lower.ts). +// +// Why fontFamily specifically: it is the only free string field ContentRun carries (there is no run-level styleId), and the marker is load-bearing rather than cosmetic for the identical reason MATH_INLINE_FONT_MARKER is -- escapeMarkdownText (src/emit/inline.ts) backslash-escapes `[`, `^`, and `]`, so a genuine reference and a literal `\[^1\]` an author escaped on purpose are the same run text by the time they reach the writer, and only a non-pattern-based marker tells them apart. The value is Word's own built-in CHARACTER STYLE name for exactly this run rather than a font name, because no producer anywhere spells a footnote reference with a distinct font (docx uses `w:rStyle w:val="FootnoteReference"` plus superscript, ODF a `Footnote_20_Symbol` text style): a consumer resolving it as a font finds none and falls back to its own default, which is a visible, harmless degrade rather than a wrong one. +export const FOOTNOTE_REFERENCE_FONT_MARKER = 'Footnote Reference'; + // Points per level of blockquote nesting src/lower/src/emit agree on for ContentParagraph.indentLeftPt -- 0.5in, a common real-world blockquote/list indent increment (matching, e.g., Word's own default list-indent step). document-schema.js's own indentLeftPt carries no "this many quote levels" semantic of its own, so SOME fixed per-level unit has to be picked for the two directions to agree; this is that choice, made once, here. export const QUOTE_INDENT_PT = 36; diff --git a/test/workers/markdown-codec.test.ts b/test/workers/markdown-codec.test.ts index b12e845..b6fbe88 100644 --- a/test/workers/markdown-codec.test.ts +++ b/test/workers/markdown-codec.test.ts @@ -15,4 +15,9 @@ describe('markdown-codec under the Cloudflare Workers runtime', () => { const roundTripped = writeMarkdown(document); expect(roundTripped).toContain('Heading text'); }); + + it('round-trips a footnote, whose definition rides a construct boundary-marker pair', () => { + const { document } = readMarkdown('Body[^1].\n\n[^1]: The note.'); + expect(writeMarkdown(document)).toContain('[^1]: The note'); + }); });