Skip to content

Add HTML to Text tool - #5

Merged
fadeltd merged 1 commit into
mainfrom
feat/html-text
Sep 25, 2026
Merged

fadeltd merged 1 commit into
mainfrom
feat/html-text

Conversation

@fadeltd

@fadeltd fadeltd commented Sep 25, 2026

Copy link
Copy Markdown
Owner

New tool at /html-text (Web category). Paste HTML, get readable plain text back.

Why a hand-written tokenizer

  • DOMParser breaks the core/ purity rule and would need jsdom to test.
  • html-to-text pulls in a parser for something that doesn't need a tree: extracting text only requires knowing which blocks, lists, cells and hidden subtrees you're inside.

core/extract.ts is one pass over the string with a small stack of open elements. There's no new dependency, and it never throws.

Rules

Input Output
div, section, header, li, tr, … own line
p, h1–h6, blockquote, pre, table, top-level lists blank line around
li - or 1. (honours <ol start>), nested lists indented
td / th tab-separated
br / pre newline / whitespace kept verbatim
other whitespace collapsed the way a browser does; &nbsp; kept as a space
script, style, head, template, svg, math, media, comments dropped
hidden attribute, inline display:none, role="img" dropped (the last one removes icon ligatures like expand_less)

Browser-matching details:

  • Missing close tags for li, p, td/th, tr and dt/dd are closed implicitly, as the browser does.
  • Numeric references in &#128;–&#159; map to windows-1252, so &#150; becomes – rather than a C1 control character.

Entities: all of Latin-1, plus about 90 common named ones and all numeric references. Unknown names are left as they are rather than guessed. The full HTML5 table would add about 10 kB gzip for names almost nobody uses.

Options (both off by default):

  • Show link URLs: docs <https://…>. Skipped when the link text already is the URL, and for # / javascript: links.
  • Skip .hidden / .sr-only: guesses from class names, so it's opt-in.

Known gap

Layout that comes from CSS can't be recovered. In the Play Console fixture, the badge, app name and date sit in one flex row, but they come out on three lines because they're <div>s. The only way to fix that is to render the pasted HTML, which we won't do.

Checks

  • 52 tests in core/:
    • the real-world fixture
    • lists, tables, <pre>, entities, skipped subtrees, links
    • malformed input: stray <, unterminated tags and comments, stray close tags
    • a fast-check property test
  • 5.2 MB of input converts in about 70 ms.
  • store: 'idb', because pasted page source can run to several MB.
  • Entry bundle 83.6 kB gzip (budget about 90 kB). The tool's own chunk is 4.8 kB gzip.
  • Prerendered page and sitemap entry are generated from the registry. Nothing else was edited by hand.
  • pnpm test, tsc --noEmit and pnpm lint are clean. The one lint warning is in diff/MergePane.tsx, which this PR doesn't touch.

A pure tokenizer in core/ (no DOM, no new dependency) that keeps paragraph,
list and table structure, drops script/style/head/svg, hidden and role=img
subtrees, and decodes entities including the windows-1252 numeric remap.
Optional link URLs and class-based hidden heuristics.
@fadeltd
fadeltd merged commit 0bb5ef3 into main Sep 25, 2026
1 check passed
@fadeltd
fadeltd deleted the feat/html-text branch September 25, 2026 07:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant