From e8fba65188860f851bb6664ecea06257454fd60e Mon Sep 17 00:00:00 2001 From: tannevaled Date: Sun, 18 Oct 2026 21:42:09 +0200 Subject: [PATCH] The links written as HTML, which the local build could not see MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 12 of the 85 sites write their card grids as `` (md_in_html), which a Markdown-link rewrite cannot see. Hugo does not mind — an attribute is an attribute — so this reached CI, where the link checker found three dead links on go-aiquota/docs' front page. Re-converted with the fixed mkdocs2hextra, and the local gate now reads the BUILT HTML for .md hrefs as well as building it. Co-Authored-By: Claude Opus 5 --- .github/workflows/docs.yml | 164 +++++++++++++++++++------- .gitmodules | 3 + README.md | 29 +++++ docs/index.md => content/en/_index.md | 27 +++-- {docs => content/en}/architecture.md | 17 ++- {docs => content/en}/fidelity.md | 19 +-- {docs => content/en}/roadmap.md | 15 ++- hugo.yaml | 87 ++++++++++++++ mkdocs.yml | 65 ---------- requirements.txt | 2 - {docs => static}/assets/logo.svg | 0 themes/hextra | 1 + 12 files changed, 289 insertions(+), 140 deletions(-) create mode 100644 .gitmodules create mode 100644 README.md rename docs/index.md => content/en/_index.md (89%) rename {docs => content/en}/architecture.md (93%) rename {docs => content/en}/fidelity.md (91%) rename {docs => content/en}/roadmap.md (92%) create mode 100644 hugo.yaml delete mode 100644 mkdocs.yml delete mode 100644 requirements.txt rename {docs => static}/assets/logo.svg (100%) create mode 160000 themes/hextra diff --git a/.github/workflows/docs.yml b/.github/workflows/docs.yml index 4a54791..86667c3 100644 --- a/.github/workflows/docs.yml +++ b/.github/workflows/docs.yml @@ -1,51 +1,105 @@ name: docs +# Every documentation version is a separate Hugo build, published under +# https://go-webengine.github.io/docs// on the gh-pages branch, where is +# the minor version of the code the pages describe (params.docs.version in +# hugo.yaml: v0.3.1 -> 0.3), or "main" while no release is tagged. Each push to +# main replaces that version's path and leaves the others alone. Pull requests +# build and check, and publish nothing. +# +# ⛔ This replaced MkDocs Material plus mike: a Python toolchain, a +# requirements.txt and a pip install to render a static site for a pure-Go +# organisation. + on: push: branches: [main] pull_request: workflow_dispatch: -# The build needs nothing beyond the source. Write access to the repository -# is granted to the deploy job alone, below, so a pull request build cannot -# push to gh-pages. permissions: contents: read +env: + HUGO_VERSION: "0.167.0" + # sha256 of hugo_${HUGO_VERSION}_linux-amd64.tar.gz, from the release's checksums.txt. + HUGO_SHA256: "4d84519b9f619e6d4c3fb45a50157abeabeb724f859c60605f44c23def6e1169" + DOCS_URL: "https://go-webengine.github.io/docs" + jobs: - # Build on every pull request as well as on main, so a change to the docs - # or to the toolchain in requirements.txt has to prove it still builds - # before it lands. Without this the workflow ran only on main, which meant - # a dependency bump had no signal at all. build: runs-on: ubuntu-latest + outputs: + version: ${{ steps.version.outputs.version }} steps: - name: Checkout uses: actions/checkout@v7 - - - name: Set up Python - uses: actions/setup-python@v7 with: - python-version: "3.15" + # themes/hextra is the tannevaled/hextra fork, pinned to a release tag. + submodules: recursive + # The whole history: each page's date comes from git. + fetch-depth: 0 + + - name: Install Hugo ${{ env.HUGO_VERSION }} + run: | + curl -fsSL -o /tmp/hugo.tar.gz \ + "https://github.com/gohugoio/hugo/releases/download/v${HUGO_VERSION}/hugo_${HUGO_VERSION}_linux-amd64.tar.gz" + echo "${HUGO_SHA256} /tmp/hugo.tar.gz" | sha256sum -c - + sudo tar -xzf /tmp/hugo.tar.gz -C /usr/local/bin hugo + hugo version + + - name: Version + id: version + run: | + documented=$(hugo config --format json | jq -r '.params.docs.version') + if [ "${documented}" = main ]; then + version=main + else + version=$(printf '%s\n' "${documented}" | sed -nE 's/^v(0|[1-9][0-9]*)\.(0|[1-9][0-9]*)\.(0|[1-9][0-9]*)$/\1.\2/p') + fi + if [ -z "${version}" ]; then + echo "::error::params.docs.version in hugo.yaml is \"${documented}\", not vMAJOR.MINOR.PATCH or main" + exit 1 + fi + echo "version=${version}" >> "$GITHUB_OUTPUT" + echo "documented ${documented}: documentation version ${version}" + + # --panicOnWarning: a relref to a page that is not there, a deprecated + # setting or a template warning fails the build rather than scrolling by. + - name: Build + env: + HUGO_ENVIRONMENT: production + run: >- + hugo --gc --minify --panicOnWarning --printPathWarnings + --baseURL "${DOCS_URL}/${{ steps.version.outputs.version }}/" - - name: Install MkDocs Material + mike + - name: Lay out the site for the link check run: | - pip install --upgrade pip - pip install -r requirements.txt + mkdir -p _check/docs + cp -R public "_check/docs/${{ steps.version.outputs.version }}" - # --strict turns a broken link or an unknown config key into a failure - # rather than a warning nobody reads. - - name: Build the site (strict) - run: mkdocs build --strict + - name: Check links + uses: lycheeverse/lychee-action@v2 + with: + args: >- + --offline --no-progress --include-fragments + --index-files index.html + --root-dir ${{ github.workspace }}/_check + '_check/docs/**/*.html' + fail: true + + - uses: actions/upload-artifact@v7 + with: + name: site + path: public + retention-days: 7 deploy: - # Only main publishes. A pull request stops after build above: it has no - # business pushing to gh-pages. - if: github.event_name == 'push' || github.event_name == 'workflow_dispatch' + if: github.event_name != 'pull_request' && github.ref == 'refs/heads/main' needs: build - # Never two deploys at once: mike pushes gh-pages and the second is - # rejected outright. cancel-in-progress stays false -- killing a publish - # half way through is worse than making it wait. + # Never two deploys at once: both would push gh-pages and the second would + # be rejected. cancel-in-progress stays false — killing a publish half way + # through is worse than making it wait. concurrency: group: gh-pages-deploy cancel-in-progress: false @@ -53,32 +107,54 @@ jobs: permissions: contents: write steps: - - name: Checkout - uses: actions/checkout@v7 + - uses: actions/checkout@v7 with: + path: gh-pages fetch-depth: 0 - - name: Set up Python - uses: actions/setup-python@v7 + - name: Switch to gh-pages + working-directory: gh-pages + run: | + if git ls-remote --exit-code --heads origin gh-pages >/dev/null; then + git fetch -q origin gh-pages + git checkout -q -B gh-pages FETCH_HEAD + else + echo "no gh-pages branch yet: starting one" + git checkout -q --orphan gh-pages + git rm -rqf . + git clean -fdxq + fi + + - uses: actions/download-artifact@v8 with: - python-version: "3.15" + name: site + path: site - - name: Install MkDocs Material + mike + # Only the version being built is replaced; every other version stays as + # it is, and /docs/latest/ follows the newest. + - name: Publish ${{ needs.build.outputs.version }} + working-directory: gh-pages run: | - pip install --upgrade pip - pip install -r requirements.txt + version="${{ needs.build.outputs.version }}" + rm -rf "docs/${version}" + mkdir -p "docs/${version}" + cp -R ../site/. "docs/${version}/" + newest=$(ls docs | grep -E '^[0-9]+\.[0-9]+$' | sort -t. -k1,1n -k2,2n | tail -1 || true) + if [ -n "${newest}" ]; then + rm -rf docs/latest + cp -R "docs/${newest}" docs/latest + fi + touch .nojekyll - - name: Configure git for mike + - name: Push gh-pages + working-directory: gh-pages run: | - git config user.name "github-actions[bot]" + git config user.name "github-actions[bot]" git config user.email "41898282+github-actions[bot]@users.noreply.github.com" - - # mike publishes versioned docs to the gh-pages branch; GitHub Pages serves - # that branch at https://go-compressions.github.io/docs/. The version label - # tracks main as "0.1" aliased "latest"; tag-driven versions can be added - # later. /docs/ redirects to the default ("latest"). - - name: Deploy versioned docs - run: | - git fetch origin gh-pages --depth=1 || true - mike deploy --push --update-aliases 0.1 latest - mike set-default --push latest + git add -A + if git diff --cached --quiet; then + echo "nothing changed" + exit 0 + fi + git commit -q -m "Deployed ${GITHUB_SHA::7} to ${{ needs.build.outputs.version }} with Hugo ${HUGO_VERSION}" + git push origin HEAD:gh-pages diff --git a/.gitmodules b/.gitmodules new file mode 100644 index 0000000..966ed98 --- /dev/null +++ b/.gitmodules @@ -0,0 +1,3 @@ +[submodule "themes/hextra"] + path = themes/hextra + url = https://github.com/tannevaled/hextra.git diff --git a/README.md b/README.md new file mode 100644 index 0000000..32f160e --- /dev/null +++ b/README.md @@ -0,0 +1,29 @@ +# go-webengine/engine + +Versioned documentation for [go-webengine](https://github.com/go-webengine), built with +[Hugo](https://gohugo.io/) and this fleet's +[Hextra fork](https://github.com/tannevaled/hextra). Published to the +`gh-pages` branch and served at . + +⛔ What this replaced was MkDocs Material plus mike, which needed a Python +toolchain, a `requirements.txt` and a `pip install` in CI to render a static +site for a pure-Go organisation. + +## Local preview + +```bash +git submodule update --init --recursive # themes/hextra, pinned at a release tag +hugo server # http://localhost:1313 +``` + +## Versions + +Each documented version is its own build under `/docs//`, and +`/docs/latest/` follows the newest — which is what `mike` did before. The version +comes from `params.docs.version` in `hugo.yaml`: set it to the release the pages +describe (`v0.16.0` → `/docs/0.16/`), or leave it at `main` while no release is +tagged. The `docs` workflow reads it, builds with that `--baseURL`, and replaces +**only** that version's path. + +⛔ The build runs with `--panicOnWarning`, so a link to a page that is not there +fails the lane instead of 404ing for a reader. diff --git a/docs/index.md b/content/en/_index.md similarity index 89% rename from docs/index.md rename to content/en/_index.md index 281d800..29f1ce9 100644 --- a/docs/index.md +++ b/content/en/_index.md @@ -1,4 +1,9 @@ -# go-webengine +--- +title: "Home" +weight: 10 +--- + +# go-webengine {#go-webengine} A pure-Go, **`CGO_ENABLED=0`** headless web engine. Give it a URL, get back an **image of the page** — HTML/CSS layout and paint, with **JavaScript executed** @@ -9,7 +14,7 @@ It is the rendering core of the service and the [wasmdesk](https://github.com/wasmdesk) in-desktop browser: a server renders pages and streams frames (plus a click hit-map) to a thin client. -## What it renders today +## What it renders today {#what-it-renders-today} - **Full box-model layout**: block, inline, floats + clear, **flexbox**, **CSS grid**, **tables**, **`position`** (relative/absolute/fixed/sticky), margin @@ -27,10 +32,10 @@ server renders pages and streams frames (plus a click hit-map) to a thin client. against a real DOM with `fetch()`/XHR and laid-out-geometry read-back; a settle-then-render loop reflects script-driven DOM mutations in the output. -See **[Fidelity](fidelity.md)** for the honest, measured account of what matches a +See **[Fidelity]({{< relref "fidelity.md" >}})** for the honest, measured account of what matches a browser and what does not. -## Pipeline +## Pipeline {#pipeline} ``` Fetch (go-browserhttp) → Parse (x/net/html → dom) → Cascade (css: +var()/@media/dark-mode) @@ -41,7 +46,7 @@ Fetch (go-browserhttp) → Parse (x/net/html → dom) → Cascade (css: +var → image.RGBA → PNG (+ optional link hit-map) ``` -## Quickstart +## Quickstart {#quickstart} ```go import ( @@ -74,7 +79,7 @@ go run github.com/go-webengine/engine/cmd/render \ -url https://example.com/ -out out.png -w 1024 -h 768 ``` -## Security +## Security {#security} The engine fetches every resource a page names: images, stylesheets, fonts, scripts and modules, `fetch`/XHR targets and form posts. It does **not** filter @@ -113,16 +118,16 @@ for the full measured writeups. None of this changes how a well-formed page renders — only how a pathological one degrades (truncated output, never a crash or unbounded allocation). -## Where to next +## Where to next {#where-to-next} -- **[Architecture](architecture.md)** — the packages, the reuse-vs-build decision, +- **[Architecture]({{< relref "architecture.md" >}})** — the packages, the reuse-vs-build decision, and why prior pure-Go browsers were studied but not built on. -- **[Roadmap](roadmap.md)** — the phases from static render to the shipping +- **[Roadmap]({{< relref "roadmap.md" >}})** — the phases from static render to the shipping remote-browser proxy, and the remaining levers. -- **[Fidelity](fidelity.md)** — an honest, measured account of what renders like a +- **[Fidelity]({{< relref "fidelity.md" >}})** — an honest, measured account of what renders like a browser today and what does not. -## Links +## Links {#links} - Source: - Remote-browser service: diff --git a/docs/architecture.md b/content/en/architecture.md similarity index 93% rename from docs/architecture.md rename to content/en/architecture.md index bd5f744..4abf5db 100644 --- a/docs/architecture.md +++ b/content/en/architecture.md @@ -1,10 +1,15 @@ -# Architecture +--- +title: "Architecture" +weight: 20 +--- + +# Architecture {#architecture} The engine is a linear pipeline from bytes on the wire to pixels in an image. Each stage is an owned package with a narrow contract, so the pure-logic stages (cascade, layout, DOM) stay testable without a network or fonts. -## Packages +## Packages {#packages} | Package | Role | |---|---| @@ -16,7 +21,7 @@ Each stage is an owned package with a narrow contract, so the pure-logic stages | `engine` (root) | The public API — `Fetch`, `Render`, `Screenshot`, `RenderHTML`, `RenderWithLinks`, `RenderInfo` — the settle-then-render loop, image + SVG sub-resource loading, and the anchor click hit-map. | | `cmd/render` | A single-binary CLI: `render -url URL -out shot.png -w 1024 -h 768` (or `-file page.html`). | -## The `Measurer` seam +## The `Measurer` seam {#the-measurer-seam} Layout never touches a font directly. It asks a `Measurer` for advance widths and line metrics. In production the measurer is backed by go-opentype faces; in tests @@ -24,7 +29,7 @@ it is a deterministic fake, so box geometry can be asserted at **exact pixel positions** with no font files and no anti-aliasing noise. This is why the layout and cascade packages carry very high statement coverage. -## Reuse vs build +## Reuse vs build {#reuse-vs-build} Everything reused is pure-Go and permissively licensed (BSD-2/BSD-3/MIT), so the whole engine builds with `CGO_ENABLED=0` and carries no copyleft. @@ -45,7 +50,7 @@ whole engine builds with `CGO_ENABLED=0` and carries no copyleft. engine, the full box-model layout engine, the DOM binding for JS, and the paint orchestration + public API. -## Why not build on a prior pure-Go browser? +## Why not build on a prior pure-Go browser? {#why-not-build-on-a-prior-pure-go-browser} The prior pure-Go browser engine `opossum` (renamed to [`mycel`](https://github.com/psilva261/mycel)) was studied in depth. Its choice @@ -58,7 +63,7 @@ it cannot give us, so the layout/box/flow engine here is clean-room. The full prior-art verdict lives in the engine's [`SURVEY.md`](https://github.com/go-webengine/engine/blob/main/SURVEY.md). -## Portability & testing +## Portability & testing {#portability-testing} Pure Go with `CGO_ENABLED=0`; cross-built in CI for all six of Go's 64-bit targets (amd64, arm64, riscv64, loong64, ppc64le, s390x). A ratchet coverage gate diff --git a/docs/fidelity.md b/content/en/fidelity.md similarity index 91% rename from docs/fidelity.md rename to content/en/fidelity.md index b5df67a..5b1d0f7 100644 --- a/docs/fidelity.md +++ b/content/en/fidelity.md @@ -1,4 +1,9 @@ -# Fidelity +--- +title: "Fidelity" +weight: 40 +--- + +# Fidelity {#fidelity} An honest account of what the renderer matches in a browser and what it does not. The authoritative, per-page and per-phase assessment (five live pages, committed @@ -7,7 +12,7 @@ golden PNGs, before/after deltas) lives in the engine's the measured-vs-Chrome numbers in [`bench/REPORT.md`](https://github.com/go-webengine/engine/blob/main/bench/REPORT.md). -## Measured vs headless Chrome +## Measured vs headless Chrome {#measured-vs-headless-chrome} Windowed SSIM (`1.0` = identical) over the common top-left region at 1024px width; `speed×` = `chrome_ms / webengine_ms` (>1 = webengine faster). Timings include the @@ -31,7 +36,7 @@ Mean SSIM over the ten pages is **≈ 0.66**. Summed over the run, webengine too spread is wide. The Wikipedia number is confounded by runtime JS chrome and by rate-limited image thumbnails. -## Works today +## Works today {#works-today} - **Full box-model layout**: block/inline flow, floats + clear, flexbox, CSS grid, tables (with `vertical-align` on cells), `position` @@ -60,7 +65,7 @@ rate-limited image thumbnails. dynamically injected `