Migrate documentation to Fern platform#1578
Draft
ramakrishnap-nv wants to merge 24 commits into
Draft
Conversation
- fern/convert_docs.py: Python script that converts all 62 Sphinx/RST docs to Fern MDX, handles pandoc 2.9 note/warning divs → Fern callout components, resolves literalinclude → embedded code blocks, rewrites image paths to flat fern/docs/images/, strips Sphinx-only directives (toctree, install-selector, autodoc, breathe). - fern/docs.yml: full navigation tree auto-generated from RST toctrees, with NVIDIA green accent colors and API Reference section. - fern/fern.config.json: org=nvidia-cuopt, version=5.67.1. - fern/docs/pages/**: 62 converted MDX pages. - fern/docs/images/**: all doc images copied flat. - .github/workflows/fern-docs.yml: CI that re-converts RST→MDX on push and runs fern check; publishes on docs-fern-migration branch merges. - C/C++ Doxygen reference stubbed pending Fern C++ library support. fern check: 0 errors, 2 warnings (auth redirect + minor color contrast). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- fern/generators.yml: register OpenAPI spec (openapi/cuopt_spec.yaml) so fern check and fern generate --docs can serve the REST API reference - fern/openapi/cuopt_spec.yaml: committed spec generated from FastAPI app without GPU (10 endpoints, 23 validated examples) - fern/docs.yml: replace stub open-api page with a proper "api: REST API" nav entry that Fern renders as an interactive API reference - fern/extract_python_api.py: AST-based Python docstring extractor (no imports, no GPU) generating MDX for routing, LP/QP, and MIP APIs; mirrors the classes/functions listed in the Sphinx autoclass:: directives - fern/convert_docs.py: call OpenAPI generation + Python extraction from the main convert script so one command rebuilds everything - .github/workflows/fern-docs.yml: add Python deps install step for OpenAPI spec generation; watch python/ source paths for changes fern check: 0 errors, 8 warnings (7 are pre-existing OpenAPI example mismatches in the FastAPI source; 1 is WCAG contrast auto-adjusted by Fern). C/C++ Doxygen reference still stubbed pending Fern C++ library support. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The custom .. install-selector:: Sphinx directive is now fully supported
in Fern by reusing the existing vanilla JS + CSS widget unchanged:
- fern/docs/scripts/install-selector.{js,css}: copied from Sphinx _static/
- fern/docs/scripts/cuopt-install-version.js: generated at build time by
convert_docs.py from cuopt_server version (replaces the Sphinx conf.py
build-time injection of window.CUOPT_INSTALL_VERSION)
- fern/docs.yml: css/js entries load the widget and version script on
every page so the selector is available wherever the div is placed
- fern/convert_docs.py: preprocessor now emits
<div id="cuopt-install-selector" data-default-iface="..."> via RST
raw:: html instead of a plain-text note stub; version script generation
added alongside the OpenAPI spec generation
Result: install.mdx and all 4 quick-start pages (python/c/server/cli)
render the full interactive install selector with correct version numbers
and interface pre-selection, identical to the Sphinx site.
fern check: 0 errors, 8 warnings (unchanged).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ersion - Fix <Note> inside list items: un-indent opening tag to break out of list item context (MDX can't parse JSX inside list items when closing tag is at column 0) - Convert .. dropdown:: RST directives to <Accordion> MDX components in preprocessor (was becoming <div class="dropdown"> which broke fenced code blocks inside) - Add postprocessor fallback converting <div class="dropdown"> to <Accordion> for any that slip through pandoc - Add libraries: section in docs.yml and DOCS_YML_HEADER for Fern C++ (cuopt-c-api) and Python (cuopt-python-api) library docs - Add Doxyfile.fern to cpp/ (Fern's input.path for C++ parsing) - Include generated Fern Python library docs (static/python-docs) as "Python Distance Engine" folder in API Reference nav section Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
MDX's JSX parser is stricter than CommonMark and fails on patterns that
are valid in plain Markdown. All 62+ pages now pass the MDX parser.
Fixes applied in convert_docs.py post-processor and extract_python_api.py:
- Angle-bracket autolinks: <https://...> and <email@...> → proper links
- Comparison operators: `<=` in text → `<=` (MDX parses `<` as JSX)
- Curly-brace math notation: {<=, =, >=} → \{...\}
- Placeholder tags in code blocks: <CUOPT_IMAGE>, <ip> → HTML entities
- Shell redirects in code blocks: `</dev/null` → `< /dev/null`
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Fern's server-side acorn processor rejects { } even inside fenced code
blocks, causing 500 errors on convex-features.mdx and milp-features.mdx.
- RST pre-processor (step 13): convert {<=, =, >=} math notation to
(<=, =, >=) parenthesis form before pandoc — eliminates all { } from
generated MDX
- Post-processor: simplify _escape_lt_outside_code now that { } are
handled at the RST level
- Fix angle-bracket autolinks and email addresses → proper MD links
- All 62 MDX pages pass both MDX parse and Fern's acorn validation
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…enAPI fixes - Python API: prefix classes with 'class', methods/functions with 'def'; use <hr /> separators between entries for visual scanning - C API: extract_c_api.py now generates real content from headers (211 symbols); typedef entries labeled with 'typedef'; functions show bare prototype (no 'function' prefix — matches cppreference/Doxygen convention); <hr /> separators - Nav: index pages inside sections renamed to 'Overview' to eliminate duplicate section-name / page-name entries in the left nav - REST API Reference moved under Server → Server API → Server API Overview - OpenAPI spec post-processing: relax IdModel additionalProperties, fix health-endpoint examples (add error_result), fix incumbents examples (null cost/bound); prevents fern docs dev from exiting with code 1 - NVIDIA cuOpt logo SVGs added (dark/light variants) - MutationObserver-based install selector widget for Next.js SPA timing Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Converts the Sphinx/RST documentation to the Fern docs platform on the
docs-fern-migration branch. Key changes across this session:
Conversion pipeline (fern/convert_docs.py, extract_*.py):
- Fix brace corruption: remove step 13 ({}→() broad replacement) and add
prose-only { escaping that skips fenced code blocks and inline code spans
- Fix duplicate page headings: strip pandoc H1 BEFORE prepending frontmatter
so the ^# regex anchors at the start of pandoc output, not after ---
- Fix inspect.cleandoc vs textwrap.dedent for Python docstring parsing so
Parameters/Returns/Raises sections render correctly
- Fix .. dropdown:: dedent (textwrap.dedent instead of hardcoded 3-space strip)
so list items inside Accordion blocks are flat, not nested
- Remove explicit H1 from extract_python_api.py and extract_c_api.py output
(frontmatter title: is the heading; duplicate H1 caused double rendering)
Nav / docs.yml:
- Add _inject_nav_after, _rename_page, _remove_page helpers so curated nav
fixes survive regeneration: REST API Reference after Server CLI, Settings
pages under Python API, duplicate section/page names resolved, Swagger stub
removed
- Split navigation into docs-v26-08.yml (required by Fern when versions is set)
- Add version switcher with all archived versions (26.06–25.05) pointing to
archive.docs.nvidia.com via JS redirect pages
Install selector widget (fern/docs/scripts/install-selector.js):
- Full rewrite with lazy resolveVersion() and polling loop (150 ms, ~10 s)
to survive Next.js async script injection race; pushState/replaceState hooks
set up unconditionally
Python source (python/cuopt/cuopt/linear_programming/io/parser.py):
- Add NumPy-style docstring to toDict so it appears in the Python API reference
CI (.github/workflows/fern-docs.yml):
- Add cpp/**/*.h and cpp/**/*.hpp to path triggers (C API extractor reads headers)
build.sh:
- Replace old Sphinx/Doxygen docs target with Fern pipeline:
checks for pandoc/node/jq, auto-installs pinned fern-api, runs
convert_docs.py + fern check, prints fern docs dev command for preview
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add fern/requirements-docs.txt (fastapi, pydantic, pyyaml) as the single source of truth for Python docs dependencies; both build.sh and CI now install from it instead of inlining package names - Update dependencies.yaml docs group: replace old Sphinx/Doxygen packages with Fern toolchain deps (jq, nodejs, pandoc, pyyaml, fastapi, pydantic) so conda environments include what ./build.sh docs actually needs - build.sh: add npm presence check alongside node (npm is separate on some distros), with install hint pointing to nodejs.org and conda Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- ./build.sh docs now runs convert_docs.py + fern check then launches fern docs dev (localhost:3000 preview) instead of just printing the command - Add --publish-docs flag to publish to Fern cloud (nvidia-cuopt.docs.buildwithfern.com) via fern generate --docs; requires FERN_TOKEN env var, checked before running - Add --publish-docs to VALIDARGS and help text Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- On every run: upload generated fern/docs.yml, docs-v*.yml, docs/pages/, and openapi/ as a GitHub Actions artifact (14-day retention) so reviewers can download and inspect exactly what will be published - On pull_request: run fern generate --docs --preview to publish a temporary Fern preview URL that doesn't affect the live site; requires FERN_TOKEN secret, skipped silently if absent Note: Fern has no native static HTML export — it is a Next.js SPA served via Fern cloud. The artifact above contains the MDX source; the preview URL gives a fully rendered browser view for sharing during PR review. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- New non-blocking link-check job runs after convert-and-check, concurrently with publish (continue-on-error: true so broken external links don't block the live site update — check the job summary for failures) - Skips fern/docs/pages/versions/ (JS redirect stubs, no real links) - fern/.lycheeignore excludes hosts that reliably cause false positives in CI: localhost, YouTube (429), LinkedIn (bot block), NGC catalog (auth required), Azure Maps sample (401), raw.githubusercontent.com (429 bursts) - --accept 429 treats rate-limit responses as passing (not a dead link) - Uploads lychee-report.md as a 14-day artifact on every run Note: fern check validates config schema only; it does not check URLs. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Covers prerequisites, ./build.sh docs local preview, manual publish with
FERN_TOKEN, and a table of what each CI trigger does — addresses the gap
Andrew pointed out ("samples in the readme").
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…-links - Add `global-theme: nvidia` to docs.yml (mandatory for NVIDIA Fern branding) - Add GitHub navbar link to docs.yml header - Fix accentPrimary light color to match NVIDIA green (#76B900) - CI: split PR preview into two stages so FERN_TOKEN is never exposed to fork PRs; stage 2 runs via workflow_run in base-branch context - CI: add `fern docs broken-links` step for internal link checking - CI: stage 1 saves PR metadata artifact so stage 2 can post preview comment - README: document fern login requirement, how to get FERN_TOKEN, custom domain requirement, and two-stage PR preview security model Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
|
Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually. Contributors can view more details about this message here. |
MDX pages are now the source of truth. Removes: - fern/convert_docs.py, extract_python_api.py, extract_c_api.py - fern/requirements-docs.txt (pandoc, pyyaml, fastapi, pydantic) - All conversion steps from build.sh and CI - RST/C++/Python path triggers from CI (no longer affect docs) - lychee link-check job (not needed for hand-edited MDX) - pandoc/pyyaml/fastapi/pydantic from dependencies.yaml docs group CI now: fern check + fern docs broken-links + PR preview + publish. build.sh docs now: install fern CLI → fern check → fern docs dev. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
OpenAPI spec, Python API reference, and C API reference are derived from code and regenerated by fern/generate_api_docs.py on every docs build. RST→MDX conversion is still gone; hand-edited MDX pages are not touched. - Restore fern/extract_python_api.py and fern/extract_c_api.py - Add fern/generate_api_docs.py (orchestrates the three dynamic outputs) - Restore fern/requirements-docs.txt (fastapi, pydantic, pyyaml) - build.sh docs: run generate_api_docs.py before fern check - CI: install Python deps, run generate_api_docs.py, then fern check - Re-add C++/Python source paths to CI triggers Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- ci/build_docs.sh: replace Sphinx build with Fern pipeline (conda env already provides jq, nodejs, pyyaml, fastapi, pydantic via rapids-dependency-file-generator; Fern CLI installed via npm at pinned version) Generates API docs, runs fern check, then publishes (preview on PR, production on branch/nightly) - build.sh docs: skip pip install when packages already present (conda); fall back to pip only when importing fails - build.yaml / pr.yaml: remove Sphinx html artifact upload; switch to cpu node type (no GPU needed for docs) - Remove .github/workflows/fern-docs.yml (superseded by existing docs-build jobs) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds explicit trigger to AGENTS.md so CI/workflow changes clearly map to skills/cuopt-developer/ — surfacing the "prefer extending existing scripts" rule before new workflow files or pip installs are created. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Python API: keep AST walking (no imports needed) but pipe docstrings through numpydoc.docscrape.NumpyDocString, giving properly structured Parameters / Returns / Raises / Examples sections in the generated MDX. C API: replace fragile regex header parser with Doxygen XML pipeline. Doxyfile targets the two public headers; XML is parsed with ElementTree and rendered to MDX. Picks up 211 symbols vs ~80 before, handles @param, @return, @brief, and enum values correctly. Add numpydoc to requirements-docs.txt and dependencies.yaml docs group. Add doxygen to dependencies.yaml docs group. Gitignore cpp/build_split/ and fern/.doxygen-xml/ build directories. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
ramakrishnap-nv
force-pushed
the
docs-fern-migration
branch
from
2081988 to
6b157a0
Compare
…tom parsers Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Python API: replace hardcoded file paths with PAGE_SOURCES config — each entry lists directories to scan non-recursively. New .py files added to a configured directory appear in the docs automatically without editing the script. C API: replace RST parser with fern/c_api_config.yaml for section organization. Doxygen XML is still the symbol source; the YAML controls section groupings and prose. Symbols not in the config auto-appear in an "Uncategorized" section on the designated catchall page so new header additions are never silently dropped. Added previously uncategorized symbols (hyper diving/heuristic params, scaling constants, presolve constants, deprecated quad create functions). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Copies example source files from docs/cuopt/source/ into fern/docs/pages/*/examples/ so they live alongside the MDX pages that reference them. The old docs/ tree is left intact for now. Adds fern/embed_examples.py which embeds example files as fenced code blocks into the *-examples.mdx pages using idempotent <!-- embed: examples/file --> / <!-- /embed --> markers. First run upgrades bare link stubs; subsequent runs refresh code content within the markers without disturbing surrounding prose. Rewrites the C API example pages (convex-examples.mdx, mip-examples.mdx) to use clean standalone link patterns and removes the repetitive Sphinx-era "It is necessary to have the path..." boilerplate. embed_examples.py is called from generate_api_docs.py on every build. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Remove: - fern/.lycheeignore (lychee CI job was deleted) - cpp/Doxyfile.fern and fern/Doxyfile.fern (remnants of abandoned fern docs md generate --local approach, superseded by fern/Doxyfile) - fern/static/python-docs/ (auto-generated by fern docs md generate; replaced by extract_python_api.py) - libraries: block from fern/docs.yml (referenced the deleted files) Add distance_engine/ to the routing PAGE_SOURCES in extract_python_api.py so WaypointMatrix is covered in the generated routing-api.mdx. Remove the folder: ./static/python-docs nav entry from docs-v26-08.yml. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Migrates cuOpt documentation from Sphinx/RST to the Fern platform. MDX pages in `fern/` are now the source of truth and edited directly.
What's included
Before going to production