sphinxdocs: relative markdown links between pages fail to resolve (myst.xref_missing)

Important

Corrected below — see this comment.
Two cells of the matrix in this description are wrong: they were action-cache hits that never re-ran Sphinx.
Stock sphinxdocs fails in all execution modes, not only sandboxed, and copying sources instead of
symlinking them does fix the default and local cases. The description below is kept as filed.

What happened

With sphinx_docs + myst_parser, a relative markdown link between two pages
(e.g. [Add an entity](add-entity.md)) resolves correctly when the Sphinx action
runs unsandboxed, but fails with myst.xref_missing when the action runs under
linux-sandbox. With -W (which we use as a docs-freshness gate) this is a hard
build failure.

_sources/how-to/register-dataset.md:8: WARNING: Unknown source document
  '/home/.../execroot/_main/bazel-out/k8-fastbuild/bin/.../_ontology_docs/_sources/how-to/add-entity'
  [myst.xref_missing]

Note the two paths involved. The document being processed is reported at the
sandbox execroot:

/home/.../sandbox/linux-sandbox/58/execroot/_main/bazel-out/.../_sources/how-to/register-dataset.md

but the resolved link target is at the real execroot:

/home/.../execroot/_main/bazel-out/.../_sources/how-to/add-entity

Inside the sandbox, bazel-out is a symlink into the real output tree. MyST
appears to resolve the relative target to a canonical (symlink-resolved) absolute
path, which drops the sandbox prefix, while Sphinx's docnames remain relative to
the srcdir it was given (the sandbox path). The two can then never match.

Reproduction matrix

Same target, same conf.py, same sources, only the execution strategy varies:

strategy	result
worker (default)	pass
`--strategy=SphinxBuildDocs=local`	pass
`--strategy=SphinxBuildDocs=sandboxed`	fail (`myst.xref_missing`)

Because worker mode is the default, this is easy to miss — and it means a docs
build is only correct as long as it is not sandboxed.

What does and doesn't break

:doc: roles and toctree entries resolve fine in all three modes.
Only relative markdown links between pages fail.

Not caused by the source-tree symlinks (checked)

_sphinx_source_tree_impl symlinks each source into the _sources dir, so the
obvious hypothesis is that the per-file symlinks are what MyST resolves through. I
patched _relocate to copy regular files instead of symlinking them and re-ran the
matrix: the failure is unchanged (the resolved target path moves from the
original source tree to the real execroot _sources, but still doesn't match the
sandbox path). So copying sources isn't a fix; the symlinked component is
bazel-out itself within the sandbox.

Environment

sphinxdocs 2.2.0 (bzlmod), Bazel 8.x, Linux
sphinx-autoapi as the API-doc generator, myst_parser for markdown, furo theme
Sources staged via srcs + renamed_srcs (a TreeArtifact mounted as pkg/)

Why it matters to us

We gate docs freshness on sphinx-build -W, and we're moving builds onto remote
execution, where actions are isolated by default. A docs rule that is only correct
unsandboxed can't be relied on there.

Possible directions (no strong opinion on which is right)

Materialize the source tree into a real (non-symlinked) directory at action
time before invoking Sphinx — this is what our current hand-rolled genrule does
(stages into a tempfile.TemporaryDirectory()), and it's why it works under any
strategy. It presumably conflicts with keeping a stable srcdir across
persistent-worker requests.
Pass Sphinx a srcdir that is already canonical, so MyST's resolution and
Sphinx's docnames agree.
If this is properly a myst_parser bug rather than a sphinxdocs one, we're happy
to take it upstream there instead — pointers welcome.

Happy to supply a minimal standalone repro repo if useful.

Oh interesting! Yes, a minimal repro would be useful.

I wonder if this would explain why it's been hard to fix all the xref errors in our own docs. It seemed like they would appear and disappear randomly, or xrefs were oddly picky in ways I couldn't figure out. I always just chalked it up to the inscrutable xref resolution logic sphinx has.

Repro below. First, though, an important correction to my original report — I had two cells of that matrix wrong, and the corrected version makes this a bigger problem than I described, not a smaller one.

Correction: it fails in every execution mode, not just sandboxed

My "worker and local pass" rows were action-cache hits that never re-executed Sphinx. When I forced re-execution (perturbing the action key with --@sphinxdocs//sphinxdocs:extra_env= and confirming from the process line that the spawn actually ran), stock sphinxdocs fails in all three modes:

sphinxdocs	worker (default)	`local`	`sandboxed`
stock 2.2.0	fail	fail	fail
sources copied instead of symlinked	pass	pass	fail
copied + `realpath(srcdir)`	pass	—	fail

So sandboxing is not the trigger; it's an additional layer. And I was wrong that copying doesn't help — copying fixes the default and local cases. Sorry for the noise on both counts.

Relevant to your flaky-xref suspicion: because stock fails regardless of strategy, any sphinxdocs user with relative markdown links between pages is affected. What varies is which absolute path shows up in the message, and whether the action re-ran at all — which would look exactly like xrefs "appearing and disappearing randomly."

Diagnosis

MyST resolves a relative link by canonicalizing the current document's path (following symlinks), then resolves the target relative to that. Sphinx then can't map the result to a docname because it no longer shares srcdir's prefix.

Stock, any mode: _sources/x.md is a symlink into the original source tree, so resolution escapes there. The reported target is a source-tree path.
With sources copied (patching _relocate to cp regular files instead of ctx.actions.symlink): _sources/x.md is a real file, resolution stays inside _sources, and it works — unsandboxed.
Sandboxed, even with copies: Bazel symlinks the action's inputs into the sandbox, so _sources/x.md is once again a symlink (now pointing at the real bazel-out copy). Resolution escapes to the real execroot while srcdir stays sandbox-prefixed.

I also tried canonicalizing srcdir in sphinx_build.py (args[1] = os.path.realpath(args[1]), verified applied). It does not fix the sandboxed case, because the escape happens per-document rather than via the srcdir argument.

That suggests the only strategy-independent fix is for the srcdir contents to be real files at action time — i.e. materialize/copy the source tree inside the action before invoking Sphinx, rather than assembling it from symlinks beforehand. I realize that likely conflicts with keeping a stable srcdir across persistent-worker requests, so I don't have a good proposal that preserves the worker design. Happy to test any patch you want to try.

Minimal repro

Two markdown pages linking to each other, myst_parser, -W. No autoapi, no wheels.

MODULE.bazel:

module(name = "sphinxdocs_xref_repro")

bazel_dep(name = "rules_python", version = "2.0.2")
bazel_dep(name = "sphinxdocs", version = "2.2.0")

python = use_extension("@rules_python//python/extensions:python.bzl", "python")
python.toolchain(
    is_default = True,
    python_version = "3.12",
)

pip = use_extension("@rules_python//python/extensions:pip.bzl", "pip")
pip.parse(
    hub_name = "pip",
    python_version = "3.12",
    requirements_lock = "//:requirements.txt",
)
use_repo(pip, "pip")

BUILD.bazel:

load("@sphinxdocs//sphinxdocs:sphinx.bzl", "sphinx_build_binary", "sphinx_docs")

sphinx_build_binary(
    name = "sphinx-build",
    deps = [
        "@pip//myst_parser",
        "@pip//sphinx",
    ],
)

sphinx_docs(
    name = "site",
    srcs = ["docs/other.md"],
    config = "docs/conf.py",
    extra_opts = ["-W"],  # how we gate docs freshness; drop it to see a warning instead
    formats = ["html"],
    renamed_srcs = {"docs/index.md": "index.md"},
    sphinx = ":sphinx-build",
    strip_prefix = "docs/",
)

docs/conf.py:

project = "repro"
extensions = ["myst_parser"]

docs/index.md:

# Index

A relative link to a sibling page: [other page](other.md)

```{toctree}
other
```

docs/other.md:

# Other

Back to the [index](index.md)

requirements.txt is a compiled lock of just:

sphinx==9.0.4
myst-parser==5.1.0

(uv pip compile requirements.in -o requirements.txt, or pip-compile — 62 lines, nothing special in it.)

Then:

bazel build //:site                                       # fails
bazel build //:site --strategy=SphinxBuildDocs=local      # fails
bazel build //:site --strategy=SphinxBuildDocs=sandboxed  # fails

Output (default, worker):

_sources/index.md:3: WARNING: Unknown source document
  '/path/to/repro/docs/other' [myst.xref_missing]
_sources/other.md:3: WARNING: Unknown source document
  '/path/to/repro/docs/index' [myst.xref_missing]

Note the resolved targets are docs/other and docs/index in the original source tree, not under _sources.

One caveat when reproducing: add something to the action key on each run (e.g.
--@sphinxdocs//sphinxdocs:extra_env=RUN=1, incrementing) and check the N processes: line actually shows a worker / linux-sandbox / local spawn. Otherwise you get an action-cache hit and it looks like a mode passed when Sphinx never ran — which is exactly how I got the original matrix wrong.

Also worth knowing: in worker mode, Sphinx's stdout/stderr is captured and only surfaced on failure, so with warnings-not-errors these are silent.

sphinxdocs: add reproduction test case for relative markdown xref res…

That suggests the only strategy-independent fix is for the srcdir contents to be real files at action time — i.e. materialize/copy the source tree inside the action before invoking Sphinx, rather than assembling it from symlinks beforehand. I realize that likely conflicts with keeping a stable srcdir across persistent-worker requests, so I don't have a good proposal that preserves the worker design.

Yeah, having to copy everything is unfortunate since its more work. I don't think its incompatible with worker mode, though. I think it just means the worker has to do more work to keep things in sync. It receives the list of input files and hashes, then has to copy those inputs with changed/new hashes to a private srcdir. That doesn't sound too bad.

sphinxdocs: add automated test asserting relative markdown xref resol…

fix(sphinxdocs): include stderr in Sphinx worker response output (baz…

test(sphinxdocs): add test coverage for directory artifact relative x…

sphinxdocs: add reproduction test case for relative markdown xref res…

sphinxdocs: add automated test asserting relative markdown xref resol…

test(sphinxdocs): add test coverage for directory artifact relative x…

I put out #3977 to fix this. It copies everything, but that was the only solution I could find. Which is too bad -- it'll make the disk usage much worse, plus it takes time to copy everything (it does it as concurrently as possible). Ah well.

Also worth knowing: in worker mode, Sphinx's stdout/stderr is captured and only surfaced on failure, so with warnings-not-errors these are silent.

Fixed by #3994

conf.py in another directory results in error

Fixed by #4006

fix(sphinxdocs): fix path resolution for conf.py in separate directory

fix(sphinxdocs): fix path resolution for conf.py in separate director…

fix(sphinxdocs): materialize source symlinks to resolve relative cros…

fix(sphinxdocs): fix path resolution for conf.py in separate directory (

Navigation Menu

Repository navigation

sphinxdocs: relative markdown links between pages fail to resolve (myst.xref_missing) #3977

What happened

Reproduction matrix

What does and doesn't break

Not caused by the source-tree symlinks (checked)

Environment

Why it matters to us

Possible directions (no strong opinion on which is right)

Correction: it fails in every execution mode, not just sandboxed

Diagnosis

Minimal repro

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

sphinxdocs: relative markdown links between pages fail to resolve (myst.xref_missing) #3977

Description

What happened

Reproduction matrix

What does and doesn't break

Not caused by the source-tree symlinks (checked)

Environment

Why it matters to us

Possible directions (no strong opinion on which is right)

Activity

rickeylev commented on Jul 30, 2026

ctcjab commented on Jul 31, 2026

Correction: it fails in every execution mode, not just sandboxed

Diagnosis

Minimal repro

rickeylev commented on Aug 3, 2026

rickeylev commented on Aug 5, 2026

rickeylev commented on Aug 5, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions