Skip to content

MAINT: switch to the src/ layout - #104

Merged
ev-br merged 9 commits into
benchx-dev:mainfrom
ev-br:src-layout
Nov 6, 2026
Merged

ev-br merged 9 commits into
benchx-dev:mainfrom
ev-br:src-layout

Conversation

@ev-br

@ev-br ev-br commented Nov 5, 2026 •

Copy link
Copy Markdown
Collaborator

Previously, we seem to have only used in-place installs, with schemas folder being outside of the installable package root. This does not play well with a proper out-of-place installs. Therefore, switch to a more traditional src/ layout expected for a python project.

Specifically: the project lives at the repository root, with the package under src/. The JSON schemas are package
data, so a wheel carries them and a non-editable install works. The wheel also ships the test suite.

The project lives at the repository root, with the package under src/. The JSON schemas are package
data, so a wheel carries them and a non-editable install works. The wheel also ships the test suite.
The worked examples and demo.sh are not in this repository any more: they live in
benchx_examples and run in the e2e workflow.

benchx/                              repository root (132 tracked files)
├── pyproject.toml                   wheel = src/benchx (+ tests via force-include); readme = docs/benchx.md
├── uv.lock
├── README.md                        repo overview; also the docs site's landing page
├── CHANGELOG.md
├── LICENSE
├── zensical.toml                    docs site config (nav: Home, Design, Schemas, Use cases,
│                                    User stories, benchx; Schemas lists all nine schema pages)
│
├── src/
│   └── benchx/                      the package
│       ├── __init__.py
│       ├── cli.py  session.py  runner.py  compare.py  bench.py  …   (16 modules)
│       ├── adapters/  (7)           gbench, pyperf, asv, base, plus the two scripts
│       │                            run in the benchmark's own interpreter
│       │                            (asv_runtime.py, benchx_params.py)
│       ├── core/  (7)               catalog.py  loader.py  rules.py  validation.py  hashing.py  …
│       └── schemas/  (32 files)     package data, read by core/catalog.py
│           ├── measurement-result/0.1.0/{schema.json, examples/}
│           ├── work-order/  comparison-document/  config/  env-identity/
│           └── environment-policy/  identity-policy/  target-description/
│               target-provider-request/
│
├── tests/  (30 files)               conftest.py, fake_gbench.py, fixtures/
│                                    no __init__.py (tests do `from conftest import …`)
│
├── docs/  (28 files)                the site's sources
│   ├── benchx.md                    the implementation README (was benchx/README.md); also
│   │                                the package's long description on PyPI
│   ├── design/  use-cases/  user_stories/
│   ├── requirements.txt
│   └── index.md, schemas/           generated by tools/gen_docs.py, gitignored
│
├── tools/gen_docs.py                builds docs/index.md and docs/schemas/ from README.md
│                                    and src/benchx/schemas
└── .github/workflows/
    ├── tests.yml                    test matrix (editable) + `wheel` job (non-editable)
    ├── e2e.yml                      runs the benchx_examples; the repo is checked out as
    │                                `benchx/`, so the project is `benchx/` (its .venv: benchx/.venv).
    │                                The numpy-venvs job sets BENCHX_SRC=<workspace>/benchx: its
    │                                run.sh guesses the checkout from the package's parent
    │                                directory, which is src/ now, and finds no pyproject.toml
    └── docs.yml                     builds and deploys the site

What the wheel contains

98 files (93 package files and 5 of .dist-info metadata):

site-packages/benchx/
├── __init__.py, cli.py, …           the 16 modules
├── adapters/  core/                 (14 files)
├── schemas/                         32 files, found by core/catalog.py next to itself
└── tests/                           30 files, copied from the repo's tests/
                                     run with: python -m pytest --pyargs benchx

The sdist has 133 files (the sources, tests/, docs/, the CI workflows and the root files).

Runtime lookups

schemas:  $BENCHX_SCHEMAS  or  <package dir>/schemas       (core/catalog.py, Path(__file__).parents[1])

editable install:   src/benchx/schemas/    tests: the checkout's tests/ (benchx/tests does not exist)
wheel / sdist:      site-packages/benchx/schemas/    tests: site-packages/benchx/tests/

What moved where

Before After
benchx/benchx/ src/benchx/
schemas/ src/benchx/schemas/
benchx/tests/ tests/ (and copied into the wheel as benchx/tests/)
benchx/pyproject.toml, benchx/uv.lock pyproject.toml, uv.lock
benchx/README.md docs/benchx.md
benchx/LICENSE, benchx/.gitignore merged into the root LICENSE and .gitignore
benchx/examples/, benchx/demo.sh gone from this repo (moved to benchx_examples by the e2e PR)
benchx/benchx in e2e.yml benchx (and BENCHX_SRC for numpy-venvs)

Docs

Same pages as before, with the same navbar except for the Schemas entries. The built site differs on
the landing page (the Contents list) and the benchx page (the install line, plus one sentence on the
shipped tests), in the schema pages' "source" links, which point at src/benchx/schemas/…, and in the
navbar's Schemas section:

  • removed: "Observed-context record", which pointed at a page that is never built (the schema was
    removed in Simplify context split and generalize identity using projections #63; Zensical does not validate nav entries, so the strict build never noticed);
  • added: "Identity policy" and "Project configuration", whose pages were built but not linked.

@ev-br

ev-br commented Nov 5, 2026 •

Copy link
Copy Markdown
Collaborator Author

One significant change is that schemas have moved one level down.

TODO here:

  • check that CI works
  • double-check in-place and regular installs
  • check that the docs make sense
  • check that the READMEs make sense --- there were two, and now there's only one.

rok
rok previously approved these changes Nov 6, 2026

@rok rok left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! Mostly moving stuff around. CI seems to be passing ok, I propose we merge and do a separate PR for release automation.

@rok rok mentioned this pull request Nov 6, 2026
2 of 10 tasks
ev-br's Claude Code Agent and others added 8 commits November 6, 2026 14:32
Move the project from benchx/ up to the repository root and the package under src/:

  benchx/benchx      -> src/benchx
  schemas            -> src/benchx/schemas   (package data, so a wheel carries them)
  benchx/tests       -> tests
  benchx/examples    -> examples
  benchx/demo.sh     -> demo.sh
  benchx/pyproject.toml, benchx/uv.lock -> pyproject.toml, uv.lock
  benchx/README.md   -> docs/benchx.md   (the page the docs site already generated from it)

This commit only renames: every file is a 100% rename, no content changes, so
`git blame` and `git log --follow` see through it. The tree does not build or pass
its tests at this commit; the next one updates the paths and the files that collide
(LICENSE, .gitignore) with their root counterparts.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…kage

The previous commit moved the files; this one makes them work from their new places.

Installs: the schemas are package data under src/benchx/schemas, and
core/catalog.py finds them next to itself (parents[1]), so an installed wheel
or sdist carries and finds them, as an editable install does. $BENCHX_SCHEMAS
still overrides. Before, a non-editable install could not even run `bx --help`.
pyproject.toml: packages = ["src/benchx"], readme = "docs/benchx.md".

The wheel also ships the test suite as benchx/tests (hatch force-include), so an installed
benchx can test itself: `python -m pytest --pyargs benchx`. tests/ stays where it is, and
gets no __init__.py, which would break its `from conftest import ...` lines.

Docs: docs/benchx.md is a tracked page now (the docs site used to generate it
from benchx/README.md), so tools/gen_docs.py stops generating it and reads the
schemas from their new place; the README and zensical.toml name the new locations.
The built site is otherwise the same: the same pages and navbar, the schema pages'
"source" links point at the new place.

Housekeeping: benchx/LICENSE was identical to the root one and benchx/.gitignore
is merged into the root .gitignore (plus dist/), which empties benchx/.

CI: tests.yml no longer runs in a subdirectory and its path filters follow the
move; a new `wheel` job builds the wheel, installs it (not editable) into a clean
venv, activates the venv, and from outside the checkout runs `bx --help` and the
test suite the wheel ships (`pytest --pyargs benchx`). docs.yml's filters follow the schemas.
e2e.yml: the project is the root of the benchx checkout now, so `benchx/benchx` becomes `benchx`
in its working directories, cache globs and venv paths.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The nav pointed at schemas/observed-context-record.md, a page that is never built:
that schema was removed in benchx-dev#63, on the day the site was published. Zensical checks
links inside pages but not nav entries, so the strict build did not notice.

Two schemas that do have reference pages were missing from the nav: identity-policy
and config. Add them ("Identity policy" next to the other identity entries, "Project
configuration" last), so the nav lists every schema the generated index page lists.

Page bodies are unchanged; only the navbar differs.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The numpy-venvs job failed with "run.sh: cannot find a benchx checkout to install in
the venvs". The example's run.sh takes the checkout from the benchx of the active
environment, as benchx.__file__'s grandparent, and requires a pyproject.toml there.
That holds only when the package sits directly in the project directory. Under src/
it is the checkout's src/, which has none.

Set BENCHX_SRC, the script's own override, to the checkout (${{ github.workspace }}/benchx)
on the step that runs it, and correct the comment above the install step: the
environment no longer needs benchx in editable mode to find its schemas, which ship
in the package.

Checked by replaying the job in a mock workspace: without BENCHX_SRC the script fails
with that message (it computes <checkout>/src); with it, run.sh builds both venvs
(numpy 1.26.4 and 2.0.2), runs the rounds, compares, and exits 0.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@ev-br
ev-br merged commit bf7d8d3 into benchx-dev:main Nov 6, 2026
12 checks passed
@rok rok mentioned this pull request Nov 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants