Add a global benchx.conf.json config file for project-wide settings

docs/design/prototype-design.md

@@ -25,24 +25,25 @@ bx compare --run KEY --profile revisions|environments --baseline VALUE
                                          # read mode: compare results already measured
 bx target describe|prepare TARGET        # a target's description (a provider's answer, or a build directory's sidecar)
 bx compare BASELINE CONTENDER --profile revisions|environments
-           [--adapter pyperf|google-benchmark]                  # default pyperf
-           --suite NAME [--filter REGEX] [--quantity wall-time]
+           [--adapter pyperf|google-benchmark]                  # default: adapter.name
+           --suite NAME [--filter REGEX] [--quantity NAMES]     # default: adapter.suite, adapter.quantities
            --rounds R [-e NAME=VALUE ...]
            [--repetitions N]                                    # google-benchmark
            [--processes N] [--values N] [--loops N] [--warmups N] # pyperf
            [--min-time S]
-           [--project P] [--label NAME=BASE,CONTENDER]
+           [--project P | --no-project] [--label NAME=BASE,CONTENDER]
            [--run-key KEY] [--out DIR] [--no-build] [--source-uri URI]
            [--k 3] [--min-rounds 3] [--json]
                                          # measure mode: run both sides, then compare
 bx bench TARGET --suite NAME [CASE …]    # the single-target form of bx compare: one order, a table
-         [--adapter …] [--filter REGEX] [--quantity …] [-e NAME=VALUE ...]
+         [--adapter …] [--filter REGEX] [--quantity NAMES] [-e NAME=VALUE ...]
          [precision flags, as for bx compare]
-         [--project P] [--run-key KEY] [--out DIR] [--no-build] [--source-uri URI]
+         [--project P | --no-project] [--run-key KEY] [--out DIR] [--no-build] [--source-uri URI]
          [--case-timeout S] [--no-ingest]
 bx bench MODULE.py [FUNCTION …] [--tree DIR] [...]
                                          # shorthand: the module's checkout, pyperf → result files → store → table
-bx config init [--dir DIR]               # write .benchx/config.json stating pyperf's defaults
+bx config init [--dir DIR] [--adapter …] # write a benchx.conf.json stating a harness's defaults
+bx config show                           # what benchx.conf.json and the defaults give, and where each value comes from
 ```
 
 `bx bench` is the single-target form of `bx compare` (measure mode). It names a
@@ -64,9 +65,8 @@ state and tree id recorded; `--suite` is the module's path in it, and a case is
 a function in it (`bench_*` and `benchmark_*` by default). A case named on the
 command line is an exact name; `--filter` is a regex; not both. `-e` fills
 `environment_variables`. Precision is resolved before the order is written,
-flags over the harness's defaults and, for pyperf, over `.benchx/config.json`
-(found by walking up from the current directory), so the order always states
-what applied; a flag the chosen harness does not take is refused, as in
+flags over `adapter.precision` in `benchx.conf.json` over the harness's defaults, so the
+order always states what applied; a flag the chosen harness does not take is refused, as in
 `bx compare`. The pyperf adapter takes `python` from the `PATH` the workers
 start with and refuses the order if pyperf is not importable there. Both
 commands default to pyperf.
@@ -76,6 +76,26 @@ is the git checkout containing the module (or `--tree`), the suite is the
 module's path in it, and the functions are exact case names. Without `--suite`,
 a TARGET ending in `.py` selects it.
 
+**The project file.** Both commands read `benchx.conf.json` in the project root, found by
+walking up from the working directory; its directory is the root, and with no file the
+root is the working directory (`schemas/config/0.1.0`). It holds `project`, `source.uri`,
+`provider.command`, an `adapter` block (`name`, `suite`, `quantities`, `precision`),
+`environment_variables`, `timeouts`, `paths.store` and `paths.results` (default
+`store.parquet` and `results/` in the root, and relative to the root), and `compare`
+(`profile`, `rounds`, `k`, `min_rounds`). A command-line argument wins; if it is missing the
+value is read from the file; if that is missing too, a built-in default applies, where one
+exists. A required value with none of the three (`adapter.name`, `adapter.suite`, the
+target, and for `bx compare` the profile and the rounds) is a hard failure that names the
+flag and the key. `--adapter` naming a harness other than the file's is refused, since the
+file's suite and precision belong to its own. `--quantity` takes a comma-separated list
+and one run collects all of them. `--no-project` makes the run ad hoc, filed in the store
+as `local/<hostname>`, whatever the file's `project` says. The provider is needed only by an
+`@CONFIG` target. The store and the results directory are left out of the identity of a
+measured tree they sit in, so a run's own output does not mark it dirty. The earlier
+`.benchx/provider.json`, `.benchx/config.json` and the `~/.benchx` home directory are gone and not read (only `$BENCHX_HOME` survives, as a store override);
+`.benchx/setup.json` (declared environment facts, §3) is a separate file.
+`schemas/config/0.1.0/examples` has recommended settings for each harness.
+
 `bx compare` has two modes, chosen by its arguments. Two positional targets
 (a path read by the harness, or `@CONFIG[=SOURCE_REF]`, a call to the project's
 target provider) mean measure mode; `--run` means read mode; giving
@@ -99,9 +119,11 @@ so the workbench, a CI script, or a notebook can do the same without a
 subprocess. Measure mode is a call into the session function, which calls the
 runner for each order and then the comparator.
 
-The local store has one place, `~/.benchx/store.parquet`, and no command
-takes another; `$BENCHX_HOME` relocates benchx's home as a whole, for tests
-and CI. `bx run` delivers what it measures there: it writes the result files,
+The local store has one place per project, `store.parquet` in the project root
+unless `paths.store` in `benchx.conf.json` says otherwise; no command takes another.
+`$BENCHX_HOME`, when set, relocates the store to `$BENCHX_HOME/store.parquet` and wins over
+`paths.store` (for tests, CI and keeping a demo out of your real store).
+`bx run` delivers what it measures there: it writes the result files,
 then ingests exactly those files. The `runner` module itself stays
 store-unaware (`runner.md` §2 principle 6); the command composes it with
 ingest, which is the delivery step of `runner.md` §4 and what the workbench
@@ -312,7 +334,7 @@ prototype records the tree as it is at run time and does not detect staleness
 
 ## 4. Store (`bx ingest` + queries)
 
-The local store is one Parquet file, always `~/.benchx/store.parquet` (§1);
+The local store is one Parquet file, the project's `paths.store` (§1);
 the one other store is the temporary file `bx compare --results` compares
 through. There are no side files:
 no JSON, no separate entity tables. Each row is one result, the
@@ -496,7 +518,7 @@ assigns `round` and `slot`. The demo script plays the user, and in that role
 the target provider: it prepares the sources and build directories with git
 and CMake, since benchx builds nothing. `prototype/examples/archery-local`
 shows the other route, where a provider script named in
-`.benchx/provider.json` builds and describes the targets and the sides are
+`benchx.conf.json` builds and describes the targets and the sides are
 given as `@plain` and `@hardened`.
 
 - **Revisions:** two git worktrees of one source at the base and head
@@ -521,7 +543,7 @@ re-ingested to show idempotency, and compared again with `--results`; that
 document must equal the one from the local store (criterion 8).
 
 Every `bx run` delivers to the local store, so both shapes' results land in
-`~/.benchx/store.parquet`, the ad hoc ones as thin results of project
+the demo project's store (`prototype/examples/demo/store.parquet`), the ad hoc ones as thin results of project
 `local/<hostname>`. Run keys carry a timestamp, so successive demo runs add
 rows rather than colliding on `ingest_key`. The store and each series'
 history grow from run to run, and the demo ends with `bx head`.
@@ -534,7 +556,7 @@ history grow from run to run, and the demo ends with `bx head`.
   `environment_variables` and defers the rest of the environment policy, and it
   refuses an order that names one rather than ignoring it (§3).
 - The target-provider hook is minimal: one provider command per project
-  (`.benchx/provider.json`), `describe` and `prepare` only, `build` targets
+  (`provider.command` in `benchx.conf.json`), `describe` and `prepare` only, `build` targets
   only, and a source ref other than `WORKSPACE` is the provider's business.
   How cached builds are named and expired stays open
   (`system-decomposition.md` §6 question 5). The demo prepares its builds by

docs/design/prototype-scope.md

@@ -44,7 +44,7 @@ one run:
 | Adapter | Google Benchmark, both halves: drive filter and repetitions, translate output; pyperf, both halves, for Python functions in a working tree, through `bx bench` and `bx compare` (the default adapter of both) | all other harnesses; pyperf's `tags`, `date`, per-run load and frequency, and `pyperf check` | — |
 | Store | one Parquet file; ingest per schema §5.4 (validated and durable, auto-create, unit-conflict rejection, idempotency, attempt integrity); one default identity policy; queries: list series, series history, get result | continuity events, quarantine, artifact storage, identity-policy authoring | — |
 | Comparator | run mode with the `revisions` and `environments` profiles; eligibility guard; comparison document | history mode; `single-run`, `variants`, and `cross-machine` profiles; batch requests; provisional noise rules | — |
-| Workbench session loop | `bx compare` in measure mode: writes one work order per side and round, alternates them, delivers the results, then compares; takes each side as a build directory or as `@CONFIG`, a call to the project's target provider (`describe` or `prepare`), whose description may be a build or a working tree; either harness, with the same `-e` variables on both sides; a bare path is a build directory for Google Benchmark and a working tree for pyperf; `bx bench` is the same path for one order; `bx target` asks for a description directly; copies the description's declared text (`how_built`, `activation`, `shell`, `provider`) into the order | provider-built revisions (a source ref other than `WORKSPACE` is the provider's business), an `artifact` target, a declared `build` | workbench packaging, front ends such as `spin bench` |
+| Workbench session loop | the project file `benchx.conf.json` (one file, in the project root: provider, project, harness and suite, quantities, precision, environment variables, timeouts, paths, comparator policy; flags over file over defaults); `bx bench` (one order) and `bx compare` in measure mode: writes one work order per side and round, alternates them, delivers the results, then compares; takes each side as a build directory or as `@CONFIG`, a call to the project's target provider (`describe` or `prepare`), whose description may be a build or a working tree; either harness, with the same `-e` variables on both sides; a bare path is a build directory for Google Benchmark and a working tree for pyperf; `bx bench` is the same path for one order; `bx target` asks for a description directly; copies the description's declared text (`how_built`, `activation`, `shell`, `provider`) into the order | provider-built revisions (a source ref other than `WORKSPACE` is the provider's business), an `artifact` target, a declared `build` | workbench packaging, front ends such as `spin bench` |
 | Documents | work order (`schemas/work-order/0.1.0`), result (schema §5.2), comparison document | — | — |
 
 **Out entirely:** scheduler, change detector, alerting, views beyond query
@@ -84,7 +84,7 @@ proto-workbench).
 - One command, `bx compare`, has a read mode and a measure mode
   (`prototype-design.md` §1).
 - The local store is a single Parquet file in one place,
-  `~/.benchx/store.parquet`, and `bx run` delivers to it. JSON is for
+  the project's `store.parquet` (`paths.store` in `benchx.conf.json`), and `bx run` delivers to it. JSON is for
   documents in transit (work orders, result files, comparison documents),
   never for the store.
 - Ingest acknowledgment = validated and durable (schema §5.4).

docs/design/system-decomposition.md

@@ -179,8 +179,8 @@ The same document reaches the workbench by three routes:
 1. **A sidecar file**, `.benchx-target.json` in a build directory, written by
    the user's own build script. It stays with a long-lived build directory
    instead of with one invocation.
-2. **A provider command**, named in the project file `.benchx/provider.json`
-   as `{"command": [...]}`, which prints the description on stdout.
+2. **A provider command**, named in the project file `benchx.conf.json`
+   as `"provider": {"command": [...]}`, which prints the description on stdout.
 3. **The workbench's own input**, when the user prepared everything by hand
    and there is no provider.
 

prototype/README.md

@@ -31,16 +31,30 @@ bx bench TARGET --suite NAME [CASE …]    # the single-target form of bx compar
 bx bench MODULE.py [FUNCTION …] [--tree DIR] [...]
                                          # shorthand: Python functions under pyperf → results → store → table
 bx bench PATH --adapter asv [CASE …] [--filter REGEX] [--tree DIR] [...]
-                                         # shorthand: an asv benchmark suite, timed by pyperf
-bx config init [--dir DIR]               # write .benchx/config.json stating pyperf's defaults
+                                          # shorthand: an asv benchmark suite, timed by pyperf
+bx config init [--dir DIR] [--adapter …] # write a benchx.conf.json stating a harness's defaults
+bx config show                           # what benchx.conf.json and the defaults give, and where each value comes from
 ```
 
+**One project file.** `benchx.conf.json` in the project root (found by walking up from the
+working directory; its directory is the root) says how the project is measured: the
+`provider`, the `project` name and `source`, an `adapter` block (`name`, `suite`,
+`quantities`, `precision`), `environment_variables`, `timeouts`, `paths` for the store
+and the results (by default `store.parquet` and `results/` in the root), and the
+comparator's `compare` policy. Every value is the command line's, else the file's, else a
+built-in default; a required one (`adapter.name`, `adapter.suite`, the target, and for
+`bx compare` the profile and `--rounds`) that has none is a hard failure that names the
+flag and the key. `--no-project` makes one run ad hoc whatever the file's `project` says.
+The schema is `schemas/config/0.1.0`, whose `examples/` hold recommended settings for
+pyperf and for Google Benchmark. The store and the results directory are left out of a
+measured tree's identity when they sit inside it. `.benchx/setup.json` (declared
+environment facts) is a separate file, written by whoever prepares an environment.
+
 Each target is `BUILD_DIR[:SOURCE_DIR]` or `@CONFIG[=SOURCE_REF]`. Two targets
 select measure mode and `--run` selects read mode; mixing them is an error.
 
-`@CONFIG` asks the project's **target provider**, the command named in
-`.benchx/provider.json` (`{"command": [...]}`, found by walking up from the
-current directory). The provider speaks the contract of
+`@CONFIG` asks the project's **target provider**, the command named by `provider.command`
+in `benchx.conf.json` (it runs with the project root as its working directory). The provider speaks the contract of
 `system-decomposition.md` §3.4: a JSON request on stdin, a target description
 (`schemas/target-description/0.1.0`) on stdout, build output on stderr, and a
 nonzero exit for failure. `prepare`, the default, may build; `describe`
@@ -62,10 +76,10 @@ working tree for [pyperf](https://pyperf.readthedocs.io), the default
 as dirty). Cases are the harness's defaults (pyperf: each `bench_*` or `benchmark_*`
 function), the exact names given, or a `--filter` regex. A flag the chosen harness does
 not take is refused. `-e` sets environment variables for the workers and lands in the
-workload's parameters. For pyperf, precision resolves as flags, then
-`.benchx/config.json` (found by walking up from the current directory;
-`bx config init` writes a template), then pyperf's defaults, and the resolved values are
-written into the order. The workers run the `python` first on `PATH`, which must have
+workload's parameters. Precision resolves as flags, then `adapter.precision` in
+`benchx.conf.json` (`bx config init` writes a template), then the harness's defaults, and the
+resolved values are written into the order. `--quantity` takes a comma-separated list,
+and one run collects them all. The workers run the `python` first on `PATH`, which must have
 pyperf.
 
 `bx bench MODULE.py [FUNCTION …]` is the shorthand for the quickest case, timing a few
@@ -126,8 +140,8 @@ every result is pyperf. Results also record the installed project's version
   `.benchx/setup.json`. It changes nothing on the machine and builds nothing,
   and it refuses an order that names a deferred field (`environment_policy`,
   a declared `build`, `components`, `schedule`) instead of ignoring it.
-- **The store** is one Parquet file, always `~/.benchx/store.parquet`
-  (`BENCHX_HOME=...` relocates benchx's home, as the tests do). It reads documents strictly, validates them, and applies the
+- **The store** is one Parquet file, by default `store.parquet` in the project root
+  (`paths.store` in `benchx.conf.json` moves it). It reads documents strictly, validates them, and applies the
   schema §5.4 contract: idempotency and the rejection codes `malformed`,
   `idempotency-conflict`, `unit-conflict`, `identity-violation`, and
   `attempt-conflict`. Each result is one row: the columns #30 derives from the
@@ -180,14 +194,14 @@ part, then:
    persistent one, lists series and history, and ends with `bx head`.
 
 Every `bx run` and every measure-mode `bx compare` delivers to the local store, so each demo run adds its
-results, ad hoc and tracked, to `~/.benchx/store.parquet`. Run keys carry a
+results, ad hoc and tracked, to `examples/demo/store.parquet` (the demo's project root, set
+in its `benchx.conf.json`). Run keys carry a
 timestamp, so runs accumulate and each series' history grows. Look at it any
 time:
 
 ```console
-$ .venv/bin/python -m benchx.cli head -n 20
-$ rm ~/.benchx/store.parquet    # start over
-$ BENCHX_HOME=/tmp/benchx ./demo.sh    # or keep the demo out of your real store
+$ (cd examples/demo && ../../.venv/bin/python -m benchx.cli head -n 20)
+$ rm examples/demo/store.parquet    # start over
 ```
 
 Both changes add two to three percent of work. The quiet benchmark flags them
@@ -211,7 +225,7 @@ benchx/
   adapters/asv.py    asv's benchmark format, timed by pyperf (bx bench --adapter asv)
   adapters/asv_runtime.py  asv's discovery and setup semantics, standalone, copied beside each driver
   bench.py           bx bench: the MODULE.py and asv shorthands, and the results table
-  config.py          .benchx/config.json: the pyperf precision defaults and their template
+  config.py          benchx.conf.json: finding it, validating it, picking a value, the template
   runner.py          one work order in, results out
   session.py         the session loop: alternate two targets over rounds, then compare
   target.py          target descriptions: the provider call, the sidecar, the declared text
@@ -223,6 +237,7 @@ benchx/
   cli.py             bx (compare has a read mode and a measure mode)
 ../schemas/          the result and work-order schemas, read in place ($BENCHX_SCHEMAS overrides)
 examples/demo-suite  a two-benchmark Google Benchmark suite for the demo
+examples/demo        the demo's project root: its benchx.conf.json, and its persistent store
 examples/pyperf-bench  Python functions under `bx bench` (own README)
 examples/archery-local  an end-to-end, Arrow-local-shaped example with a target provider (own README)
 tests/               test_prototype.py: the success criteria and the runner and store