Skip to content

feat(cli): say when the installed BaseRT is worth updating - #33

Merged
IsuraManchanayake merged 1 commit into
rc-0.1.2from
feat/basert-update-notice
Sep 17, 2026
Merged

IsuraManchanayake merged 1 commit into
rc-0.1.2from
feat/basert-update-notice

Conversation

@IsuraManchanayake

Copy link
Copy Markdown
Collaborator

Summary

The client checked for newer releases of itself only. A BaseRT it found was used as it was: ✓ Found BaseRT 0.2.4, and on with the run. Since BaseRT 0.2.5 shipped the headline-first protocol, someone still on 0.2.4 keeps producing reports signed as computearena-throughput-legacy/1 (comparable: false) and nothing tells them so.

ComputeArena now says two things about the BaseRT it found, and still never refuses to run an older one:

  • The harness predates the current benchmark protocol. No network needed: the harness describes what it supports. The notice names what the report will be signed as and the release to move to.
  • A newer BaseRT release exists. The lookup reuses the client's own update check: GitHub's latest release, on a worker thread, remembered for 24 hours (basert-update-check.json), failures ignored. It never delays a run. If the answer arrives while the benchmark is running, the notice follows the run instead, and nothing is said twice.

Where it shows:

  • run: before the benchmark plan.
    ! BaseRT 0.2.4 predates the current benchmark protocol.
      Its runs share one context across the sweep and are signed as computearena-throughput-legacy/1, marked not comparable. BaseRT 0.2.5 measures PP512 and TG128 first, in a freshly loaded model with a 4K reservation, and reports telemetry from the timed repetitions.
      Update with `computearena basert install`, or the official installer: curl -LsSf https://basecompute.co/install.sh | sh
    
    With a current harness and a newer release it is just ! BaseRT 0.2.6 is available (installed: 0.2.5). and the update line.
  • Printed session: where it reports the runtime it found.
  • Full-screen interface: a BaseRT panel above the menu (summary and key only on small terminals, so the menu keeps its rows), a Protocol row in the benchmark plan, and u on the menu to install the latest release. u takes a second press, because it downloads a release and replaces the files in ~/.basert; any other key stands it down.

The advice fits the situation: a harness chosen with --runtime-path or COMPUTEARENA_BASERT_HARNESS is not offered an install that would not replace it, a platform without a prebuilt BaseRT (Linux x86-64) is pointed at the release to build from, and versions that cannot be compared (development builds) never invent an update. llama.cpp is unaffected; its builds are tagged several times a day, so "newer" would always be true.

updates.rs becomes a small ReleaseFeed shared by both checks, so there is still one implementation of the lookup and its cache; the client's own notice behaves as before. COMPUTEARENA_BASERT_RELEASE_API names another endpoint for mirrors and tests. Only a version number is read from the answer, so the feed cannot put words on the screen.

If this ships in 0.1.2, the sentence in the release notes (#32) that says the client "does not yet tell you when a newer BaseRT exists" needs to change; I will update #32 to match.

Validation

  • cargo fmt --all -- --check, cargo clippy --workspace --all-targets --all-features -- -D warnings: clean
  • cargo test --workspace --all-targets: 104 unit and 38 integration tests pass
    • advice matrix: current harness, newer release, older protocol offline and with the release known, isolated-only harness, harness chosen by flag or variable, no prebuilt bundle, unversioned and development builds, a harness newer than the client understands
    • release feed: fresh answers are reused without a lookup, stale ones are offered while being refreshed, the two feeds do not share a cache
    • full-screen interface: notice and plan row, u needs two presses and any other key stands it down, offered only on the menu and only where an install helps
    • integration: the notice comes before the plan and is said once; the report is still signed (legacy/1) and verifies; a current harness hears only about a newer release; the lookup runs beside the benchmark against a loopback feed and is remembered for the next run; an unreachable feed neither delays nor fails a run; the printed session names an older BaseRT where it finds it; llama.cpp output never mentions BaseRT
  • No test contacts GitHub: every fixture points the lookup at a closed port, seeds the remembered answer, or serves one from a loopback feed
  • Drove the real full-screen interface in a pty with a fake BaseRT 0.2.4 at 110x34, 80x24 and 72x16: full notice, short notice with the menu scrolling by one item, short notice; first u shows the confirmation, another key clears it
  • Ran the real lookup once from a scratch data directory: GitHub's answer (v0.2.5) was parsed, shown in the notice, and written to basert-update-check.json

The client checked for newer releases of itself only. A BaseRT it found
was used as it was, so someone on BaseRT 0.2.4 kept producing reports
signed as computearena-throughput-legacy/1 (not comparable) after 0.2.5
shipped the headline-first protocol, with nothing telling them so.

ComputeArena now says two things about the BaseRT it found, and still
never refuses to run an older one:

- The harness predates the current benchmark protocol. This needs no
  network, because the harness describes what it supports: the notice
  names what the report will be signed as and the release to move to.
- A newer BaseRT release exists. The lookup reuses the client's own
  update check: GitHub's latest release, on a worker thread, remembered
  for 24 hours in basert-update-check.json, failures ignored. It never
  delays a run; if the answer arrives while the benchmark is running,
  the notice follows the run instead, and nothing is said twice.

`run` prints the notice before the benchmark plan, the printed session
prints it where it reports the runtime it found, and the full-screen
interface shows it above the menu (in a short form on small terminals),
adds a Protocol row to the plan, and offers `u` to install the latest
release, which takes a second press because it replaces an
installation. The advice fits the situation: a harness chosen with
--runtime-path or COMPUTEARENA_BASERT_HARNESS is not offered an install
that would not replace it, and a platform without a prebuilt BaseRT is
pointed at the release to build from.

updates.rs becomes a small release feed shared by both checks, so there
is still one implementation of the lookup and its cache.
COMPUTEARENA_BASERT_RELEASE_API names another endpoint for mirrors and
tests; only a version number is read from the answer. No test contacts
GitHub: the fixtures point the lookup at a closed port, seed the
remembered answer, or serve one from a loopback feed.
IsuraManchanayake added a commit that referenced this pull request Sep 17, 2026
The notes said the client does not tell you when a newer BaseRT exists.
With #33 it does: before the plan, in the printed session, and above the
full-screen menu with an update key.
@IsuraManchanayake
IsuraManchanayake merged commit 051905d into rc-0.1.2 Sep 17, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant