Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
109 changes: 109 additions & 0 deletions docs/release-notes/0.1.1.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,109 @@
# ComputeArena CLI 0.1.1

This release makes model identity verifiable. Reports now state exactly which
published model file was benchmarked, the client checks that claim against
Hugging Face before uploading, and computearena.ai verifies it again on its
own. It also adds update notices, server-driven upgrade guidance, and clearer
BaseRT support on Linux x86-64.

## Install or upgrade

On macOS or Linux:

```sh
curl -LsSf https://computearena.ai/install.sh | sh
```

The same command upgrades an existing installation. Saved reports, the
installation signing key, login sessions, and installed runtimes are kept, and
the data directory gains a `model-provenance/` folder for the receipts
described below. Release archives and checksums are also available below for
manual installation.

## Highlights

- **Verifiable model identity.** Every new report carries a
`computearena-model/1` identity: the upstream Hugging Face model, the exact
artifact (repository, immutable revision, file path, format, and
runtime-specific quantization), the SHA-256 of the bytes that were
benchmarked, and how they were obtained. The fields from 0.1.0 remain
populated, and reports saved by earlier releases still submit.
- **Managed downloads leave a receipt.** GGUF downloads are pinned to an
immutable revision, hashed while streaming, and checked against the Hub's
published SHA-256. BaseRT models pulled through the picker are matched to
their exact published file by hash. Models installed before this release are
matched the same way the first time they are benchmarked; when no published
file matches, the report records the repository and the hash rather than a
guessed file name.
- **BaseRT catalogue in the model picker.** The BaseRT picker lists BaseRT's
public catalogue and runs `basert pull` with a backend-aware target, so
BaseRT keeps choosing the compatible artifact, downloading split files, and
converting. Existing `.base` and `.gguf` files can still be entered directly.
- **`computearena identify`.** Bind a file you downloaded or copied yourself to
one exact Hub file: `computearena identify ./model.gguf <hf-file-url>`
succeeds only when the local SHA-256 matches the published object. A
repository URL or a model name is not enough, and a later `identify` can
correct an earlier receipt.
- **Checks before submission.** Before uploading, the client checks each
report's artifact against Hugging Face and lists it as verified, unresolved,
or check unavailable. A file whose hash contradicts the claimed published
file is set aside as invalid; in the full-screen interface the remaining
reports are still uploaded, and the summary says how many were left out.
When computearena.ai has verified an artifact, its outcome is shown after
the upload.
- **Model families.** A repository's `base_model` link counts only when the
repository declares itself a quantization of that model. Finetunes,
adapters, and merges stay their own model, so an instruct model is not folded
into its pretraining base. computearena.ai decides the final grouping, and a
difference between the client's record and that grouping never blocks a
submission.
- **Update notices and upgrade guidance.** Interactive sessions check for the
latest release in the background, keep the answer for a day, and mention a
newer version when one exists; the check never blocks offline use. API
requests carry the client version, and when the service retires a release
the client prints the installed version, the minimum required, and the
install command, then stops the submission batch.
- **BaseRT on Linux x86-64.** ComputeArena warns before entering BaseRT on
Linux x86-64, where no prebuilt BaseRT runtime is published, and points to
llama.cpp or a compatible harness you built yourself instead of an installer
that cannot help there.

## Supported systems

Prebuilt ComputeArena binaries are available for:

- macOS on Apple Silicon;
- Linux x86_64 with glibc 2.31 or newer; and
- Linux arm64 with glibc 2.31 or newer.

Windows and Intel Macs are not supported in this release. ComputeArena does not
bundle an inference runtime or model; install BaseRT or llama.cpp and obtain a
compatible model before benchmarking. Prebuilt BaseRT runtimes exist for macOS
on Apple Silicon and for Linux arm64 with CUDA; on Linux x86-64, use llama.cpp
or a BaseRT benchmark harness you built yourself.

The macOS binary is not signed with an Apple Developer ID. The release archive
is checksum-verified and carries a Sigstore keyless signature from the release
workflow. Browser-downloaded copies may need their quarantine attribute removed
as described in the README.

## Model identity and trust

A verified artifact means the bytes that were benchmarked are a file published
on Hugging Face, and the model family follows what that file's publisher
declares. It does not prove that the publisher labelled the model correctly,
that the runtime executed those bytes honestly, or that two fine-tunes with
different lineage are equivalent. The client's checks are advisory;
computearena.ai repeats them independently, because a public client is not a
trust boundary. Files with no evidence, gated repositories the service cannot
read, and ambiguous merges stay unresolved rather than being grouped by file
name. Set `HF_TOKEN` in your environment to download from gated repositories.

Accuracy benchmarks, MLX, vLLM, Windows, and Intel macOS support are not part
of v0.1.1.

For usage and verification details, see the
[README](https://github.com/basecompute/computearena-cli#readme) and
[docs/model-identity.md](https://github.com/basecompute/computearena-cli/blob/main/docs/model-identity.md).
Questions and feedback are welcome in the
[ComputeArena Discord](https://discord.gg/CCT24GWhPG).
Loading