Skip to content

sync: engine 0.1.7 — hub cache fix, MLX import fix, default-q6 profile #24

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
prabod merged 1 commit into from
Jul 21, 2026
Merged

sync: engine 0.1.7 — hub cache fix, MLX import fix, default-q6 profile #24

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 8 additions & 8 deletions base-convert/Cargo.lock
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion base-convert/Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ members = [
]

[workspace.package]
version = "0.1.6"
version = "0.1.7"
edition = "2021"
license = "Apache-2.0"
repository = "https://github.com/basecompute/baseRT"
Expand Down
12 changes: 10 additions & 2 deletions base-convert/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,14 +47,22 @@ basert list # installed models (table; --json for JSON)
basert list --remote # also show catalog models not yet installed
```

Downloads stage under `<models dir>/.src/hf` — not the global HuggingFace
cache — and are moved into place on success, so a pulled model exists
exactly once on disk. Interrupted pulls leave their partial download in
staging and resume on retry. Convert-on-pull deletes the downloaded
safetensors sources after a successful conversion; set
`BASERT_KEEP_HF_SOURCES=1` to keep them (faster re-pulls of another quant
of the same repo, at the cost of a second multi-GB copy).

Gated/private repos use the standard HuggingFace token chain
(`$HF_TOKEN` / `~/.cache/huggingface/token`). Convert-on-pull in the
public build uses only generic profiles; tuned quantization is delivered
through pre-converted catalog artifacts.

## Runtime tools
## Runtime commands

`basert <cmd>` forwards to the matching runtime binary (`basert-<cmd>`):
Run a model with the BaseRT runtime:

```
basert serve <model> [--model <model2> …] # OpenAI-compatible server (--model repeats for multi-model)
Expand Down
Loading
Loading