pull: weight-file symlink never created for large HF downloads (ranged path), leaves cache permanently broken

[matts-path]

Summary

basert pull <org>/<model> fails during conversion with No such file or directory opening a .safetensors weight file, even though the underlying blob was fully and correctly downloaded. The root cause is in HfFetcher::get_file (base-convert/crates/base-hub/src/fetch.rs): large weight files take a different download path than small metadata files, and that path never creates the snapshots/<rev>/<filename> symlink the rest of the pipeline expects.

basert version: 0.2.3 (macOS / Apple Silicon)

Repro 1 — Qwen/Qwen3-4B (sharded, 3 files)

$ basert pull Qwen/Qwen3-4B
...
Error: converting Qwen/Qwen3-4B

Caused by:
    0: opening shard ".../snapshots/<rev>/model-00001-of-00003.safetensors"
    1: opening ".../snapshots/<rev>/model-00001-of-00003.safetensors"
    2: No such file or directory (os error 2)

Inspecting the cache: all 3 weight blobs were present under blobs/ with byte-for-byte correct sizes (verified against HF's Content-Length/X-Linked-Size for each shard), but none of the 3 model-0000X-of-00003.safetensors symlinks existed in snapshots/<rev>/. Every non-weight symlink (config.json, tokenizer.json, vocab.json, merges.txt, generation_config.json) was present and correct.

Repro 2 — Qwen/Qwen3-0.6B (single file)

Same failure shape: model.safetensors blob fully downloaded (1,503,300,328 bytes, matches HF's reported size exactly) but its symlink was missing from the snapshot dir. Metadata symlinks again all present and correct.

Root cause

base-convert/crates/base-hub/src/fetch.rs, HfFetcher::get_file (~lines 352–395):

let facts = facts.filter(|f| !(f.xet && prefer_xet()) && f.size >= RANGED_MIN_BYTES);
let Some(facts) = facts else {
    return handle
        .download_file()
        .filename(filename)
        .revision(revision)
        .progress(Progress::new(BarProgress::new(filename.to_string())))
        .send()
        .with_context(|| format!("downloading {filename} from {repo}@{revision}"));
};

// Plain HTTPS blob, large: parallel and resumable (see `download`).
let dst = self.blob_path(repo, &facts.key);
if std::fs::metadata(&dst).map(|m| m.len()).unwrap_or(0) == facts.size {
    return Ok(dst);
}
...
crate::download::download_ranged(&mint, repo, revision, filename, facts.size, &dst, resolve_max_retries())
    .with_context(...)?;
Ok(dst)

• Small files (< RANGED_MIN_BYTES, 32MiB) go through hf-hub's own handle.download_file()...send(), which writes the blob and creates the snapshots/<rev>/<filename> symlink internally. The module doc even says so explicitly (fetch.rs:29-32): "hf-hub's own single-stream download (and its snapshot/symlink cache bookkeeping) is the better trade" — implying awareness that this bookkeeping exists, but apparently not carried over to the other branch.
• Large files (weight shards) take the download_ranged branch. self.blob_path() only computes a path under blobs/<hash> — no snapshot directory is involved. get_file downloads into that blob path and returns it directly (Ok(dst)). No symlink is ever created for this branch. I grepped the whole base-hub/base-convert crates — the only std::os::unix::fs::symlink calls are in test fixtures, never in this production path.

The caller, download_source in base-convert/crates/base-convert/src/hub.rs (~lines 762–791), makes this worse by trusting the omission:

let cfg = fetcher.get_file(repo, revision, "config.json")?;
...
let snapshot = cfg.parent().map(|p| p.to_path_buf())...;
for f in &wanted {
    if f.as_str() == "config.json" { continue; }
    fetcher.get_file(repo, revision, f)?;   // return value discarded
}
Ok(snapshot)

It anchors the snapshot dir on config.json's parent, then calls get_file for every other wanted file (including the weights) and discards the returned path, assuming it landed beside config.json. For large files that assumption is false — the blob sits in blobs/, never linked into the snapshot dir that download_source just promised was complete.

Why re-running pull doesn't self-heal

The "already downloaded" check for large files (fetch.rs:379-381) only compares the blob's on-disk byte length against the expected size:

if std::fs::metadata(&dst).map(|m| m.len()).unwrap_or(0) == facts.size {
    return Ok(dst);
}

This never checks whether the snapshot symlink exists, so a missing symlink is invisible to the "is it cached" logic — pull (with or without --force) just re-confirms the blob is present and proceeds straight to conversion, hitting the same No such file or directory every time.

Why the test suite didn't catch this

The test double used to exercise download_source (StagedFetcher in hub.rs, around lines 1050–1088) always creates blob and symlink together in its .stage() setup helper — it models the correct end state, not the actual divergent behavior of HfFetcher::get_file's ranged-download branch. No test exercises a large file going through download_ranged and checks that a snapshot symlink results.

Workaround

Match blob sizes in blobs/ against the expected file size (via HF's resolve/<rev>/<filename> Content-Length/X-Linked-Size header) to identify which blob is which weight file, then manually create the missing symlink(s):

ln -s ../../blobs/<hash> model-00001-of-00003.safetensors

After that, basert pull converts successfully.

Suggested fix

In HfFetcher::get_file's ranged-download branch, create the snapshots/<rev>/<filename> symlink pointing at the blob after download_ranged succeeds (and after the size-match early-return), matching what hf-hub's download_file() does for the small-file path. Additionally, the "already cached" check should verify the snapshot symlink resolves correctly, not just that the blob byte count matches, so a partially-broken cache can be detected and repaired on a subsequent pull.