Repository navigation
ci: nightly macos leg runs only on ram-48g runners - #37
Merged
Merged
Conversation
Any of three self-hosted Macs took the leg. On the 24 GB and 16 GB ones the adapter keeps one shared KV pool, having too little Metal memory free for a buffer per lane, and two of its tests fail whatever the llama.cpp build: b11812 and b11889 were reported as breaking the adapter, and b11889 passes on a 48 GB Mac. Only runners labelled ram-48g take the leg now.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The nightly's llama.cpp macOS leg ran on
[self-hosted, macOS, ARM64], which any of three Macs answers. Only one of them can pass it:m4-max-1runner-base-m4prom1-mini-1With
streamsleft to the adapter, it gives each lane its own KV buffer only when 60% of the free Metal memory holds them all, and keeps one shared pool otherwise (runtime.rs,n_streams).the_ledger_follows_copy_chains_and_the_limits_follow_the_configasks for ten 8192-token buffers of Qwen3-0.6B, some 9.4 GB of KV, so it needs about 16 GB free, which a 24 GB Mac never has.the_sequence_table_stops_at_llama_cpps_limitfalls short in the same way. On the smaller runners both tests fail whatever the build, and the nightly then reports the build as breaking the adapter (#36).b11889 passes all of the adapter's tests on a 48 GB M5 Pro (34.2 GB budget), these two included, on the adapter code
rc-0.1.5has.The leg now asks for
[self-hosted, macOS, ARM64, ram-48g]. The label is onm4-max-1. With no such runner online the job waits, and after a day the record job notes that it did not finish and records nothing.Once merged,
gh workflow run nightly-runtimes.yml --ref main -f llamacpp=b11889re-tests b11889 and opens its PR.