test: add a documented local test entry point that cannot report a vacuous pass

Makefile

@@ -0,0 +1,16 @@
+# Local entry points.
+#
+# `make test` is the supported way to run the unit suites. A bare `swift test` on a fresh
+# checkout can die before the suite runs (MLX cannot find its Metal library) and still
+# exit 0, which looks like a short green pass; scripts/run-tests.sh bootstraps the metallib
+# and fails the run when too few tests executed.
+
+.PHONY: test test-filter
+
+test:
+	./scripts/run-tests.sh
+
+# make test-filter FILTER=PromptCacheTests
+test-filter:
+	@test -n "$(FILTER)" || { echo "usage: make test-filter FILTER=<XCTest filter>"; exit 2; }
+	./scripts/run-tests.sh --filter "$(FILTER)"

README.md

@@ -51,6 +51,39 @@ Then start the server (models download automatically if not cached):
 
 *(Add `--stream-experts` when running oversized MoE models to bypass macOS virtual memory swapping and stream expert layers directly from NVMe SSD.)*
 
+### Run the Tests
+
+```bash
+make test
+```
+
+That is the supported entry point. It is a thin wrapper over:
+
+```bash
+./scripts/bootstrap_local_tests.sh   # build the harness, make mlx.metallib findable
+swift test --skip-build
+```
+
+Do **not** trust a bare `swift test` on a fresh checkout. MLX aborts the test process when it
+cannot load its Metal library, and `swift test` can still exit 0 after printing
+`Failed to load the default metallib` — on the pre-6.4 layout that happens at the second
+test, so the run looks like a short green pass with almost nothing executed. `make test`
+runs the bootstrap and then **fails if fewer than 200 tests executed**, so this failure mode
+is reported instead of hidden.
+
+The bootstrap handles both toolchains automatically:
+
+- **Swift 6.4+ (Xcode 27)** — SwiftPM compiles the Cmlx kernels and embeds `default.metallib`
+  in each test bundle, so nothing is copied. It only checks the Metal Toolchain is installed
+  (`xcodebuild -downloadComponent MetalToolchain` if it is not). Copying a metallib into
+  `<bundle>.xctest/Contents/MacOS` here invalidates the bundle signature and breaks the next
+  incremental build at CodeSign.
+- **Swift 6.2 (Xcode 26, the CI runner)** — the metallib is not embedded; the bootstrap
+  fetches `mlx.metallib` from the `mlx` Python wheel and copies it next to the test binaries.
+
+To run a subset, `make test-filter FILTER=PromptCacheTests` (or pass `swift test` arguments
+to `scripts/run-tests.sh` directly).
+
 ## 📊 Performance: Mac mini M6 (base, 32 GB)
 
 The first SwiftLM numbers from a **32 GB** Mac. Every other table in this README comes from 64 GB hardware. Running the same context lengths on this M6 exposed memory bugs that 64 GB machines had been hiding, and they are fixed in this release (see [What 32 GB exposed](#what-32-gb-exposed)).

scripts/run-tests.sh

@@ -0,0 +1,55 @@
+#!/usr/bin/env bash
+# run-tests.sh — run the local unit suites and refuse to report a run that barely ran.
+#
+# Usage: scripts/run-tests.sh [extra `swift test` arguments...]
+#        scripts/run-tests.sh --filter PromptCacheTests
+#
+# Why this exists: MLX aborts the test process when it cannot load its Metal library. On the
+# pre-6.4 build layout that happens on the second test, and `swift test` can still exit 0
+# after printing "Failed to load the default metallib" — the run looks like a short pass
+# while ~270 tests never executed. `scripts/bootstrap_local_tests.sh` installs the metallib;
+# this wrapper calls it and then checks the run actually executed the suite.
+set -euo pipefail
+cd "$(dirname "$0")/.."
+
+# The suite is a few hundred tests (326 at the time of writing). The floor sits well below
+# that so ordinary additions and removals never need a bump, but far above the one or two a
+# broken metallib leaves behind.
+MIN_TESTS="${MIN_TESTS:-200}"
+
+bash scripts/bootstrap_local_tests.sh
+
+LOG="$(mktemp -t swiftlm-tests.XXXXXX)"
+trap 'rm -f "$LOG"' EXIT
+
+set +e
+swift test --skip-build --disable-swift-testing "$@" 2>&1 | tee "$LOG"
+status=${PIPESTATUS[0]}
+set -e
+
+# XCTest prints "Executed N tests" per suite and once for the whole run; the largest number
+# is the total for the run.
+executed=$(grep -oE "Executed [0-9]+ tests" "$LOG" | grep -oE "[0-9]+" | sort -n | tail -1)
+executed=${executed:-0}
+
+if [ "$status" -ne 0 ]; then
+    exit "$status"
+fi
+
+# The floor only makes sense for an unfiltered full run. A caller passing --filter is
+# deliberately narrowing, and a dozen passing tests is a correct result there.
+if [ "$#" -eq 0 ] && [ "$executed" -lt "$MIN_TESTS" ]; then
+    cat >&2 <<EOF
+
+error: ${executed} test(s) executed, expected at least ${MIN_TESTS}.
+
+A run this short usually means MLX could not load its Metal library: the test process
+died early while \`swift test\` still exited 0. Look for "Failed to load the default
+metallib" in the output above, then re-run scripts/bootstrap_local_tests.sh.
+
+If the suite genuinely shrank, set MIN_TESTS to the new floor.
+EOF
+    exit 1
+fi
+
+echo "=> ${executed} tests executed (floor ${MIN_TESTS})."