During the Gemma 4 gibberish hunt the same throwaway harness was written five times: load the stock model (with the right key_mapping), load the distributed model, same prompt, greedy generate, compare ids/first-step scores.
Proposal: a drift verify <model> --initial_peers <maddr> subcommand that does this in one shot, plus a --per-span mode that captures hidden states at each span boundary (what a monkeypatched _ServerInferenceSession.step did during debugging) and reports cosine similarity and max-abs-diff per span against a local reference forward. The cos+maxdiff pair is the key diagnostic: cos~1.0 with huge maxdiff reads "scalar multiple" (a scale/buffer bug), low cos reads "wrong values" -- either localizes the fault to a span in one run.
Would have reduced a 4-hour debugging session (the layer_scalar bug, 5a90806) to one command.
🤖 Generated with Claude Code
During the Gemma 4 gibberish hunt the same throwaway harness was written five times: load the stock model (with the right key_mapping), load the distributed model, same prompt, greedy generate, compare ids/first-step scores.
Proposal: a
drift verify <model> --initial_peers <maddr>subcommand that does this in one shot, plus a--per-spanmode that captures hidden states at each span boundary (what a monkeypatched_ServerInferenceSession.stepdid during debugging) and reports cosine similarity and max-abs-diff per span against a local reference forward. The cos+maxdiff pair is the key diagnostic: cos~1.0 with huge maxdiff reads "scalar multiple" (a scale/buffer bug), low cos reads "wrong values" -- either localizes the fault to a span in one run.Would have reduced a 4-hour debugging session (the layer_scalar bug,
5a90806) to one command.🤖 Generated with Claude Code