fix(hicache): accept no-layout logical pools on the v2 registration path - #317
Open
Wine93 wants to merge 1 commit into
Open
fix(hicache): accept no-layout logical pools on the v2 registration path#317Wine93 wants to merge 1 commit into
Wine93 wants to merge 1 commit into
Conversation
SGLang v0.5.17's DSV4 hybrid stack (build_deepseek_v4_hicache_stack -> attach_storage_backend) registers the *logical anchor* "kv" pool through register_mem_host_pool_v2 at attach time, and its DeepSeekV4PagedHostPool.get_page_buffer_meta() declares "no host layout" as a 2-tuple containing None (the older LogicalHostPool returned None). The v2 registration probe predates the logical-anchor fix (60c0522) and hard-raised "pool component probe returned no layout" on both wire forms; the RuntimeError propagated out of attach_storage_backend and killed the whole SGLang scheduler, forcing users to disable HiCache (observed with DFKV 2.1.1/2.10.0/2.15.1 + deepseek-v4-flash-0731). - _meta_has_no_layout(): None or a 2-tuple with None members is an explicit no-layout declaration; malformed shapes (wrong arity, empty/mismatched components) still fail closed. - register_mem_host_pool_v2: a no-layout pool registers marker-only (_pool_logical, replicated=True so MLA follower ranks skip writes) instead of raising; re-registering with a real layout clears the flag. - _v2_io: logical pools write the one-byte anchor markers on put (the same "kv" sub-keys as the v1 anchor path, riding _put_flat's exist gate and retry) and no-op complete on get so the hybrid controller proceeds to load the real side pools; a late no-layout declaration at transfer time flips the pool to marker-only the same way. - batch_set_v1/batch_get_v1: the logical-anchor branch now accepts the tuple-with-None wire form too. Test: FakeTupleNoneAnchorPool + 3 new cases (v1 tuple-None contract writes markers, v2 registration accepts both wire forms with set/get keeping marker semantics, physical re-registration clears the logical flag); test_dfkv_hicache 76 passed, device-direct 23 passed/10 skipped.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
SGLang v0.5.17's DSV4 hybrid stack (build_deepseek_v4_hicache_stack -> attach_storage_backend) registers the logical anchor "kv" pool through register_mem_host_pool_v2 at attach time, and its
DeepSeekV4PagedHostPool.get_page_buffer_meta() declares "no host layout" as a 2-tuple containing None (the older LogicalHostPool returned None). The v2 registration probe predates the logical-anchor fix (60c0522) and hard-raised "pool component probe returned no layout" on both wire forms; the RuntimeError propagated out of attach_storage_backend and killed the whole SGLang scheduler, forcing users to disable HiCache (observed with DFKV 2.1.1/2.10.0/2.15.1 + deepseek-v4-flash-0731).
Test: FakeTupleNoneAnchorPool + 3 new cases (v1 tuple-None contract writes markers, v2 registration accepts both wire forms with set/get keeping marker semantics, physical re-registration clears the logical flag); test_dfkv_hicache 76 passed, device-direct 23 passed/10 skipped.