Skip to content

fix(hicache): accept no-layout logical pools on the v2 registration path - #317

Open
Wine93 wants to merge 1 commit into
mainfrom
fix/hicache-v2-register-logical-pool
Open

fix(hicache): accept no-layout logical pools on the v2 registration path#317
Wine93 wants to merge 1 commit into
mainfrom
fix/hicache-v2-register-logical-pool

Conversation

@Wine93

@Wine93 Wine93 commented Aug 11, 2026

Copy link
Copy Markdown
Contributor

SGLang v0.5.17's DSV4 hybrid stack (build_deepseek_v4_hicache_stack -> attach_storage_backend) registers the logical anchor "kv" pool through register_mem_host_pool_v2 at attach time, and its
DeepSeekV4PagedHostPool.get_page_buffer_meta() declares "no host layout" as a 2-tuple containing None (the older LogicalHostPool returned None). The v2 registration probe predates the logical-anchor fix (60c0522) and hard-raised "pool component probe returned no layout" on both wire forms; the RuntimeError propagated out of attach_storage_backend and killed the whole SGLang scheduler, forcing users to disable HiCache (observed with DFKV 2.1.1/2.10.0/2.15.1 + deepseek-v4-flash-0731).

  • _meta_has_no_layout(): None or a 2-tuple with None members is an explicit no-layout declaration; malformed shapes (wrong arity, empty/mismatched components) still fail closed.
  • register_mem_host_pool_v2: a no-layout pool registers marker-only (_pool_logical, replicated=True so MLA follower ranks skip writes) instead of raising; re-registering with a real layout clears the flag.
  • _v2_io: logical pools write the one-byte anchor markers on put (the same "kv" sub-keys as the v1 anchor path, riding _put_flat's exist gate and retry) and no-op complete on get so the hybrid controller proceeds to load the real side pools; a late no-layout declaration at transfer time flips the pool to marker-only the same way.
  • batch_set_v1/batch_get_v1: the logical-anchor branch now accepts the tuple-with-None wire form too.

Test: FakeTupleNoneAnchorPool + 3 new cases (v1 tuple-None contract writes markers, v2 registration accepts both wire forms with set/get keeping marker semantics, physical re-registration clears the logical flag); test_dfkv_hicache 76 passed, device-direct 23 passed/10 skipped.

SGLang v0.5.17's DSV4 hybrid stack (build_deepseek_v4_hicache_stack ->
attach_storage_backend) registers the *logical anchor* "kv" pool through
register_mem_host_pool_v2 at attach time, and its
DeepSeekV4PagedHostPool.get_page_buffer_meta() declares "no host layout"
as a 2-tuple containing None (the older LogicalHostPool returned None).
The v2 registration probe predates the logical-anchor fix (60c0522) and
hard-raised "pool component probe returned no layout" on both wire
forms; the RuntimeError propagated out of attach_storage_backend and
killed the whole SGLang scheduler, forcing users to disable HiCache
(observed with DFKV 2.1.1/2.10.0/2.15.1 + deepseek-v4-flash-0731).

- _meta_has_no_layout(): None or a 2-tuple with None members is an
  explicit no-layout declaration; malformed shapes (wrong arity,
  empty/mismatched components) still fail closed.
- register_mem_host_pool_v2: a no-layout pool registers marker-only
  (_pool_logical, replicated=True so MLA follower ranks skip writes)
  instead of raising; re-registering with a real layout clears the flag.
- _v2_io: logical pools write the one-byte anchor markers on put (the
  same "kv" sub-keys as the v1 anchor path, riding _put_flat's exist
  gate and retry) and no-op complete on get so the hybrid controller
  proceeds to load the real side pools; a late no-layout declaration at
  transfer time flips the pool to marker-only the same way.
- batch_set_v1/batch_get_v1: the logical-anchor branch now accepts the
  tuple-with-None wire form too.

Test: FakeTupleNoneAnchorPool + 3 new cases (v1 tuple-None contract
writes markers, v2 registration accepts both wire forms with set/get
keeping marker semantics, physical re-registration clears the logical
flag); test_dfkv_hicache 76 passed, device-direct 23 passed/10 skipped.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant