diff --git a/docs/source/io_pattern_design.md b/docs/source/io_pattern_design.md
new file mode 100644
index 0000000000..a1bb5fbea6
--- /dev/null
+++ b/docs/source/io_pattern_design.md
@@ -0,0 +1,1284 @@
+---
+orphan: true
+---
+## 5.4.1 设计概述
+
+```
+flowchart TD
+ subgraph 推理框架层 ["推理框架层 (vLLM/SGLang)"]
+ MC["MooncakeConnector
(vLLM v1)"]
+ HC["HiCache Connector
(SGLang)"]
+ PC["Prefix Cache Manager"]
+ end
+
+ subgraph CFM ["CFM (Cache Flow Manager)"]
+ COL["Collector
(数据采集)"]
+ ANA["Analyzer
(模式分析)"]
+ ENG["Policy Engine
(策略决策)"]
+ COL --> ANA --> ENG
+ EV["Eviction Ops"]
+ PF["Prefetch Ops"]
+ AD["Admission Ops"]
+ ENG --> EV & PF & AD
+ end
+
+ MC -->|"CFM Client
(采集/策略/预取)"| COL
+ HC -->|"CFM Client"| COL
+ PC -->|"CFM Client"| COL
+
+ subgraph CVM ["CVM (Cache View Manager)"]
+ VIEW["视图计算 / 发布 / 系统事件感知 / 全局 KV 映射表"]
+ end
+
+ EV --> VIEW
+ PF --> VIEW
+ AD --> VIEW
+
+ subgraph 存储层 ["存储层"]
+ L0["L0: HBM
(UB2PCIe/d2h)"]
+ L1["L1: Host DRAM/SSD
(计算节点本地内存/SSD(xds)"]
+ L2["L2: Segment DRAM
(池化内存 URMA mem)"]
+ L3["L3: Nof SSD
(SSU/远端池化 SSD)"]
+ end
+
+ VIEW --> L0
+ VIEW --> L2
+ VIEW --> L3
+ L0 <-.->|"tier down/up"| L1
+ L1 <-.->|"tier down/up"| L2
+ L2 <-.->|"offload/promotion"| L3
+```
+
+**模块总体架构图**
+
+```
+classDiagram
+ class IoPatternCollector {
+ <>
+ +ReportInferenceMetrics(metrics) void
+ +RecordAccess(key, record) void
+ +RecordStorageMetric(metric) void
+ +GetSnapshot() IoPatternSnapshot
+ }
+
+ class IoPatternAnalyzer {
+ <>
+ +AnalyzePattern(snapshot) PatternResult
+ +DetectWorkloadType(window) WorkloadType
+ +CalculateConfidence(key) float
+ }
+
+ class PolicyEngine {
+ <>
+ +ExecutePolicy(context, tier, bytes, trace, admissions) PolicyResult
+ }
+
+ class EvictionOps {
+ <>
+ +Evaluate(context, tier, bytes) EvictionPlan
+ }
+
+ class PrefetchOps {
+ <>
+ +Evaluate(context, trace) PrefetchPlan
+ }
+
+ class AdmissionOps {
+ <>
+ +Evaluate(object, tier, context) AdmissionResult
+ }
+
+ class CfmClient {
+ +ReportInferenceMetrics(metrics) void
+ +ReceivePolicy指令() void
+ +ExecutePrefetch(candidates) void
+ }
+
+ class ScoreBasedEviction {
+ +Evaluate(context, tier, bytes) EvictionPlan
+ }
+
+ class TraceBasedPrefetch {
+ +Evaluate(context, trace) PrefetchPlan
+ }
+
+ class PrefixMatchAdmission {
+ +Evaluate(object, tier, context) AdmissionResult
+ }
+
+ PolicyEngine *-- EvictionOps : contains
+ PolicyEngine *-- PrefetchOps : contains
+ PolicyEngine *-- AdmissionOps : contains
+ IoPatternCollector --> IoPatternAnalyzer : reports
+ IoPatternAnalyzer --> PolicyEngine : analyzes
+ CfmClient --> IoPatternCollector : reports metrics
+ CfmClient --> PolicyEngine : receives policy
+ EvictionOps <|.. ScoreBasedEviction : implements
+ PrefetchOps <|.. TraceBasedPrefetch : implements
+ AdmissionOps <|.. PrefixMatchAdmission : implements
+```
+
+## 5.4.2 IO Pattern 三层架构
+
+IO Pattern 模块采用**采集层 -> 分析层 -> 策略层**的三层架构。
+
+### 5.4.2.1 采集层 (IO Pattern Collector)
+
+采集层负责从各数据源采集原始 IO 指标,采用异步上报机制避免阻塞数据路径。
+
+**采集来源分三层:**
+
+| 采集层 | 数据源 | 采集指标 | 现有代码锚点 |
+| ----------------- | ------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------ | ----------------------------------------------------------------------- |
+| 推理框架层 | vLLM MooncakeConnector / SGLang HiCache Connector | prefix match length, request priority, token 序列, recompute cost | `mooncake_connector_v1.py`, SGLang hicache connector |
+| SuperCache SDK 层 | Client / Master / SubMaster | Get/Put/Tier 命中率, 访问时序, key 频率, 前缀树深度/fanout, 副本分布, 迁移 ETA, 写路径指标 (batch_size, overwrite_ratio) | `client_service.cpp`, `master_service.cpp`, `local_hot_cache.cpp` |
+| 存储后端层 | SSD/Nof Segment | 读写带宽, 读写延迟, GC 状态, 盘内 Superblock 布局, 容量水位 | `client_metric.h:SsdMetric`, `allocation_strategy.h:SsdMetricsProvider` |
+
+**采集机制设计:**
+
+1. **轻量级埋点**:复用和扩展现有 `CountMinSketch`(频率统计)、`SsdMetric`(SSD 延迟/吞吐)、`storage_backend.h:last_access_ns_`(最后访问时间)等埋点,避免重复建设
+2. **异步上报**:CFM Client 定期异步上报指标至 SubMaster,采用 batch 聚合减少 RPC 开销。上报间隔自适应负载(低负载 100ms,高负载退避至 500ms-1s)
+3. **全局聚合**:SubMaster 聚合各节点上报的指标,维护全局 token 指标流动视图
+4. **采样降级**:在高负载场景下支持采样率动态调整,优先保障数据路径性能
+5. **多租户隔离**:指标按 `TenantId` 分桶采集,避免高频租户淹没低频租户,沿用现有 `CountMinSketch` 的 `tenant_id.MakeScopedKey` 模式
+
+固定 100ms 上报间隔在大规模集群下可能产生可观开销。采用自适应间隔:
+
+| 负载状态 | 上报间隔 | 触发条件 |
+| -------- | -------- | ----------------------------------------- |
+| 低负载 | 100ms | mem_used_ratio < 50% |
+| 中负载 | 200ms | 50% <= mem_used_ratio < 80% |
+| 高负载 | 500ms | mem_used_ratio >= 80% |
+| 极高负载 | 1000ms | mem_used_ratio >= 95% 或 RPC 延迟 > 100ms |
+
+>
+> 高负载时拉长间隔减少 RPC 开销,但保持最低 1s 上报频率确保策略时效性。量化估算:4000 节点集群,100ms 间隔下每秒 40000 RPC,单 RPC ~2KB,总带宽 ~80MB/s
+
+#### 5.4.2.1.1 指标采集与上报流程
+
+```
+sequenceDiagram
+ participant INF as 推理框架 (vLLM/SGLang)
+ participant CFM as CFM Client
+ participant SUB as SubMaster
+
+ INF->>CFM: 1. 请求完成/前缀匹配
+ CFM->>SUB: 2. 批量上报指标 (InferenceMetrics)
+ CFM->>SUB: 3. SDK 层埋点 (AccessRecord)
+ CFM->>SUB: 4. 存储后端指标 (StorageMetric)
+ Note over SUB: 5. 全局聚合
IoPatternSnapshot
+```
+
+#### 5.4.2.1.2 指标分类
+
+IO Pattern 采集指标分为六大类,对应分级缓存淘汰/准入/预取流程的采集指标定义:
+
+**时序指标**
+
+| 指标名 | 类型 | 描述 | 采集来源 | 现有代码锚点 |
+| --------------------- | --------- | ------------------------ | ---------- | ----------------------------------- |
+| `last_access_time` | timestamp | 最近一次访问时间 | SDK 层 | `storage_backend.h:last_access_ns_` |
+| `access_count_window` | uint32 | 最近时间窗口内访问次数 | SDK 层 | `count_min_sketch.h:CountMinSketch` |
+| `idle_time` | duration | = now - last_access_time | 分析层计算 | - |
+
+**价值指标**
+
+| 指标名 | 类型 | 描述 | 采集来源 | 现有代码锚点 |
+| ---------------- | ------ | ------------------------------ | ---------- | ---------------- |
+| `recompute_cost` | float | 重新计算时间 (token 数 / 时间) | 推理框架层 | connector 层估算 |
+| `block_size` | uint64 | 数据大小 (bytes) | SDK 层 | object metadata |
+| `token_count` | uint32 | token 数 | 推理框架层 | connector 层 |
+
+**结构指标**
+
+| 指标名 | 类型 | 描述 | 采集来源 | 现有代码锚点 |
+| -------------------------- | ------ | ---------------------------- | ---------- | -------------------- |
+| `prefix_depth` | uint32 | 前缀深度 (prefix tree level) | 推理框架层 | prefix cache manager |
+| `prefix_fanout` | uint32 | 共享该前缀的请求/分支数量 | 推理框架层 | prefix cache manager |
+| `match_length` | uint32 | 前缀匹配长度 | 推理框架层 | connector 层 |
+| `continuous_prefix_length` | uint32 | 连续前缀长度 | 推理框架层 | connector 层 |
+
+**副本指标**
+
+| 指标名 | 类型 | 描述 | 采集来源 | 现有代码锚点 |
+| --------------------- | -------- | -------------------------- | ---------- | -------------------------- |
+| `replica_tiers` | bitmap | 当前在哪些层有副本 (L0-L3) | SDK 层 | `master_service.h:Replica` |
+| `transfer_eta` | duration | 迁移路径预计耗时 | 分析层计算 | - |
+| `ssd_replica_exists` | bool | SSD 层是否有副本 | SDK 层 | replica metadata |
+| `other_replica_count` | uint32 | 其他层副本数 | SDK 层 | replica metadata |
+
+**状态指标**
+
+| 指标名 | 类型 | 描述 | 采集来源 | 现有代码锚点 |
+| -------- | ---- | ------------ | -------- | ----------------------------- |
+| `active` | bool | 是否正在使用 | SDK 层 | `local_hot_cache.h:ref_count` |
+| `pinned` | bool | 是否不可迁移 | SDK 层 | promotion task pinning |
+
+**存储后端指标**
+
+| 指标名 | 类型 | 描述 | 采集来源 | 现有代码锚点 |
+| ------------------- | --------- | ----------------- | ---------- | ---------------------------------------------------------------------------- |
+| `ssd_read_latency` | histogram | SSD 读延迟分布 | 存储后端层 | `client_metric.h:SsdMetric` |
+| `ssd_write_latency` | histogram | SSD 写延迟分布 | 存储后端层 | `client_metric.h:SsdMetric` |
+| `ssd_gc_status` | enum | SSD GC 状态 | 存储后端层 | Nof TGT |
+| `mem_used_ratio` | float | DRAM 内存水位比例 | SDK 层 | `MasterMetricManager::get_global_mem_used_ratio()` (Master 侧全局 DRAM 水位) |
+| `ssd_used_bytes` | int64 | SSD 已用容量 | 存储后端层 | `allocation_strategy.h:SsdMetricsProvider` |
+
+#### 5.4.2.1.3 指标采集接口
+
+```
+// IO Pattern Collector 接口 (新增, 位于 include/io_pattern_collector.h)
+class IoPatternCollector {
+ public:
+ virtual ~IoPatternCollector() = default;
+
+ // 推理框架层指标 (通过 CFM Client 上报)
+ virtual void ReportInferenceMetrics(const InferenceMetrics& metrics) = 0;
+
+ // SDK 层指标 (内部埋点)
+ virtual void RecordAccess(const std::string& key,
+ const AccessRecord& record) = 0;
+
+ // 存储后端层指标
+ virtual void RecordStorageMetric(const StorageMetric& metric) = 0;
+
+ // 获取聚合后的指标快照
+ virtual IoPatternSnapshot GetSnapshot() const = 0;
+};
+
+struct AccessRecord {
+ std::string key;
+ std::chrono::steady_clock::time_point access_time;
+ uint64_t block_size;
+ ReplicaType replica_type;
+ bool is_hit;
+ std::chrono::microseconds latency;
+};
+
+struct InferenceMetrics {
+ std::string session_id;
+ uint32_t prefix_depth;
+ uint32_t prefix_fanout;
+ uint32_t match_length;
+ uint32_t continuous_prefix_length;
+ uint32_t token_count;
+ float recompute_cost;
+ uint8_t request_priority;
+};
+```
+
+### 5.4.2.2 分析层 (IO Pattern Analyzer)
+
+分析层对采集的原始指标进行模式识别和特征提取,输出结构化的 IO Pattern 描述。
+
+**分析能力:**
+
+| 分析类型 | 描述 | 输入指标 | 输出 | 对应需求 |
+| ------------- | ------------------------------ | ------------------------------------------------------ | ------------------------------------ | -------------- |
+| 热度分析 | 基于 LFU/滑动窗口的频率统计 | access_count_window, last_access_time | hot/cold 分类, 频率评分 | 淘汰/准入/预取 |
+| 前缀分析 | Prefix tree 深度和 fanout 分析 | prefix_depth, prefix_fanout, match_length | 前缀共享度, 预取候选 | 预取/准入 |
+| 时序预测 | 访问间隔和空闲时间分析 | idle_time, access pattern, access_count_window | 空闲评分, 预取优先级 | 淘汰/Tier down |
+| 代价评估 | 重计算代价和迁移代价评估 | recompute_cost, token 数, block_size, transfer_eta | 代价评分, 迁移 ROI | 淘汰/Tier up |
+| 访问模式识别 | 顺序/随机、大包/小包、读写比 | IO size distribution, access sequence | 模式分类 (SEQ/RANDOM/KV_LOOKUP等) | 缓存分区/分流 |
+| 副本分析 | 多层副本分布分析 | replica_tiers, active/pinned | 副本冗余度, 迁移安全性 | 淘汰/Tier down |
+| 写路径分析 | 写入模式分析 | write_batch_size, write_burst, overwrite_ratio | 写穿风险, GC 预警 | 准入/GC |
+| Workload 识别 | 推理场景类型自动识别 | token_count, prefix_fanout, block_size, frequency 分布 | workload_type (Code Agent/推荐/对话) | 策略模板选择 |
+
+**Workload Type 感知策略模板**
+
+不同推理场景的 IO Pattern 差异巨大,单一通用评分公式无法覆盖所有场景。
+
+**配置参数:**
+
+| 参数 | 默认值 | 说明 |
+| -------------------------------------- | ------------- | ------------------------------------------------------------------ |
+| `workload_detection_window_sec` | 60 | workload 识别滑动窗口大小 |
+| `workload_detection_method` | `auto` | 识别方法:`auto`(阈值+聚类)、`threshold`(仅阈值)、`kmeans`(仅聚类) |
+| `workload_template_transition_windows` | 3 | 模板切换过渡窗口数,控制平滑度 |
+| `workload_mixed_load_mode` | `per_session` | 混合负载处理:`per_session`(按会话标记)、`global`(全局统一) |
+
+**运维观测:**
+
+| 指标 | 描述 |
+| --------------------------------------- | ------------------------ |
+| `workload_current_type` | 当前识别的 workload type |
+| `workload_type_switch_count` | workload type 切换次数 |
+| `workload_detection_latency_us` | 单次识别延迟 |
+| `workload_template_transition_progress` | 模板过渡进度 (0.0-1.0) |
+
+**Workload 识别机制:**
+
+不同推理场景的 KVCache 访问模式差异巨大,单一通用评分公式无法覆盖所有场景。IO Pattern 分析层基于滑动窗口内的指标分布统计,自动识别 workload type 并切换对应策略模板。
+
+**Workload Type 特征矩阵:**
+
+| Workload Type | 典型场景 | 访问特征 | 识别信号 |
+| ------------- | ------------------------------ | ------------------------------------------------------------------------- | ----------------------------------------------------------------------- |
+| Code Agent | Cursor/Copilot 长程代码生成 | 长上下文(>32K token)、高前缀复用、大 block_size(>512KB)、多轮访问为主 | 高 token_count + 低 prefix_fanout + 大 block_size + 低 frequency |
+| 生成式推荐 | 京东/字节 GR 精排召回 | 高频小 block(<128KB)、高复用、密集访问、低重计算代价 | 低 token_count + 高 frequency + 小 block_size + 低 recompute_cost |
+| 多轮对话 | ChatGPT 类对话、Agent 工具调用 | 中等 block、高前缀共享、渐进式增长、prefix cache 命中率高 | 中 token_count + 高 prefix_fanout + 高 match_length + 高 recompute_cost |
+
+**识别算法:**
+
+分析层维护一个滑动窗口(默认 60s),统计窗口内所有请求的 `token_count`/`prefix_fanout`/`block_size`/`frequency` 四维分布。采用两阶段识别:
+
+```
+阶段 1: 特征提取
+ for each request in window:
+ feature_vector = (median(token_count), p90(prefix_fanout),
+ median(block_size), median(frequency))
+
+阶段 2: 分类决策
+ if feature_vector matches阈值规则:
+ -> 直接分类 (快速路径, 延迟 < 1ms)
+ else:
+ -> K-means 聚类 (慢速路径, 延迟 < 10ms, 用于混合负载场景)
+```
+
+**阈值规则(快速路径):**
+
+| 判定条件 | → Workload Type |
+| ---------------------------------------------------------------------------------- | ----------------------- |
+| `median(token_count) > 16KB && p90(prefix_fanout) > 16 && p90(match_length) > 256` | Code Agent |
+| `median(block_size) < 128KB && median(frequency) > 20` | 生成式推荐 |
+| `p90(prefix_fanout) > 16 && p90(match_length) > 256` | 多轮对话 |
+| 不满足以上任一 | 混合负载 → K-means 聚类 |
+
+**策略模板对照:**
+
+每种 workload type 对应一组完整的策略参数模板,覆盖淘汰/准入/预取/Tier 四个维度:
+
+| 策略维度 | Code Agent | 生成式推荐 | 多轮对话 |
+| ------------- | ----------------------------------------- | --------------------------------------- | -------------------------------------- |
+| **预取** | 保守(仅 prefix > 512 预取,best_effort) | 激进(prefix > 64 预取,wait_complete) | 前缀优先(prefix > 256 预取,timeout) |
+| **淘汰** | 激进(低 idle_thres,快速释放 L0) | 保守(高 idle_thres,保留热数据) | 前缀感知(prefix_fanout 高权重保留) |
+| **准入** | 低阈值(access_count > 2 即准入) | 高阈值(access_count > 20 才准入) | 前缀准入(match_length > 128 即准入) |
+| **Tier down** | 快速降级(L0→L2 跳级,跳过 L1) | 缓慢降级(L0→L1→L2 逐层) | 前缀亲和(共享前缀的 block 同层迁移) |
+| **淘汰权重** | α=0.8, γ=0.2, δ=0.3, ε=0.1 | α=0.3, γ=0.8, δ=0.2, ε=0.1 | α=0.5, γ=0.4, δ=0.6, ε=0.8 |
+
+**模板切换机制:**
+
+```
+flowchart TD
+ WIN["滑动窗口指标统计
(60s)"]
+ FEAT["特征提取
4 维分布向量"]
+ RULE["阈值规则匹配"]
+ KMEANS["K-means 聚类
(混合负载)"]
+ CLASSIFY["Workload Type 判定"]
+ TEMPLATE["策略模板加载
(淘汰/准入/预取/Tier 参数)"]
+ APPLY["应用至 Policy Engine"]
+
+ WIN --> FEAT --> RULE
+ RULE -->|"匹配成功"| CLASSIFY
+ RULE -->|"不匹配"| KMEANS --> CLASSIFY
+ CLASSIFY --> TEMPLATE --> APPLY
+```
+
+**切换平滑性:** workload type 变化时,策略参数不是瞬间切换,而是通过加权过渡(新旧模板权重在 3 个窗口周期内从 100:0 渐变到 0:100),避免策略突变导致缓存抖动。
+
+**混合负载处理:** 当 K-means 识别出多种 workload type 共存时(如同一集群同时服务对话和推荐),采用 per-session workload 标记——在请求入口处根据 session 特征打标签,各 session 独立使用对应模板,而非全局统一。
+
+### 5.4.2.3 策略层 (IO Pattern Policy Engine)
+
+策略层基于分析层的输出,通过可注册的 Ops 接口驱动各缓存机制:
+
+```
+flowchart TD
+ PE["Policy Engine"]
+
+ subgraph EvictionOps ["EvictionOps (淘汰策略)"]
+ LRU["LRU"]
+ LFU["LFU"]
+ SBE["ScoreBased
(L0-L3 四层)"]
+ end
+
+ subgraph PrefetchOps ["PrefetchOps (预取策略)"]
+ BE["BestEffort"]
+ TO["Timeout"]
+ WC["WaitComplete"]
+ TB["TraceBased"]
+ end
+
+ subgraph AdmissionOps ["AdmissionOps (准入策略)"]
+ FREQ["Frequency"]
+ PM["PrefixMatch"]
+ WM["Watermark"]
+ CA["CostAware"]
+ end
+
+ PE --> EvictionOps
+ PE --> PrefetchOps
+ PE --> AdmissionOps
+```
+
+## 5.4.3 缓存机制集成与关键流程
+
+IO Pattern 模块不是重写现有机制,而是在现有机制之上增加统一的数据采集和分析层,通过 Ops 抽象接口驱动各机制。
+
+### 5.4.3.2 淘汰 (Eviction)
+
+**现有机制**:`EvictionStrategy` 抽象类(`eviction_strategy.h`)提供 LRU 和 FIFO 两种实现;`storage_backend.h` 中基于 `last_access_ns_` 维护 LRU 索引。
+
+**IO Pattern 增强**:引入基于评分的淘汰策略 (ScoreBasedEviction),按4层缓存层级分别使用不同评分公式。所有指标先经归一化处理(`normalize(x) = x / max_observed_x`,映射到 `[0, 1]`),消除量纲差异后再加权求和。归一化基准基于滑动窗口(默认 60s)内的最大观测值动态更新。
+
+```
+flowchart TD
+ SNAP["IO Pattern Snapshot"]
+ NORM["归一化处理
norm(x) = x / max_observed"]
+ PE["Policy Engine
选择层级策略"]
+ SNAP --> NORM --> PE
+
+ PE --> L0S["L0 HBM Evict"]
+ PE --> L1S["L1 Host DRAM/SSD Evict"]
+ PE --> L2S["L2 Segment DRAM Evict"]
+ PE --> L3S["L3 Nof SSD Evict"]
+
+ L0S --> L0F["α\*norm(idle) - γ\*norm(freq)
- δ\*norm(recompute) - ε\*norm(fanout)"]
+ L1S --> L1F["α\*norm(idle) - γ\*norm(freq)
+ δ\*norm(lower_replica)
- ε\*norm(fanout) - ζ*norm(recompute)"]
+ L2S --> L2F["α\*norm(idle) - γ\*norm(freq)
+ δ\*norm(lower_replica)
- ε\*norm(fanout) - ζ*norm(recompute)"]
+ L3S --> L3F["α\*norm(idle)\*norm(block_size)
- γ\*norm(freq) - δ\*norm(recompute)
+ η*norm(other_replica)"]
+
+ L0F --> L0OUT["-> L1"]
+ L1F --> L1OUT["-> L2"]
+ L2F --> L2OUT["-> L3"]
+ L3F --> L3OUT["-> 丢弃"]
+```
+
+**各层淘汰策略说明:**
+
+| 层级 | 评分侧重 | 淘汰去向 | 说明 |
+| ---------------- | --------------------------------------------------------------------- | -------- | ------------------------- |
+| L0 HBM | `idle_time` 主导,`recompute_cost`/`prefix_fanout` 高权重保留 | → L1 | HBM 最贵,冷数据快速降级 |
+| L1 Host DRAM/SSD | 下层已有副本可安全淘汰,`prefix_fanout`/`recompute_cost` 高的数据保留 | → L2 | 本地 DRAM/SSD 到远端 DRAM |
+| L2 Segment DRAM | `prefix_fanout`/`recompute_cost` 高的数据保留 | → L3 | 池化内存到远端 SSD |
+| L3 Nof SSD | `block_size` 大 + 其他层已有副本优先淘汰 | → 丢弃 | 最底层,无下降空间 |
+
+**集成方式**:扩展现有 `EvictionStrategy` 接口,新增 `ScoreBasedEvictionStrategy`,由 Policy Engine 根据层级动态选择策略。
+
+### 5.4.3.3 准入 (Admission)
+
+**现有机制**:
+
+- Client 侧:`CountMinSketch` + `admission_threshold_` 频率准入(`client_service.cpp`),仅频繁访问的 key 提升 hot cache
+- Master 侧:Promotion-on-Hit 的 `promotion_admission_threshold_` 频率门控 + watermark 门控(`master_service.cpp`)
+
+**IO Pattern 增强**:扩展准入策略为多层逐级准入控制,每层提升需满足对应条件:
+
+| 准入路径 | 条件 | 说明 |
+| -------- | -------------------------------------------------------------- | ----------------------------------------------------- |
+| L3→L2 | `access_count_window >= threshold` | 频率达标才从 Nof SSD 提升至 Segment DRAM |
+| L2→L1 | `access_count_window >= threshold && upper_space <= max_space` | 频率达标且上层有空间才提升 Segment DRAM→Host DRAM/SSD |
+| L1→L0 | `max_length >= threshold (64)` | 前缀长度达标才从 Host DRAM/SSD 提升至 HBM |
+
+>
+> SSD→HBM 跨层直达(跳过中间层)仅由推理框架 prefix cache 命中时触发,Mooncake 侧不自主执行跨层晋升到 HBM。
+
+```
+flowchart TD
+ REQ["访问请求"]
+ ANA["IO Pattern Analyzer
计算准入条件"]
+ REQ --> ANA
+
+ ANA --> P1["L3→L2
access_cnt >= thres"]
+ ANA --> P2["L2→L1
access_cnt >= thres
&& upper_space <= max"]
+ ANA --> P3["L1→L0
max_length >= 64"]
+
+ P1 --> FA["频率准入
(CountMin Sketch)"]
+ P2 --> FA2["频率+空间准入
(Frequency + Watermark)"]
+ P3 --> PA["前缀准入
(PrefixMatch Admission)"]
+```
+
+**集成方式**:扩展现有 `CountMinSketch` 准入逻辑,新增 `PrefixMatchAdmission` 和 `CostAwareAdmission` 策略。
+
+### 5.4.3.4 Tier Down / SSD Offload
+
+**现有机制**:
+
+- `enable_ssd_offload` + `ssd_offload_path` 配置 SSD offload 路径(`real_client.cpp`)
+- `offload_on_evict` 模式:在淘汰时延迟 offload 到 LOCAL_DISK(`master_service.cpp`)
+- `offload_force_evict`:超过 offload cap 时直接淘汰不 offload
+
+**IO Pattern 增强**:基于热度阈值的逐级 tier down,数据按 L0→L1→L2→L3 顺序逐层降级:
+
+```
+flowchart TD
+ START["L0 HBM 容量/水位检测"]
+ C1{"idle_time >= L0 cold_thres || frequency < L0 hot_thres ?"}
+ C2{"L2 seg_dram_avail ?"}
+ C3{"L3 nof SSD avail ?"}
+ C4{"L1 host DRAM<= thres ?"}
+ C5{"xds available ?"}
+ C6{"idle_time >= L1 cold_thres || frequency < L1 hot_thres ?"}
+ C7{"idle_time >= L2 cold_thres || frequency < L2 hot_thres ?"}
+ TD_L1A["L1 Host DRAM"]
+ TD_L1B["L1 SSD(xds)"]
+ TD_L2["L2 Segment DRAM"]
+ TD_L3["L3 Nof SSD"]
+
+ START --> C1
+ C1 -->|是| C4
+ C4 -->|是| TD_L1A
+ C4 -->|否| C5
+ C5 -->|是| TD_L1B
+ C5 -->|否| C2
+ C2 -->|是| TD_L2
+ C2 -->|否| C3
+ C3 -->|否| WAIT["下层均不可用
等待重试 / 强制 evict"]
+ C3 -->|是| TD_L3
+ TD_L1A --> C6
+ C6 -->|是| C2
+ TD_L2 --> C7
+ C7 -->|是| C3
+```
+
+>
+> 当所有下层均不可用时,数据暂留当前层并等待下层恢复,或触发强制 evict 释放空间。冷数据不会保留在高速层——高速层是最昂贵的资源,冷数据必须逐级降级。
+
+### 5.4.3.5 Tier Up / Promotion-on-Hit
+
+**现有机制**:
+
+- `promotion_on_hit` 模式(`master_service.cpp:379`):Get 观察到 LOCAL_DISK-only key 时队列异步拷贝回 MEMORY
+- `CountMinSketch` 频率门控(`master_service.cpp:6944`)
+- watermark 门控:DRAM 低于 `eviction_high_watermark_ratio_` 才允许 promotion
+- `promotion_queue_limit` + `promotion_max_per_heartbeat` 控制 promotion 速率
+- `PromotionCandidate` 跟踪 + 重试 + TTL 过期
+
+**IO Pattern 增强**:IO Pattern 分析层为 promotion 提供更丰富的决策输入:
+
+- 前缀匹配度:高前缀匹配的 key 优先 promotion
+- 重计算代价:高 recompute_cost 的 key 优先 promotion
+- 迁移 ETA:根据带宽和 block_size 估算 transfer_eta,避免迁移耗时过长
+
+>
+> Mooncake 侧 promotion 仅执行逐级提升(L3→L2→L1),不自主晋升到 L0 HBM。L0 HBM 层的数据加载由推理框架 prefix cache 命中时自主触发。
+
+```
+flowchart TD
+ START["prefix cache 命中/预取触发/get"]
+ CALC["Tier Up Priority 计算:
priority = w0 * recompute_cost
+ w1 * continuous_prefix
+ w2 * request_priority
- w3 * transfer_eta"]
+ SORT["按优先级排序"]
+ EXEC["执行逐级 tier up
L3→L2→L1"]
+
+ START --> CALC --> SORT --> EXEC
+```
+
+### 5.4.3.6 预取 (Prefetch)
+
+**现有机制**:当前无显式预取机制,Promotion-on-Hit 在 Get 命中 LOCAL_DISK 时异步提升到 MEMORY。
+
+**IO Pattern 增强**:新增 `PrefetchOps` 抽象,SubMaster 根据 trace 和置信阈值生成预取器:
+
+**预取触发条件:** 低速层 prefix match length > 阈值 (256) 时触发预取至上一层(如 L3→L2、L2→L1)
+
+**预取策略(三种模式):**
+
+| 策略 | 描述 | 适用场景 |
+| --------------- | ----------------------------------- | -------------------- |
+| `best_effort` | check & match,无论是否完成立即返回 | 对 TTFT 时延敏感业务 |
+| `timeout` | 预取完成或超时立即返回 | 兼顾时延和命中率 |
+| `wait_complete` | 死等数据加载完成 | 追求极致命中率 |
+
+分析层基于 trace 历史命中率和置信阈值生成策略输入。置信度 = 滑动窗口内命中次数 / 总访问次数,低于阈值时不触发操作避免误判。例如预取器生成:
+
+- SubMaster 根据 trace 历史 + 置信阈值(如 `confidence > 0.6 && prefix match length > 256`)生成预取器
+- 置信度低于阈值时不触发操作,避免误判导致的缓存污染
+- 置信阈值精确定义和各策略默认值详见
+
+**置信度计算:** 基于滑动窗口内的历史命中率,衡量当前预测的可信程度。
+
+```
+confidence = hit_count_in_window / total_access_in_window
+```
+
+**置信阈值应用:**
+
+| 策略 | 置信阈值 | 含义 | 默认值 |
+| -------- | ----------------------------------------------- | --------------------------------------- | ------------------------------------ |
+| 预取触发 | `confidence > 0.6 && match_length > 256` | 历史命中率 > 60% 且前缀匹配足够长才预取 | prefix_threshold=256, confidence=0.6 |
+| 准入提升 | `confidence > 0.5 && access_count >= threshold` | 历史命中率 > 50% 且频率达标才提升 | confidence=0.5 |
+| 淘汰保守 | `confidence > 0.8` 时降低淘汰权重 | 高置信热数据更保守淘汰 | confidence=0.8 |
+
+>
+> 置信度低于阈值时不触发操作,避免误判导致的缓存污染。置信度窗口默认 60s,可通过 `confidence_window_sec` 配置。
+
+预取流程仅看 `match_length > 256` 触发
+
+```
+flowchart TD
+ START["低速层 prefix match
(L1-L3)"]
+ C1{"match_length > 256 ?"}
+ NOP["不预取"]
+ SEL["选择预取策略"]
+
+ START --> C1
+ C1 -->|否| NOP
+ C1 -->|是| SEL
+ SEL --> BE["best_effort"]
+ SEL --> TO["timeout"]
+ SEL --> WC["wait_complete"]
+```
+
+- `max_prefetch_ratio`:预取占用带宽上限比例,默认 20%,可通过 `prefetch_max_bw_ratio` 配置
+- 带宽不足时延迟重试,而非直接丢弃预取请求
+
+**集成方式**:在 SubMaster 中新增预取器,根据 IO Pattern 分析层的置信阈值异步预取 key 至上层。
+
+## 5.4.5 上层推理框架对接
+
+### 5.4.5.1 vLLM 集成
+
+**现有对接**:`MooncakeConnector`(`mooncake_connector_v1.py`)实现 vLLM `KVConnectorBase_V1` 接口,支持 PD disaggregation(Prefill/Decode 分离)。
+
+>
+> **上层框架改动**:vLLM 侧无需改动。`MooncakeConnector` 作为 vLLM 的 out-of-tree connector(通过 `--kv_connector_module_path` 加载),在 connector 内部新增 CFM Client 调用即可上报指标和接收策略指令,不涉及 vLLM scheduler/engine 接口变更。vLLM v0.13.0+ 已内置 mooncake connector,后续可考虑将 CFM Client 合入上游。
+
+**IO Pattern 对接增强**:
+
+```
+flowchart TD
+ VLLM["vLLM Engine"]
+
+ subgraph MC ["MooncakeConnector (KVConnectorBase_V1)"]
+ GNMT["get_num_new_matched_tokens()
上报 match_length"]
+ USA["update_state_after_alloc()
上报 prefix_depth"]
+ RF["request_finished()
上报 token_count, recompute_cost"]
+ end
+
+ subgraph CFMC ["CFM Client (新增)"]
+ RIM["IoPatternCollector
.ReportInferenceMetrics()"]
+ POP["PrefetchOps
接收预取指令"]
+ AOP["AdmissionOps
接收准入策略"]
+ end
+
+ VLLM --> MC
+ VLLM --> CFMC
+ GNMT --> RIM
+ USA --> RIM
+ RF --> RIM
+```
+
+**采集对接**:在 `MooncakeConnector` 中增加 CFM Client 调用,将以下指标上报至 IO Pattern Collector:
+
+- `match_length`:prefix cache 命中长度(来自 `get_num_new_matched_tokens()`)
+- `prefix_depth` / `prefix_fanout`:前缀树结构(来自 `update_state_after_alloc()` 及 prefix cache manager)
+- `token_count`:请求 token 数(来自 `request_finished()`)
+- `recompute_cost`:重计算代价估算(connector 侧基于 token_count 和模型 FLOPS 估算)
+- `request_priority`:请求优先级(connector 层从 request metadata 提取)
+
+**策略对接**:CFM Client 接收 Policy Engine 的策略指令:
+
+- 预取指令:根据 prefix match length > 256 触发异步预取
+- 准入指令:根据频率/前缀匹配控制数据提升层级
+- 淘汰指令:根据淘汰评分驱动 L0-L3 层间淘汰
+
+### 5.4.5.2 SGLang 集成
+
+**现有对接**:SGLang HiCache 通过 `--hicache-storage-backend: mooncake` 将 Mooncake 作为存储后端,支持 layer_first / page_first 布局。
+
+**IO Pattern 对接增强**:
+
+```
+flowchart TD
+ SGL["SGLang Engine"]
+
+ subgraph HCC ["HiCache Connector"]
+ HR["hicache-ratio
容量配比"]
+ HML["hicache-mem-layout
layer_first / page_first"]
+ HIO["hicache-io-backend
direct / async"]
+ end
+
+ subgraph CFMS ["CFM Client (新增)"]
+ RIM2["IoPatternCollector
.ReportInferenceMetrics()"]
+ DLA["数据布局适配
(layer_first / page_first)"]
+ PAS["预取/准入/淘汰策略"]
+ end
+
+ SGL --> HCC
+ SGL --> CFMS
+ HR --> RIM2
+ HML --> DLA
+ HIO --> RIM2
+```
+
+**数据布局适配**:IO Pattern 需感知推理框架的 KV cache 布局模式,vLLM 和 SGLang 均需适配:
+
+| 框架 | 布局模式 | 描述 | IO Pattern 适配 |
+| ------ | ------------------------------ | ------------------------------------------------------------------------- | ----------------------------------------------- |
+| SGLang | `layer_first` | (2, layer, slot, num_head, head_dim) | 前缀分析按 layer 维度,预取按 layer 批量 |
+| SGLang | `page_first` | (2, page_num, layer, num_head, head_dim) | 前缀分析按 page 维度,预取按 page 批量 |
+| SGLang | `page_first_direct` | 混合模型 (Full Attention + SWA/Mamba) | 分区准入,full KV 固定分区 + SWA/mamba 灵活分配 |
+| vLLM | `page-based` (block_size 粒度) | vLLM v1 默认 page-based 布局,connector 通过 `get_kv_cache_layout()` 检测 | 前缀分析按 block 维度,预取按 block 批量 |
+| vLLM | `HMA multi-group` | 混合模型 (attention + Mamba2),`SupportsHMA` 多 group 布局 | 分组准入,各 group 独立淘汰/预取策略 |
+
+>
+> vLLM connector 已在初始化时调用 `get_kv_cache_layout()` 检测布局(`mooncake_connector_v1.py:511`),并通过 `SupportsHMA` 支持 hybrid 模型多 group 布局。SGLang 通过 `--hicache-mem-layout` 参数显式配置布局。两者均需在 CFM Client 上报时附带布局信息,供 IO Pattern 分析层选择对应的预取/准入粒度。
+
+### 5.4.5.3 CFM Client 设计
+
+CFM Client 部署在推理节点侧,作为推理框架与 SuperCache 之间的策略桥梁:
+
+```
+flowchart TD
+ subgraph CFMClient ["CFM Client"]
+ MR["Metrics Reporter
(采集上报)"]
+ PR["Policy Receiver
(策略接收)"]
+ PE["Prefetch Executor
(预取执行)"]
+ RPC["CFM RPC Channel
(to SubMaster / PrefixCache Master)"]
+ MR --> RPC
+ PR --> RPC
+ PE --> RPC
+ end
+```
+
+**职责:**
+
+1. **Metrics Reporter**:定期(100ms)批量上报推理框架指标至 SubMaster
+2. **Policy Receiver**:接收 Policy Engine 的淘汰/预取/准入策略指令
+3. **Prefetch Executor**:执行异步预取,支持 best_effort / timeout / wait_complete 三种模式
+
+## 5.4.6 Ops 抽象接口设计
+
+> **接口修订(2026-09)**:本节原始的 string/vector 简化签名仅用于
+> 查询示例,不能承载租户、字节预算、评分、Tier、超时和置信度等执行
+> 元数据。实际实现统一采用文末“Revised Ops contract”中的完整计划接口。
+
+### 5.4.6.1 EvictionOps
+
+```
+// include/eviction_ops.h (扩展现有 eviction_strategy.h)
+class EvictionOps {
+ public:
+ virtual ~EvictionOps() = default;
+
+ // 基于 IO Pattern 评分选择淘汰 key
+ virtual std::vector SelectEvictionCandidates(
+ const IoPatternSnapshot& snapshot,
+ CacheTier tier,
+ size_t target_bytes) = 0;
+
+ // 注册淘汰算法
+ static void Register(const std::string& name,
+ std::function()> factory);
+};
+
+// 已有实现: LRU, FIFO (eviction_strategy.h)
+// 新增实现: ScoreBasedEviction (L0-L3 四层不同评分公式)
+```
+
+### 5.4.6.2 PrefetchOps
+
+```
+// include/prefetch_ops.h (新增)
+class PrefetchOps {
+ public:
+ virtual ~PrefetchOps() = default;
+
+ // 基于 trace 和置信阈值生成预取候选
+ virtual std::vector GeneratePrefetchPlan(
+ const IoPatternSnapshot& snapshot,
+ const TraceHistory& trace) = 0;
+
+ // 执行预取
+ virtual ErrorCode ExecutePrefetch(
+ const std::vector& candidates,
+ PrefetchStrategy strategy) = 0;
+
+ static void Register(const std::string& name,
+ std::function()> factory);
+};
+
+enum class PrefetchStrategy {
+ kBestEffort, // 无论是否完成立即返回
+ kTimeout, // 预取完成或超时立即返回
+ kWaitComplete, // 死等数据加载完成
+};
+```
+
+### 5.4.6.3 AdmissionOps
+
+```
+// include/admission_ops.h (新增, 扩展现有 CountMinSketch 准入)
+class AdmissionOps {
+ public:
+ virtual ~AdmissionOps() = default;
+
+ // 准入决策:是否允许数据进入目标层
+ virtual AdmissionDecision CheckAdmission(
+ const std::string& key,
+ CacheTier target_tier,
+ const IoPatternSnapshot& snapshot) = 0;
+
+ static void Register(const std::string& name,
+ std::function()> factory);
+};
+
+enum class AdmissionDecision {
+ kAdmit, // 允许进入
+ kRejectFrequency, // 频率不足
+ kRejectWatermark, // 水位过高
+ kRejectPrefix, // 前缀匹配不足
+ kDefer, // 延迟决策 (记录候选)
+};
+
+// 已有实现: FrequencyAdmission (CountMinSketch)
+// 新增实现: PrefixMatchAdmission, CostAwareAdmission
+```
+
+### 5.4.6.4 Ops 注册机制
+
+```
+classDiagram
+ class EvictionOps {
+ <>
+ +SelectEvictionCandidates(snapshot, tier, bytes) vector~string~
+ +Register(name, factory) void
+ }
+ class PrefetchOps {
+ <>
+ +GeneratePrefetchPlan(snapshot, trace) vector~PrefetchCandidate~
+ +ExecutePrefetch(candidates, strategy) ErrorCode
+ +Register(name, factory) void
+ }
+ class AdmissionOps {
+ <>
+ +CheckAdmission(key, tier, snapshot) AdmissionDecision
+ +Register(name, factory) void
+ }
+ class PolicyEngine {
+ -ops_registry_ : map
+ +SelectOps(type, name) Ops
+ +ExecutePolicy(snapshot) PolicyResult
+ }
+
+ PolicyEngine --> EvictionOps : 查找/执行
+ PolicyEngine --> PrefetchOps : 查找/执行
+ PolicyEngine --> AdmissionOps : 查找/执行
+```
+
+## 5.4.7 写路径 IO Pattern
+
+### 5.4.7.1 写路径采集指标
+
+| 指标名 | 类型 | 描述 | 采集来源 |
+| ------------------- | ------ | ------------------------------ | ------------------- |
+| `write_batch_size` | uint32 | 批量写入 key 数 | SDK 层 (`BatchPut`) |
+| `write_object_size` | uint64 | 单次写入数据大小 | SDK 层 |
+| `write_burst` | bool | 是否突发写入(短时间大量 Put) | 分析层计算 |
+| `write_frequency` | uint32 | key 写入频率 | SDK 层 |
+| `overwrite_ratio` | float | 覆盖写比例 (同 key 重复 Put) | 分析层计算 |
+
+### 5.4.7.2 写路径 Pattern 对策略的影响
+
+| Pattern | 影响策略 | 处理方式 |
+| ---------- | ---------------------- | ------------------------------------------------------------------ |
+| 突发写入 | 准入:避免写穿 SSD | 突发写入期间提高 `admission_threshold`,冷数据暂留 DRAM 不 offload |
+| 高覆盖写 | 准入:跳过 SSD offload | 覆盖写比例高的 key 不 offload 到 SSD,避免无效写入 |
+| 大批量写入 | GC:提前触发 | 预估写入量,提前通知 SSD 后端准备 GC 空间 |
+| 低频写入 | 淘汰:降低保留优先级 | 低频写入的 key 在淘汰评分中 `frequency` 低,优先淘汰 |
+
+## 5.4.8 健壮性与可观测性
+
+### 5.4.8.1 失败降级
+
+IO Pattern 模块自身故障时,必须不影响数据路径,降级到现有基础机制:
+
+```
+flowchart TD
+ START["策略执行请求"]
+ C1{"IO Pattern 模块可用?"}
+ C2{"分析层响应
超时?"}
+ NORMAL["正常路径:
ScoreBasedEviction / PrefixMatchAdmission / TraceBasedPrefetch"]
+ DEGRADE["降级路径:
LRU / FIFO / FrequencyAdmission
(现有基础机制)"]
+
+ START --> C1
+ C1 -->|是| C2
+ C1 -->|否| DEGRADE
+ C2 -->|否| NORMAL
+ C2 -->|是| DEGRADE
+```
+
+| 故障场景 | 降级行为 | 触发条件 |
+| -------------- | --------------------------------- | ------------------- |
+| SubMaster 崩溃 | 回退到 Client 本地 LRU/FIFO | RPC 连续失败 > 3 次 |
+| 分析层超时 | 使用上一次成功快照 | 响应延迟 > 500ms |
+| 分析层 OOM | 丢弃 per-key 指标,仅保留全局指标 | 内存占用 > 阈值 |
+| RPC 网络抖动 | 延长上报间隔,本地缓存策略 | 丢包率 > 5% |
+
+### 5.4.8.2 反馈闭环
+
+策略执行后需评估效果并自适应调优参数,形成闭环:
+
+```
+flowchart LR
+ EXEC["策略执行
(eviction/prefetch/admission)"]
+ EVAL["效果评估
(命中率/eviction抖动/TTFT)"]
+ TUNE["参数调优
(权重/阈值自适应)"]
+ EXEC --> EVAL --> TUNE --> EXEC
+```
+
+**效果评估指标:**
+
+| 指标 | 描述 | 评估窗口 |
+| ------------------- | ------------------------------ | ------------- |
+| `hit_rate_delta` | 策略执行后命中率变化 | 60s 滑动窗口 |
+| `eviction_churn` | 淘汰抖动(刚淘汰又被访问) | 120s 滑动窗口 |
+| `ttft_delta` | TTFT 时延变化 | 30s 滑动窗口 |
+| `prefetch_accuracy` | 预取命中率(预取后是否被访问) | 60s 滑动窗口 |
+
+**参数自适应:** 当 `hit_rate_delta < 0` 持续超过 3 个评估窗口时,自动回退权重调整(如降低 `α` 权重),或切换到更保守的策略(如 ScoreBased → LRU)。
+
+### 5.4.8.3 IO Pattern 自观测
+
+IO Pattern 模块自身的运行指标,用于运维和调优:
+
+| 指标 | 描述 |
+| -------------------------------- | ----------------------------------------- |
+| `io_pattern_collect_latency_us` | 单次采集延迟 |
+| `io_pattern_analyze_latency_us` | 单次分析延迟 |
+| `io_pattern_policy_decision_qps` | 策略决策 QPS |
+| `io_pattern_strategy_hit_rate` | 策略命中率(策略命中 vs 总决策) |
+| `io_pattern_false_positive_rate` | 误判率(预取未被访问 / 淘汰后被重新加载) |
+| `io_pattern_degrade_count` | 降级次数 |
+| `io_pattern_report_drop_count` | 上报丢弃数(采样降级) |
+
+
+# IO Pattern implementation design
+
+This page records the implementation state of the IO Pattern design and is
+updated together with the code. It is intentionally separate from the original
+proposal so that unresolved decisions are visible.
+
+## Current architecture
+
+```text
+Store/Get/Put -> Collector -> bounded Analyzer -> PolicyEngine -> Ops
+ | | |-> Eviction handler
+ | | |-> Prefetch handler
+ | | `-> Admission handler
+ | `-> per-session K-means fallback
+ `-> CfmIngress <- report_metric_batch/report_snapshot (coro_rpc)
+ (merge into the SubMaster's own runtime)
+```
+
+`MasterService` owns the runtime because it owns the authoritative replica
+metadata. Its handlers use the existing safe quota-eviction and
+promotion-on-hit queues; HBM stays inference-runtime-owned and is never moved
+by the Store master.
+
+### Current implementation architecture
+
+CFM is an embedded component of every SubMaster: the SubMaster's own
+`IoPatternRuntime` collects observations from its Store data path, evaluates
+policy and executes through the same storage-safe handlers. There is no
+standalone CFM Master deployment, no separate CFM endpoint and no auth token.
+A reporting client (inference connector / Store client) observes keys that may
+live on many SubMasters, resolves the owning SubMaster for every key through
+the CVM key->slot->submaster mapping, aggregates observations per owner and
+sends metric batches to each owning SubMaster's regular `coro_rpc` endpoint.
+The receiving SubMaster merges the report into its local runtime so
+collection, analysis and execution all stay on the SubMaster that owns the
+reported keys.
+
+```mermaid
+flowchart TB
+ subgraph producers["Metric producers"]
+ direction LR
+ inference["vLLM / SGLang bridge\nInferenceMetrics"]
+ access["Store Get/Put paths\nAccessRecord"]
+ storage["Storage and watermark paths\nStorageMetric"]
+ end
+
+ subgraph local["SubMaster MasterService (embedded CFM)"]
+ direction TB
+ runtime["IoPatternRuntime"]
+ collector["IoPatternCollectorImpl\nper-tenant/object aggregation\nrolling snapshot"]
+ ingress["CfmIngress\nmerge ownership-addressed reports"]
+ reporter["IoPatternReporter\nbounded MetricBatch queue\nadaptive 100/200/500/1000 ms flush"]
+ analyzer["ResilientAnalyzer\nSlidingWindowAnalyzer\nbudget + timeout fallback"]
+ policy["DegradingPolicyEngine\nWorkloadPolicyEngine\nper-session templates"]
+ executor["TierOperationExecutor"]
+ feedback["PolicyFeedbackWindow +\nAdaptivePolicyTuner"]
+ admission_worker["Admission worker\nbounded deferred queue"]
+
+ ingress --> runtime
+ runtime --> collector
+ collector --> reporter
+ collector --> analyzer
+ analyzer --> policy
+ policy --> executor
+ executor --> feedback
+ feedback -. "tune eviction weights" .-> policy
+ policy --> admission_worker
+ end
+
+ subgraph local_ops["Store-owned safe execution handlers"]
+ direction LR
+ evict["Eviction\ntenant-qualified quota eviction"]
+ prefetch["Prefetch\nLOCAL_DISK → MEMORY promotion queue"]
+ admit["Admission\npost-write retention / promotion"]
+ end
+
+ subgraph client["Reporting client (connector / Store client)"]
+ direction LR
+ owner["CvmOwnershipClient\nCVM key→slot→submaster bucketing"]
+ codec["CfmBinaryCodec\nversioned CFM2 wire format"]
+ channel["CfmRpcChannel\nencode/decode"]
+ rpc["CoroRpcCfmTransport\ncoro_rpc to the owning SubMaster"]
+ owner --> codec --> channel --> rpc
+ end
+
+ rpc_service["CfmRpcService::Send\non the SubMaster's coro_rpc port"]
+
+ inference -->|"ownership-addressed metric batches"| owner
+ access --> runtime
+ storage --> runtime
+ executor --> evict
+ executor --> prefetch
+ executor --> admit
+
+ reporter -->|"report_metric_batch"| ingress
+ rpc -->|"report_metric_batch / report_snapshot / execute_*"| rpc_service
+ rpc_service --> ingress
+ observability["IoPatternObservability\nlatency, hit rate, false positives,\ndegradation, report drops"] -.-> runtime
+```
+
+`CfmOwnershipClient` (or a single-endpoint `CfmRpcChannel` when one SubMaster
+is the only owner) is the normal report path. `CfmClientImpl` wraps a single
+channel for connectors; the client owns aggregation per owning SubMaster and
+explicitly drops observations whose owner cannot be resolved. The receiver
+merges every accepted report into the local runtime — there is no policy
+queue, poll, ACK or producer role to configure.
+
+Every accepted report also drives the local policy pipeline. The receiving
+SubMaster runs a background, coalescing `IoPatternRuntime` cycle after each
+`report_metric_batch` / `report_snapshot` merge. The cycle derives its
+decision inputs from the freshly aggregated snapshot: when the merged L1
+host-memory storage watermark (peak `memory_used_ratio` for `kL1Host` entries)
+exceeds the configured high ratio the cycle requests an eviction plan for host
+memory (target bytes = excess over the target ratio × reported capacity, only
+when the snapshot also contains L1 keys); keys that were recently served as
+hits and still carry L2/L3 replicas are fed to the prefetch ops, and hot
+lower-tier keys (not pinned, no L1 replica) are offered to the admission ops.
+When no storage watermark is exceeded but the runtime is configured with the
+cold-data eviction driver enabled (`report_driven_cold_eviction`), the cycle
+instead requests a bounded eviction of the coldest (idle) L1 keys so eviction
+is driven by cold/hot analysis and not only by memory pressure; the cycle
+report marks such passes with `cold_eviction=true`.
+Each cycle executes through the same storage-safe handlers, logs one
+`[IO-PATTERN-REPORT-CYCLE]` summary line (including whether the pass was a
+cold-eviction driver pass) and reports per-dimension outcomes to
+`MasterMetricManager` (`io_pattern_report_*` counters, visible on the master
+`/metrics` endpoint and in the periodic "Master Admin Metrics" log). This is
+what makes remote-mode policy execution observable: policy is no longer only
+run by the local memory-watermark thread, the local Put admission path or
+explicit `execute_*` RPCs.
+
+## Implemented
+
+- `IoPatternCollectorImpl` aggregates inference, access and storage metrics by
+ tenant/object and returns deterministic snapshots.
+- `ThresholdAnalyzer` classifies Code Agent, recommendation, conversation and
+ mixed workloads and calculates continuous confidence scores.
+- `ScoreBasedEvictionOps`, `PrefixMatchAdmissionOps` and
+ `TraceBasedPrefetchOps` provide the first production policy implementations.
+- `WorkloadPolicyEngine` selects workload templates, applies real weighted
+ transition over three detection windows, and selects independent templates
+ for K-means-labelled sessions.
+- `OpsRegistry` and `RegistryPolicyEngine` resolve named policy implementations.
+- `PolicyEngine::ExecutePolicy` returns one `PolicyResult` containing eviction,
+ prefetch and admission outcomes.
+- `IoPatternReporter` provides bounded, non-blocking batches with explicit
+ report/drop counters and a transport-agnostic sink.
+- `MetricBatchTransport` defines the transport seam, and the reporter exposes
+ load-sensitive 100/200/500/1000 ms flush recommendations.
+- `IoPatternRuntime` wires collection, bounded analysis, policy execution,
+ feedback tuning and storage handlers; `MasterService` feeds it from actual
+ Get/Put/watermark paths.
+- A coalescing report-driven execution worker (`report_driven_execution`
+ runtime config, enabled by `MasterService`) runs one full
+ Collector -> Analyzer -> PolicyEngine -> execution cycle after every merged
+ `report_metric_batch` / `report_snapshot`. The cycle derives an eviction
+ request only when the merged L1 host-memory storage watermark is above the
+ configured high ratio and the snapshot contains L1 keys, rebuilds a
+ prefetch trace from recently hit lower-tier keys and offers hot non-head
+ keys to the admission ops; an empty-candidate eviction is a clean no-op, not
+ a failure. Every cycle logs one `[IO-PATTERN-REPORT-CYCLE]` line and reports
+ its per-dimension outcome (eviction/prefetch/admission candidates, handler
+ statuses, degradation, cold-driver flag) to the process observer
+ (`MasterMetricManager` `io_pattern_report_*` counters on the master, visible
+ in `/metrics` and the "Master Admin Metrics" log). This keeps remote-mode
+ policy execution observable and data-driven instead of relying only on the
+ local eviction thread, the Put admission hook or explicit `execute_*` RPCs.
+- The cold-data eviction driver (`report_driven_cold_eviction` runtime config
+ plus `report_driven_cold_eviction_bytes` per-cycle budget and optional
+ `report_driven_cold_idle_threshold_us` gate) runs on the same report-driven
+ worker: when a cycle finds no storage-watermark pressure it still plans a
+ bounded eviction of the coldest non-pinned L1 keys, marks the pass
+ `cold_eviction=true` and executes through the same storage handlers, so
+ eviction is not only triggered at the memory high watermark. `MasterService`
+ surfaces it through `--io_pattern_cold_eviction`,
+ `--io_pattern_cold_eviction_bytes_per_cycle` and
+ `--io_pattern_cold_idle_threshold_us`.
+- `cfm_client_bench` optionally seeds real KV objects (`--master_server`,
+ `--num_keys`, `--value_size`, `--replica_num`, `--protocol` and the other
+ RealClient flags shared with `stress_cluster_bench`) before the simulated
+ vLLM request stream and reads a subset back, so the IO Pattern handlers run
+ against replicas that actually exist on the SubMaster (report-only runs leave
+ eviction/promotion/prefetch counters at zero because the handlers cannot act
+ on objects the master never stored).
+- `CfmClientImpl` wraps a single reporting channel for connectors
+ (`ReportSnapshot` / `ReportMetricBatch` / `ExecutePrefetch`); policy runs in
+ the SubMaster's own runtime, so there is no client-side dispatch loop.
+- `CfmOwnershipClient` is the multi-SubMaster report path: it resolves the
+ owning SubMaster of every key through a CVM-backed resolver, aggregates
+ observations per owner and delivers one batch per owner; unresolvable
+ observations are counted as drops.
+- `CfmIngress` is the CFM-to-Store endpoint on each SubMaster: it decodes
+ snapshot and metric-batch payloads into that SubMaster's local runtime and
+ executes prefetch/eviction plans through the same storage-safe handlers.
+- `ResilientCfmChannel` adds bounded retries and consecutive-failure
+ degradation state around a concrete reporting transport.
+- `PolicyFeedbackWindow` aggregates bounded execution-effect windows, and
+ `AdaptivePolicyTuner` adjusts eviction weights after repeated negative
+ hit-rate deltas.
+- `IoPatternObservability` provides thread-safe counters for collection and
+ analysis latency, policy hit rate, false positives, degradation and report
+ drops.
+- Its windowed snapshot also exposes strategy hit rate, false-positive rate and
+ policy decision QPS.
+- `SlidingWindowAnalyzer` keeps timestamp-bounded snapshots and computes
+ median/p90 workload features before threshold classification.
+- `IoPatternCollectorImpl` enforces an optional per-tenant key quota and
+ exposes dropped-observation counts for overload protection.
+- The vLLM connector accumulates match, allocation and completion metrics per
+ request and reports a complete layout-aware record through its optional
+ `io_pattern_bridge`. `SglangHiCacheIoPatternBridge` provides the matching
+ bounded, layout-aware adapter for HiCache request-finished/prefix hooks.
+- `TierOperationExecutor` bridges `PolicyResult` to storage-owned eviction,
+ prefetch and admission handlers and marks missing handlers as degraded.
+- `ResilientAnalyzer` caches the last successful result and falls back to it
+ (or conservative mixed mode) when analysis throws, with failure tracking.
+- `CfmBinaryCodec` defines the versioned `CFM2` protocol and fully round-trips
+ snapshots, metric batches and every policy command. `InProcessCfmRpcTransport`
+ provides embedded operation for tests, while `CfmChannelPool` reuses and
+ fails over a bounded set of reporting channels.
+- `CfmRpcChannel::SendMetricBatch` and `MakeCfmMetricBatchSink` connect the
+ bounded Reporter to the RPC path; producers only enqueue and Flush performs
+ the transport call outside the data-path critical section.
+- `IoPatternReporter::Start/Stop` provides a background flush worker with
+ adaptive intervals; `Stop` performs a final synchronous drain.
+- `IoPatternCollectorImpl` derives write-path fields for PUT records:
+ frequency, batch size, object size, overwrite ratio and burst flag.
+- `DegradingPolicyEngine` switches to a caller-provided fallback engine after
+ repeated failures and supports explicit recovery.
+- `AdaptivePolicyTuner` also reacts to eviction churn, TTFT regression and
+ prefetch accuracy, exposes conservative mode and supports persistence
+ callbacks for tuned weights. The runtime accepts feedback samples and
+ applies the resulting weights to both global and per-session engines.
+- Analyzer execution has a single in-flight worker, timeout fallback to the
+ last safe result, and an explicit key-count budget; collector key quotas and
+ reporter bounds provide the associated overload/OOM protection.
+- Access and write frequencies use timestamped buckets pruned against a true
+ rolling 60-second cutoff. Sliding analysis deduplicates objects across
+ snapshots and enforces a hard total retained-key budget (including a single
+ oversized snapshot), so repeated high-watermark evaluations cannot multiply
+ complete snapshots without bound. CFM ingress rebases process-local monotonic
+ timestamps to receiver time, and each object also has a hard bucket-count cap.
+- Store-side eviction executes only the tenant-qualified objects selected by
+ the policy. The legacy `BatchEvict` path runs only when policy execution
+ fails, avoiding a second unplanned eviction pass.
+- Non-memory PUT completions enqueue bounded, asynchronous L1 retention/
+ promotion evaluation. This is a post-write cache-admission hook, not initial
+ replica placement: the existing `PutStart` contract selects and allocates
+ replicas before write metrics such as batch and overwrite are known.
+- Reporter intervals follow the documented memory/RPC load thresholds
+ (100/200/500/1000 ms), and in-process transport callbacks execute outside the
+ transport mutex.
+
+## Interface decision: complete plans versus document shorthand
+
+The proposal's shorthand methods returned only keys, candidates or a decision.
+The implementation also needs tenant identity, byte sizes, scores, confidence,
+timeout and strategy metadata. Therefore the complete plan interfaces are the
+only Ops execution seam:
+
+- `EvictionOps::Evaluate` returns `EvictionPlan`.
+- `PrefetchOps::Evaluate` returns `PrefetchPlan`.
+- `AdmissionOps::Evaluate` returns `AdmissionResult`.
+
+The former shorthand methods (`SelectEvictionCandidates`,
+`GeneratePrefetchPlan`, `ExecutePrefetch`, and `CheckAdmission`) have been
+removed from the C++ interfaces. Callers must use complete plans and
+`TierOperationExecutor` for execution.
+
+### Revised Ops contract (2026-09)
+
+The following contract supersedes the shorthand signatures in section 5.4.6:
+
+```cpp
+class EvictionOps {
+ public:
+ virtual EvictionPlan Evaluate(const PolicyContext&, CacheTier,
+ uint64_t target_bytes) const = 0;
+};
+
+class PrefetchOps {
+ public:
+ virtual PrefetchPlan Evaluate(const PolicyContext&,
+ const TraceHistory&) const = 0;
+};
+
+class AdmissionOps {
+ public:
+ virtual AdmissionResult Evaluate(const ObjectRef&, CacheTier,
+ const PolicyContext&) const = 0;
+};
+```
+
+`EvictionPlan` carries tenant-qualified objects, byte budgets and scores;
+`PrefetchPlan` carries source/target tiers, strategy, timeout and confidence;
+`AdmissionResult` carries tenant identity, target tier, decision and
+confidence. These fields are required by execution, observability and
+multi-tenant isolation and must not be collapsed into strings.
+
+`PolicyEngine::ExecutePolicy` is the single orchestration entry point and
+returns `PolicyResult` with explicit `degraded` propagation.
+
+Registry ownership is external and thread-safe. Factories return independent
+Ops instances; callers own the returned smart pointers. Concrete storage and
+RPC resources are injected through execution handlers and CFM channels.
+
+## Production CFM wiring
+
+Every Master registers the CFM report handler (`CfmRpcService::Send`) on its
+existing `coro_rpc` port — the same endpoint all other Mooncake RPCs use. No
+extra `io_pattern_cfm_endpoint`, `io_pattern_cfm_auth_token`,
+`io_pattern_cfm_node_id` or producer credential is configured: Mooncake RPCs
+run inside the trusted deployment, and CFM is a component of the SubMaster
+that owns the reported keys.
+
+A SubMaster's embedded CFM works as follows:
+
+- The local Store data path records `AccessRecord` / `StorageMetric` straight
+ into the SubMaster's own `IoPatternRuntime`; eviction runs through the same
+ storage-safe quota/promotion handlers (no per-node runtime, policy queue or
+ poll loop).
+- A reporting client (vLLM/SGLang connector or the Store client) observes keys
+ that may belong to different SubMasters. It resolves the owner of each key
+ with the CVM mapping (`cvm::KeySlot` over the `/cvm/` master registry),
+ aggregates observations by owning SubMaster, and delivers one
+ `report_metric_batch` / `report_snapshot` per owner over coro_rpc.
+- The receiving SubMaster's `CfmIngress` merges the batch into its local
+ runtime and normalizes remote StorageMetric watermarks against the
+ transport source, so collection, analysis and execution never leave the
+ SubMaster that owns the keys.
+
+Reports that fail to resolve to an owner are counted as dropped observations
+on the client, so callers degrade explicitly instead of guessing an owner.
+Explicit `execute_prefetch` / `execute_policy` RPCs are also accepted by the
+same handler and run through the receiving SubMaster's local storage
+handlers, preserving the storage-safe execution seam for cross-node commands.
+
+The SGLang adapter remains framework-neutral because SGLang source is not
+vendored in this repository.
diff --git a/docs/yh/deployment.md b/docs/yh/deployment.md
new file mode 100644
index 0000000000..df0a0879a2
--- /dev/null
+++ b/docs/yh/deployment.md
@@ -0,0 +1,777 @@
+
+
+
+
+# POC项目Mooncake部署文档
+
+## 1 ETCD环境搭建
+### 1.1 安装
+
+```bash
+wget --no-check-certificate https://github.com/etcd-io/etcd/releases/download/v3.5.9/etcd-v3.5.9-linux-arm64.tar.gz
+tar -xzvf etcd-v3.5.9-linux-arm64.tar.gz
+rm etcd-v3.5.9-linux-arm64.tar.gz
+mv etcd-v3.5.9-linux-arm64/ etcd
+cd etcd/
+cp etcd etcdctl /usr/local/bin/
+```
+
+### 1.2 配置
+
+#### 1.2.1 方法1,一同创建(推荐)
+
+##### 1.2.1.1 设置system脚本
+
+```bash
+vim /etc/systemd/system/etcd.service
+```
+
+`ETCD_NAME, ETCD_INITIAL_ADVERTISE_PEER_URLS, ETCD_ADVERTISE_CLIENT_URLS`根据本机实际配置
+`ETCD_INITIAL_CLUSTER`配置集群全部节点的地址
+
+```
+[Unit]
+Description=etcd key-value store
+After=network.target
+
+[Service]
+Type=notify
+Environment=ETCD_NAME=
+Environment=ETCD_DATA_DIR=/var/lib/etcd
+Environment=ETCD_LISTEN_PEER_URLS=http://0.0.0.0:2380
+Environment=ETCD_LISTEN_CLIENT_URLS=http://0.0.0.0:2379
+Environment=ETCD_INITIAL_ADVERTISE_PEER_URLS=http://:2380
+Environment=ETCD_ADVERTISE_CLIENT_URLS=http://:2379
+Environment=ETCD_INITIAL_CLUSTER="=http://:2380,=http://:2380"
+Environment=ETCD_INITIAL_CLUSTER_STATE="new"
+Environment=ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
+ExecStart=/usr/local/bin/etcd
+Restart=on-failure
+RestartSec=5
+User=root
+
+[Install]
+WantedBy=multi-user.target
+```
+
+##### 1.2.1.2 清除缓存,应用配置,启动etcd服务
+
+为所有节点配置完成后,启动全部服务
+
+```bash
+systemctl disable etcd
+systemctl stop etcd
+rm -rf /var/lib/etcd
+systemctl daemon-reload
+systemctl start etcd
+systemctl enable etcd
+```
+
+##### 1.2.1.3 查看集群
+
+```bash
+etcdctl --endpoints=http://:2379 member list --write-out=table
+```
+
+#### 1.2.2 方法2,逐个创建
+
+##### 1.2.2.1 设置system脚本
+
+```bash
+vim /etc/systemd/system/etcd.service
+```
+
+主机 (ip换为本机ip)
+
+```
+[Unit]
+Description=etcd key-value store
+After=network.target
+
+[Service]
+Type=notify
+Environment=ETCD_NAME=
+Environment=ETCD_DATA_DIR=/var/lib/etcd
+Environment=ETCD_LISTEN_PEER_URLS=http://0.0.0.0:2380
+Environment=ETCD_LISTEN_CLIENT_URLS=http://0.0.0.0:2379
+Environment=ETCD_INITIAL_ADVERTISE_PEER_URLS=http://:2380
+Environment=ETCD_ADVERTISE_CLIENT_URLS=http://:2379
+Environment=ETCD_INITIAL_CLUSTER="=http://:2380"
+Environment=ETCD_INITIAL_CLUSTER_STATE="new"
+Environment=ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
+ExecStart=/usr/local/bin/etcd
+Restart=on-failure
+RestartSec=5
+User=root
+
+[Install]
+WantedBy=multi-user.target
+```
+
+##### 1.2.2.2 清除缓存,应用配置,启动etcd服务
+
+```bash
+systemctl stop etcd
+rm -rf /var/lib/etcd
+systemctl daemon-reload
+systemctl start etcd
+```
+
+##### 1.2.2.3 添加节点
+
+```bash
+etcdctl --endpoints=http://:2379 member add --peer-urls=http://:2380
+```
+
+有如下打印则为添加成功
+
+```
+[root@slot1 system]# etcdctl --endpoints=http://:2379 member add --peer-urls=http://:2380
+Member 6a5201d77502faa1 added to cluster 592c5fccdb3ab88c
+
+ETCD_NAME=""
+ETCD_INITIAL_CLUSTER="=http://:2380,=http://:2380"
+ETCD_INITIAL_ADVERTISE_PEER_URLS="http://:2380"
+ETCD_INITIAL_CLUSTER_STATE="existing"
+```
+
+##### 1.2.2.4 将打印的部分内容复制到新节点etcd的配置文件
+
+例:的/etc/systemd/system/etcd.service配置文件
+
+```
+[Unit]
+Description=etcd key-value store
+After=network.target
+
+[Service]
+Type=notify
+Environment=ETCD_NAME=
+Environment=ETCD_DATA_DIR=/var/lib/etcd
+Environment=ETCD_LISTEN_PEER_URLS=http://0.0.0.0:2380
+Environment=ETCD_LISTEN_CLIENT_URLS=http://0.0.0.0:2379
+Environment=ETCD_INITIAL_ADVERTISE_PEER_URLS=http://:2380
+Environment=ETCD_ADVERTISE_CLIENT_URLS=http://:2379
+Environment=ETCD_INITIAL_CLUSTER="=http://:2380,node182=http://:2380"
+Environment=ETCD_INITIAL_CLUSTER_STATE="existing"
+Environment=ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
+ExecStart=/usr/local/bin/etcd
+Restart=on-failure
+RestartSec=5
+User=root
+
+
+[Install]
+WantedBy=multi-user.target
+```
+
+##### 1.2.2.5 在新节点的机器上启动服务
+
+```bash
+systemctl stop etcd
+rm -rf /var/lib/etcd
+systemctl daemon-reload
+systemctl start etcd
+```
+
+##### 1.2.2.6 查看集群
+
+```bash
+etcdctl --endpoints=http://:2379 member list --write-out=table
+```
+
+##### 1.2.2.7 重复3-6步骤添加新节点进集群
+
+### 1.3 其他
+
+#### 1.3.1 删除节点
+
+查看当前集群成员列表
+
+```bash
+etcdctl member list
+```
+
+删除指定ID的节点
+
+```bash
+etcdctl member remove
+```
+
+#### 1.3.2 查看etcd集群的Leader
+
+```bash
+etcdctl get mooncake-store/mooncake_cluster/master_view --print-value-only
+```
+
+
+
+## 2 MinIO(S3)环境搭建
+### 2.1 下载 MinIO 二进制文件
+
+```bash
+wget https://dl.min.io/server/minio/release/linux-amd64/minio
+```
+
+### 2.2 赋予执行权限
+
+```bash
+chmod +x minio
+```
+
+### 2.3 移动到系统路径
+
+```bash
+sudo mv minio /usr/local/bin/
+```
+
+### 2.4 创建数据目录
+
+```bash
+mkdir -p /home/minio_data
+```
+
+### 2.5 启动MinIO
+
+设置管理员用户名和密码
+
+```bash
+export MINIO_ROOT_USER=admin
+export MINIO_ROOT_PASSWORD=adminadmin
+```
+
+配置中写好所有节点的地址,在所有设备中运行
+
+```bash
+minio server \
+ --address ":9000" \
+ --console-address ":9001" \
+ http:///home/minio_data \
+ http:///home/minio_data
+```
+
+
+### 2.6 配置自启动服务(可选)
+
+创建 `/etc/systemd/system/minio.service`文件:
+
+```ini
+[Unit]
+Description=MinIO Object Storage
+Documentation=https://docs.min.io
+Wants=network-online.target
+After=network-online.target
+
+[Service]
+User=root
+Group=root
+# 直接设置环境变量,不依赖外部文件
+Environment="MINIO_ROOT_USER=admin"
+Environment="MINIO_ROOT_PASSWORD=admin12345"
+Environment="MINIO_VOLUMES=http:///home/minio_data http:///home/minio_data"
+ExecStart=/usr/local/bin/minio server --address ":9000" --console-address ":9001" $MINIO_VOLUMES
+Restart=always
+LimitNOFILE=65536
+
+[Install]
+WantedBy=multi-user.target
+```
+
+保存后执行:
+
+```bash
+sudo systemctl daemon-reload
+sudo systemctl start minio
+sudo systemctl status minio
+```
+
+### 2.8 下载MC
+
+```bash
+# 下载 mc 客户端
+wget https://dl.min.io/client/mc/release/linux-arm64/mc
+
+# 赋予执行权限
+chmod +x mc
+
+# 移动到系统路径,方便全局使用
+sudo mv mc /usr/local/bin/
+```
+
+### 2.9 连接MinIO服务器并添加别名
+
+```bash
+mc alias set <别名> http://<你的服务器IP>:9000 <管理员用户名> <密码>
+```
+
+### 2.10 查看服务器信息
+
+```bash
+mc admin info myminio
+```
+
+### 2.11 创建桶
+
+```bash
+mc mb myminio/mooncake-snapshot
+```
+
+### 2.12 查看所有桶
+
+```bash
+mc ls myminio
+```
+
+
+## 3 Mooncake S3环境编译
+### 3.1 配置AWS-S3环境
+
+#### 3.1.1 检查环境
+
+```bash
+ls -l /usr/local/lib64/libaws-cpp-sdk-s3.so
+```
+
+或
+
+```bash
+ls -l /usr/local/lib/libaws-cpp-sdk-s3.so
+```
+
+若文件存在,则环境搭建完成
+
+#### 3.1.2 安装(AWS SDK C++)
+
+```bash
+git clone --depth 1 --recurse-submodules --shallow-submodules https://github.com/aws/aws-sdk-cpp.git
+cd aws-sdk-cpp
+mkdir build && cd build
+cmake -DCMAKE_BUILD_TYPE=Release -DBUILD_ONLY="s3" -DENABLE_TESTING=OFF ..
+make -j$(nproc)
+make install
+ldconfig
+```
+
+### 3.2 Mooncake安装
+
+```bash
+git config --global http.sslVerify false
+git clone --recurse-submodules https://github.com/<对应仓>/Mooncake.git
+cd Mooncake
+git checkout HA
+export GOPROXY="http://mirrors.aliyun.com/goproxy,direct"
+export GOINSECURE="go.etcd.io/etcd"
+export GOSUMDB="sum.golang.org"
+export CXXFLAGS="${CXXFLAGS} -w"
+export CFLAGS="${CFLAGS} -w"
+cmake -B build -DUSE_UB=ON -DBUILD_SHARED_LIBS=ON -DWITH_TE=ON -DWITH_STORE=ON -DWITH_P2P_STORE=ON -DUSE_ETCD=ON -DCMAKE_BUILD_TYPE=Release -DBUILD_UNIT_TESTS:BOOL=OFF -DSTORE_USE_ETCD=ON -DHAVE_AWS_SDK=ON
+cmake --build build -j$(nproc)
+git config --global http.sslVerify true
+```
+
+## 4 Mooncake HA测试脚本
+### 4.1 master运行脚本
+
+```bash
+export URMA_RPC_ENABLE=0
+export URMA_RPC_DEVICE=bonding_dev_0
+export URMA_RPC_EID_INDEX=0
+
+export MOONCAKE_SNAPSHOT_LOCAL_PATH=/home/mooncake_snapshot
+export MOONCAKE_AWS_ACCESS_KEY_ID="admin"
+export MOONCAKE_AWS_SECRET_ACCESS_KEY="adminadmin"
+export MOONCAKE_AWS_REGION="us-east-1"
+export MOONCAKE_AWS_BUCKET_NAME="mooncake-snapshot"
+export MOONCAKE_AWS_S3_ENDPOINT="http://:9000"
+export MOONCAKE_MASTER_SERVICE_SNAPSHOT_TEST_SKIP_CLEANUP=1
+export MC_LOG_DIR="/home/master_log"
+
+mooncake_master \
+ --default_kv_lease_ttl=300000 \
+ --default_kv_soft_pin_ttl=300000 \
+ --metrics_port=9006 \
+ --rpc_port=50052 \
+ --ha_backend_type=etcd \
+ --etcd_endpoints="etcd://:2379;:2379" \
+ --enable_ha=true \
+ --enable_oplog=true \
+ --rpc_address= \
+ --enable_metrics_report_to_backend=true \
+ --enable-offload=true \
+ --enable_snapshot=false \
+ --enable_snapshot_restore=false \
+ --snapshot_interval_seconds=10 \
+ --snapshot_retention_count=5 \
+ --snapshot_object_store_type="s3" \
+ --v=0 \
+ --cluster_id=mooncake_cluster
+
+```
+
+### 4.2 client运行脚本
+
+```bash
+export MC_URMA_ACTIVE_PORT=0
+export MOONCAKE_OFFLOAD_FILE_STORAGE_PATH=/home/mooncake_ssd
+export MC_STORE_CLIENT_METRIC=0
+export MC_STORE_CLIENT_METRIC_INTERVAL=3
+export MC_URMA_BONDING_MULTIPATH_ENABLE=on
+export MC_HIFREQ_LOG_SAMPLE_RATE=1
+export MOONCAKE_OFFLOAD_BUCKET_KEYS_LIMIT=1
+export GLOG_v=1
+export MC_LOG_DIR="/home/client_log"
+
+export URMA_RPC_ENABLE=0
+export URMA_RPC_DEVICE=bonding_dev_0
+export URMA_RPC_EID_INDEX=0
+
+mooncake_client \
+ --host= \
+ --metadata_server="etcd://:2379;:2379" \
+ --master_server_address="etcd://:2379;:2379" \
+ --protocol=ub \
+ --device_names=bonding_dev_0 \
+ --global_segment_size=214748364080 \
+ --port=50053 \
+ --threads=32 \
+ --v=0 \
+ --enable_offload=true
+```
+
+### 4.3 数据写入脚本
+
+```bash
+export MC_STORE_CLIENT_SETUP_RETRIES=3
+export no_proxy="127.0.0.1,localhost,local,.local,192.168.0.0/16,10.0.0.0/8,172.16.0.0/12,141.61.0.0/16"
+export MC_STORE_CLIENT_METRIC_BANDWIDTH=0
+export MC_TCP_BIND_ADDRESS=
+export MC_URMA_TRANS_MODE=RM
+
+export MC_URMA_BONDING_MULTIPATH_ENABLE=on
+export MC_HIFREQ_LOG_SAMPLE_RATE=1
+export MC_URMA_ACTIVE_PORT=0
+
+export URMA_RPC_ENABLE=0
+export URMA_RPC_DEVICE=bonding_dev_0
+export URMA_RPC_EID_INDEX=0
+
+export MC_LOG_DIR="/home/w00889253/client_log"
+
+stress_cluster_bench \
+ --metadata-server='etcd://:2379;:2379' \
+ --master-server='etcd://:2379;:2379' \
+ --local-hostname=$MC_TCP_BIND_ADDRESS \
+ --global-segment-size=0 \
+ --local-buffer-size=10737418240 \
+ --device-name=bonding_dev_0 \
+ --scenario=remote_memory \ # 四个场景segment_write,segment_read, remote_memory writer/reader
+ --role=writer \
+ --num-keys=10000 \
+ --protocol=ub \
+ --verify=true \
+ --num_threads=1 \
+ --batch-size=16 \
+ --duration=0 \
+ --master_admin_port=9006 \
+ --segments="," \
+ --replica_num=1
+
+
+
+```
+
+### 4.4 数据读取脚本
+
+```bash
+export MC_STORE_CLIENT_SETUP_RETRIES=3
+export no_proxy="127.0.0.1,localhost,local,.local,192.168.0.0/16,10.0.0.0/8,172.16.0.0/12,141.61.0.0/16"
+export MC_STORE_CLIENT_METRIC_BANDWIDTH=0
+export MC_TCP_BIND_ADDRESS=
+export MC_URMA_TRANS_MODE=RM
+
+export MC_URMA_BONDING_MULTIPATH_ENABLE=on
+export MC_HIFREQ_LOG_SAMPLE_RATE=1
+export MC_URMA_ACTIVE_PORT=0
+
+export URMA_RPC_ENABLE=0
+export URMA_RPC_DEVICE=bonding_dev_0
+export URMA_RPC_EID_INDEX=0
+
+export MC_LOG_DIR="/home/w00889253/client_log"
+
+stress_cluster_bench \
+ --metadata-server='etcd://:2379;:2379' \
+ --master-server='etcd://:2379;:2379' \
+ --local-hostname=$MC_TCP_BIND_ADDRESS \
+ --global-segment-size=0 \
+ --local-buffer-size=10737418240 \
+ --device-name=bonding_dev_0 \
+ --scenario=remote_memory \
+ --role=reader \
+ --num-keys=10000 \
+ --protocol=ub \
+ --verify=true \
+ --num_threads=1 \
+ --batch-size=16 \
+ --duration=0 \
+ --master_admin_port=9006 \
+ --segments="," \
+ --replica_num=1
+
+
+```
+
+
+
+
+## 5 多副本参数配置
+### 配置说明:replica_num / nof_replica_num 与 client TTL
+
+> 代码基线:kvcache-ai/Mooncake `main`
+> 内容:① 多副本参数 `replica_num` / `nof_replica_num` 的设置位置与方式;② client TTL 的两种配置途径(摘自《故障感知延迟分析-master多久发现节点故障.md》)
+
+---
+
+### 5.1 replica_num 与 nof_replica_num
+
+#### 5.1.1 定义
+
+定义在 `mooncake-store/include/replica.h` L81-83 的 `ReplicateConfig` 结构体:
+
+```cpp
+struct ReplicateConfig {
+ size_t replica_num{1}; // 内存副本数,默认 1
+ size_t nof_replica_num{0}; // NoF(NVMe-oF SSD)副本数,默认 0
+ ...
+};
+```
+
+#### 5.1.2 设置方式(三种场景)
+
+**方式 1:C++ 代码中直接赋值**
+
+```cpp
+ReplicateConfig config;
+config.replica_num = 2; // 2 个内存副本
+config.nof_replica_num = 1; // 1 个 NoF 副本
+auto result = client->Put(key, slices, config);
+```
+
+参考:`mooncake-store/tests/task_integration_test.cpp`、`mooncake-store/tests/replica_count_verify_test.cpp`。
+
+**方式 2:Python API**
+
+通过 pybind11 暴露为可读写属性(`mooncake-integration/store/store_py.cpp` L1832-1833,`def_readwrite`):
+
+```python
+from mooncake.store import ReplicateConfig
+
+config = ReplicateConfig()
+config.replica_num = 1 # 1 个内存副本
+config.nof_replica_num = 1 # 1 个 NoF 副本
+rc = store.put(key, data, config)
+```
+
+参考:`mooncake-wheel/tests/test_replicated_distributed_object_store.py`、`scripts/test_copy_move_api.py`。
+
+**方式 3:命令行参数 / 环境变量(E2E 与 Benchmark)**
+
+| 场景 | 参数 | 默认值 |
+|---|---|---|
+| E2E client(`store_client_e2e.py`) | `--memory-replica-num` / `--nof-replica-num` | 1 / 1 |
+| E2E 脚本(`run_nof_heartbeat_tcp_e2e.sh`) | 环境变量 `CLIENT_MEMORY_REPLICA_NUM` / `CLIENT_NOF_REPLICA_NUM` | 1 / 1 |
+| Benchmark(`store_kv_bench.py`) | `--memory-replica-num` / `--nof-replica-num` | 1 / 0 |
+
+E2E 用法示例:
+
+```bash
+CLIENT_MEMORY_REPLICA_NUM=1 CLIENT_NOF_REPLICA_NUM=1 bash run_nof_heartbeat_tcp_e2e.sh
+```
+
+#### 5.1.3 这两个值如何决定副本写入模式
+
+`DetermineReplicaWriteMode`(`mooncake-store/include/replica.h` L156-165):
+
+| `replica_num` | `nof_replica_num` | 写入模式 | 语义 |
+|---|---|---|---|
+| 1 | 0 | `SINGLE_REPLICA` | 单副本 |
+| 1 | 1 | `FLEXIBLE_DUAL_REPLICA` | 灵活双副本:1 内存 + 1 NoF,best-effort,任一类型成功即可(`HasExpectedReplicaAllocation` 只检查 `memory+nof > 0`) |
+| >1 或 >1 | 任意 | `RELIABLE_MULTI_REPLICA` | 可靠多副本:必须严格凑齐(`allocated_memory == replica_num` 且 `allocated_nof == nof_replica_num`) |
+| 0 | 0 | `SINGLE_REPLICA` | 退化为单副本 |
+
+#### 5.1.4 关键约束
+
+1. **`nof_replica_num > 0` 必须 USE_NOF=ON 编译**:否则 master 的 `PutStart` 直接返回 `INVALID_PARAMS`(`master_service.cpp` L3081-3088,`#ifndef USE_NOF` 分支;测试见 `replica_count_verify_test.cpp` Test 4)。
+2. **`replica_num` 与 `nof_replica_num` 不能同时为 0**:`store_kv_bench.py` L653 有显式校验 `ValueError`。
+3. **best-effort 语义**:`nof_replica_num == 0` 时,`HasExpectedReplicaAllocation`(`master_service.cpp` L121-133)只检查 `allocated_memory > 0`,不要求严格等于 `replica_num`。实测 `replica_num=5` 而只有 4 个 segment 时,分配 4 个副本仍算成功。
+
+---
+
+### 5.2 client TTL 的两种配置
+
+#### 5.2.1 背景结论
+
+- **master 多久发现 client 故障**:默认 **9~11 秒**(= client TTL ± 1s,TTL 默认 10s)。机制是"租约到期",不是"心跳中断检测"。
+- **可配置项**:client TTL 可配置;client 心跳间隔(1s,`client_service.cpp` L3743 硬编码)和 master 检查周期(1s,`master_service.h` L1980 硬编码)不可配。
+- **配置位置**:这是 **`mooncake_master` 的启动参数,不是 client 的**。
+
+#### 5.2.2 两种配置途径
+
+**途径 1:命令行参数 `--client_ttl`**。gflag 定义(`master.cpp` L260-264),启动时 `mooncake_master --client_ttl=5` 即生效:
+
+```cpp
+DEFINE_int64(
+ client_ttl, mooncake::DEFAULT_CLIENT_LIVE_TTL_SEC,
+ "Seconds a client stays considered alive after the last heartbeat. "
+ "If this TTL elapses without a refresh, the master treats the "
+ "client as disconnected and may unmount its segments");
+```
+
+**途径 2:master 配置文件(键名 `client_live_ttl_sec`)**。master 启动时若指定了配置文件,会从中读这个键;读不到则用 gflag 的值兜底(`master.cpp` L465-467,第三个参数就是兜底值 `FLAGS_client_ttl`):
+
+```cpp
+ default_config.GetInt64("client_live_ttl_sec",
+ &master_config.client_live_ttl_sec,
+ FLAGS_client_ttl);
+```
+
+**优先级:命令行显式设置 > 配置文件**。命令行显式传了 `--client_ttl`(`!info.is_default` 判断非默认值),或根本没用配置文件(`!conf_set`),则用命令行值覆盖(`master.cpp` L949-953):
+
+```cpp
+ if ((google::GetCommandLineFlagInfo("client_ttl", &info) &&
+ !info.is_default) ||
+ !conf_set) {
+ master_config.client_live_ttl_sec = FLAGS_client_ttl;
+ }
+```
+
+#### 5.2.3 配置速查表
+
+| 参数 | 默认 | 可配 | 配置在哪 | 源码位置 |
+|---|---|---|---|---|
+| client TTL | 10s | ✅ | `mooncake_master --client_ttl=<秒>` / master 配置文件 `client_live_ttl_sec` | `master.cpp` L260-264 / L465-467 / L949-953;`types.h` L95 `DEFAULT_CLIENT_LIVE_TTL_SEC=10` |
+| client 心跳间隔 | 1s | ❌ | —(client 代码硬编码) | `client_service.cpp` L3743 |
+| master 检查周期 | 1s | ❌ | —(master 代码硬编码) | `master_service.h` L1980 |
+
+一句话:同一个参数,命令行叫 `client_ttl`、配置文件里叫 `client_live_ttl_sec`,最终都写进 `master_config.client_live_ttl_sec`,两头都设时命令行优先。
+
+实测:E2E 中 master 以 `--client_ttl=5` 启动,kill 数据节点后感知延迟落在 4~6s 区间,与理论吻合。
+
+### 5.3 bench方案验证
+
+验证思路:验证内存存在副本和disk磁盘存在副本两种方式进行验证。
+
+#### 5.3.1 内存存在副本验证思路
+
+写入500个key,如果存在两个client,则应该两边分别写入250个key,同时应该在另外一个client上面存在副本,则每个节点client内存应该有500个key。
+操作步骤:先写入500个key,然后断连一个节点client,读取全部的key,可以从另外一个节点上全部读取。
+
+#### 5.3.2 disk磁盘存在副本验证思路
+
+写入500个key,如果存在两个client,则两边都存在250个key,,同时应该在另外一个client上面存在副本,则每个节点client内存应该有500个key。同时将这些key卸载到本地磁盘当中,磁盘当中应该在每个节点上有完整的500个key。
+
+操作步骤:先写入500个key,然后断连两个节点client, 重启其中一个client,此时磁盘中的卸载的数据可以在master上重新加载,有500个key,此时读取数据应该能够将所有的数据都读到。
+
+参考脚本:
+master
+
+```
+mooncake_master \
+ --enable_http_metadata_server=true \
+ --http_metadata_server_host=0.0.0.0 \
+ --http_metadata_server_port=9109 \
+ --default_kv_lease_ttl=300000 \
+ --default_kv_soft_pin_ttl=300000 \
+ --metrics_port=9006 \
+ --rpc_port=50052 \
+ --enable-offload=true
+
+```
+
+client
+
+```
+export MC_TCP_BIND_ADDRESS=
+export MC_URMA_ACTIVE_PORT=0
+export MOONCAKE_OFFLOAD_FILE_STORAGE_PATH=/home/mooncake_ssd
+export MC_STORE_CLIENT_METRIC=0
+export MC_STORE_CLIENT_METRIC_INTERVAL=3
+export MC_URMA_BONDING_MULTIPATH_ENABLE=on
+export MC_HIFREQ_LOG_SAMPLE_RATE=0
+export MOONCAKE_OFFLOAD_BUCKET_KEYS_LIMIT=1(重要,确保所有key存入到磁盘当中)
+
+export URMA_RPC_ENABLE=0
+export URMA_RPC_DEVICE=bonding_dev_0
+export URMA_RPC_EID_INDEX=0
+mooncake_client \
+ --host= \
+ --metadata_server=http://:9109/metadata \
+ --master_server_address=:50052 \
+ --protocol=ub \
+ --device_names=bonding_dev_0 \
+ --global_segment_size=21474836408 \
+ --port=50053 \
+ --threads=32 \
+ --v=0 \
+ --enable_offload=true
+
+```
+
+写入/读取脚本
+
+```
+export MC_STORE_CLIENT_SETUP_RETRIES=3
+export no_proxy="127.0.0.1,localhost,local,.local,192.168.0.0/16,10.0.0.0/8,172.16.0.0/12,141.61.0.0/16"
+export MC_STORE_CLIENT_METRIC_BANDWIDTH=0
+export MC_TCP_BIND_ADDRESS=
+export MC_URMA_TRANS_MODE=RM
+export MC_LOG_ENABLE=off
+# export MC_LOG_DIR=/home/log
+export MC_URMA_BONDING_MULTIPATH_ENABLE=on
+export MC_HIFREQ_LOG_SAMPLE_RATE=0
+export MC_URMA_ACTIVE_PORT=0
+
+export URMA_RPC_ENABLE=0
+export URMA_RPC_DEVICE=bonding_dev_0
+export URMA_RPC_EID_INDEX=0
+
+stress_cluster_bench \
+ --metadata-server='http://:9109/metadata' \
+ --master-server=':50052' \
+ --local-hostname=$MC_TCP_BIND_ADDRESS \
+ --global-segment-size=0 \
+ --local-buffer-size=10737418240 \
+ --device-name=bonding_dev_0 \
+ --scenario=remote_memory \ (使用remote memory,这样key可以以固定结构写入)
+ --role=reader \ (写入数据使用writer,读取使用reader)
+ --num-keys=500 \
+ --protocol=ub \
+ --verify=false \
+ --num_threads=1 \
+ --batch-size=16 \
+ --duration=0 \
+ --master_admin_port=9006 \
+ --replica_num=2 (副本数量)
+
+```
+
+
+# 6 mooncake基础信息上报
+
+在上述启动master的脚本中,配置```enable_metrics_report_to_backend=true``,可以通过etcd命令行查找到对应数据
+参考示例如下:
+
+```
+[root@node1 Mooncake]# etcdctl --endpoints=:2379,:2379 get --prefix /mooncake_cluster/masters/primary
+/mooncake_cluster/masters/primary
+{"id":"4056379308262733915-15491753023959092137","hostname":":50052","role":"primary","mem_total_bytes":42949672816,"mem_used_bytes":4194304000,"mem_available_bytes":38755368816,"nof_total_bytes":0,"nof_used_bytes":0,"nof_available_bytes":0,"file_total_bytes":4398046511104,"file_used_bytes":9080668160,"file_available_bytes":4388965842944,"key_count":500,"active_clients":2,"updated_at":"2026-08-06T16:59:35+08:00"}
+[root@node1 Mooncake]# etcdctl --endpoints=:2379,:2379 get --prefix /mooncake_cluster/masters/primary
+/mooncake_cluster/masters/primary
+{"id":"4056379308262733915-15491753023959092137","hostname":":50052","role":"primary","mem_total_bytes":42949672816,"mem_used_bytes":4194304000,"mem_available_bytes":38755368816,"nof_total_bytes":0,"nof_used_bytes":0,"nof_available_bytes":0,"file_total_bytes":4398046511104,"file_used_bytes":9080668160,"file_available_bytes":4388965842944,"key_count":500,"active_clients":2,"updated_at":"2026-08-06T17:00:40+08:00"}
+```
+
+
+# 7 mooncake性能信息
+
+使用脚本```Mooncake/mooncake-store/benchmarks/cluster_mooncake_diag.py```
+参考其中的使用方法,在对应脚本当中设置对应log日志,则可以进行读取。
\ No newline at end of file
diff --git a/mooncake-common/etcd/etcd_wrapper.go b/mooncake-common/etcd/etcd_wrapper.go
index 7cedada048..f75a1e3415 100644
--- a/mooncake-common/etcd/etcd_wrapper.go
+++ b/mooncake-common/etcd/etcd_wrapper.go
@@ -811,8 +811,49 @@ func EtcdStoreBatchCreateWrapper(keys **C.char, values **C.char, count C.int, er
return 0
}
+//export EtcdStoreBatchPutWithLeaseWrapper
+func EtcdStoreBatchPutWithLeaseWrapper(keys **C.char, keySizes *C.int, values **C.char, valueSizes *C.int, count C.int, leaseId int64, errMsg **C.char) int {
+ cli := getStoreClient()
+ if cli == nil {
+ *errMsg = C.CString("etcd client not initialized")
+ return -1
+ }
+
+ n := int(count)
+ if n == 0 {
+ return 0
+ }
+
+ // Unsafe casting to access C arrays as Go slices; key/value buffers are
+ // binary-safe (carry explicit sizes).
+ keyPtrs := (*[1 << 28]*C.char)(unsafe.Pointer(keys))[:n:n]
+ keySizeList := (*[1 << 28]C.int)(unsafe.Pointer(keySizes))[:n:n]
+ valPtrs := (*[1 << 28]*C.char)(unsafe.Pointer(values))[:n:n]
+ valSizeList := (*[1 << 28]C.int)(unsafe.Pointer(valueSizes))[:n:n]
+
+ ops := make([]clientv3.Op, 0, n)
+ for i := 0; i < n; i++ {
+ k := C.GoStringN(keyPtrs[i], keySizeList[i])
+ v := C.GoStringN(valPtrs[i], valSizeList[i])
+ // Bind all keys to the single master lease. Caller is responsible for
+ // fencing (the keys must already be absent / owned by self); this is an
+ // unconditional atomic batch, so it can also be reused for reaffirm.
+ ops = append(ops, clientv3.OpPut(k, v, clientv3.WithLease(clientv3.LeaseID(leaseId))))
+ }
+
+ ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
+ defer cancel()
+
+ _, err := cli.Txn(ctx).Then(ops...).Commit()
+ if err != nil {
+ *errMsg = C.CString(err.Error())
+ return -1
+ }
+ return 0
+}
+
//export EtcdStoreTxnCompareAndPutWrapper
-func EtcdStoreTxnCompareAndPutWrapper(compareKeys **C.char, compareKeySizes *C.int, compareKinds *C.int, compareValues **C.char, compareValueSizes *C.int, compareCount C.int, putKeys **C.char, putKeySizes *C.int, putValues **C.char, putValueSizes *C.int, putCount C.int, errMsg **C.char) int {
+func EtcdStoreTxnCompareAndPutWrapper(compareKeys **C.char, compareKeySizes *C.int, compareKinds *C.int, compareValues **C.char, compareValueSizes *C.int, compareCount C.int, putKeys **C.char, putKeySizes *C.int, putValues **C.char, putValueSizes *C.int, putCount C.int, deleteKeys **C.char, deleteKeySizes *C.int, deleteCount C.int, errMsg **C.char) int {
cli := getStoreClient()
if cli == nil {
*errMsg = C.CString("etcd client not initialized")
@@ -821,6 +862,7 @@ func EtcdStoreTxnCompareAndPutWrapper(compareKeys **C.char, compareKeySizes *C.i
cmpN := int(compareCount)
putN := int(putCount)
+ deleteN := int(deleteCount)
cmps := make([]clientv3.Cmp, 0, cmpN)
if cmpN > 0 {
@@ -844,7 +886,7 @@ func EtcdStoreTxnCompareAndPutWrapper(compareKeys **C.char, compareKeySizes *C.i
}
}
- ops := make([]clientv3.Op, 0, putN)
+ ops := make([]clientv3.Op, 0, putN+deleteN)
if putN > 0 {
putKeyPtrs := (*[1 << 28]*C.char)(unsafe.Pointer(putKeys))[:putN:putN]
putKeySizeList := (*[1 << 28]C.int)(unsafe.Pointer(putKeySizes))[:putN:putN]
@@ -856,6 +898,14 @@ func EtcdStoreTxnCompareAndPutWrapper(compareKeys **C.char, compareKeySizes *C.i
ops = append(ops, clientv3.OpPut(k, v))
}
}
+ if deleteN > 0 {
+ deleteKeyPtrs := (*[1 << 28]*C.char)(unsafe.Pointer(deleteKeys))[:deleteN:deleteN]
+ deleteKeySizeList := (*[1 << 28]C.int)(unsafe.Pointer(deleteKeySizes))[:deleteN:deleteN]
+ for i := 0; i < deleteN; i++ {
+ k := C.GoStringN(deleteKeyPtrs[i], deleteKeySizeList[i])
+ ops = append(ops, clientv3.OpDelete(k))
+ }
+ }
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
diff --git a/mooncake-common/include/rpc_client_io_context.h b/mooncake-common/include/rpc_client_io_context.h
index b9842f0c99..fb575df939 100644
--- a/mooncake-common/include/rpc_client_io_context.h
+++ b/mooncake-common/include/rpc_client_io_context.h
@@ -8,6 +8,7 @@
#include
#include
#include
+#include
#include
#include
@@ -26,9 +27,10 @@ coro_io::io_context_pool& GetRpcClientIoContextPool(uint32_t thread_count) {
}
/**
- * A replaceable client pool for callers that communicate with one target at a
- * time. Requests retain a shared_ptr to the old pool while they are in flight;
- * after an address switch the old pool is destroyed when those requests end.
+ * A client pool accessor that caches one client pool per target address, so
+ * callers that alternate between several targets (e.g. submaster routing)
+ * reuse existing connections instead of recreating a pool on every switch.
+ * The most recently selected pool is also exposed via GetClientPool().
*/
class RpcClientPool {
public:
@@ -38,18 +40,22 @@ class RpcClientPool {
explicit RpcClientPool(coro_io::io_context_pool& io_context_pool,
PoolConfig config = {})
: io_context_pool_(io_context_pool), config_(std::move(config)) {
- // Address replacement supersedes background recovery of the old host.
+ // Explicit target selection supersedes background recovery of an old
+ // host; the pool's own connect/retry still applies per request.
config_.host_alive_detect_duration = std::chrono::seconds(0);
}
std::shared_ptr GetOrCreateClientPool(
std::string_view address) {
std::lock_guard lock(mutex_);
- if (!client_pool_ || address_ != address) {
- client_pool_ =
- ClientPool::create(address, config_, io_context_pool_);
- address_ = address;
+ std::string addr(address);
+ auto it = pools_.find(addr);
+ if (it == pools_.end()) {
+ auto pool = ClientPool::create(addr, config_, io_context_pool_);
+ it = pools_.emplace(addr, std::move(pool)).first;
}
+ address_ = std::move(addr);
+ client_pool_ = it->second;
return client_pool_;
}
@@ -58,12 +64,21 @@ class RpcClientPool {
return client_pool_;
}
+ std::string GetAddress() const {
+ std::shared_lock lock(mutex_);
+ return address_;
+ }
+
private:
mutable std::shared_mutex mutex_;
coro_io::io_context_pool& io_context_pool_;
PoolConfig config_;
std::string address_;
std::shared_ptr client_pool_;
+ // Cache of client pools keyed by target address, so switching back and
+ // forth between submaster addresses reuses existing connections instead of
+ // recreating the pool on every switch.
+ std::unordered_map> pools_;
};
} // namespace mooncake
diff --git a/mooncake-common/tests/rpc_client_io_context_test.cpp b/mooncake-common/tests/rpc_client_io_context_test.cpp
index c3b18c7127..d63fd5e34e 100644
--- a/mooncake-common/tests/rpc_client_io_context_test.cpp
+++ b/mooncake-common/tests/rpc_client_io_context_test.cpp
@@ -41,18 +41,19 @@ TEST(RpcClientIoContextPoolTest, UsesConfiguredSizeAndReusesPool) {
EXPECT_NE(&first_pool, &second_pool);
}
-TEST(RpcClientIoContextPoolTest, ReplacesPoolWhenTargetChanges) {
+TEST(RpcClientIoContextPoolTest, CachesPoolsPerAddress) {
RpcClientPool pools(GetFirstTestRpcClientIoContextPool());
auto first = pools.GetOrCreateClientPool("127.0.0.1:10001");
- std::weak_ptr old_pool = first;
EXPECT_EQ(pools.GetOrCreateClientPool("127.0.0.1:10001"), first);
auto second = pools.GetOrCreateClientPool("127.0.0.1:10002");
EXPECT_NE(first, second);
- first.reset();
- EXPECT_TRUE(old_pool.expired());
EXPECT_EQ(pools.GetClientPool(), second);
+
+ // Switching back reuses the cached pool instead of recreating it.
+ EXPECT_EQ(pools.GetOrCreateClientPool("127.0.0.1:10001"), first);
+ EXPECT_EQ(pools.GetClientPool(), first);
}
TEST(RpcClientIoContextPoolTest, SendsToNewAddressAfterSwitch) {
diff --git a/mooncake-store/benchmarks/CMakeLists.txt b/mooncake-store/benchmarks/CMakeLists.txt
index 817230b940..f851e0b2be 100644
--- a/mooncake-store/benchmarks/CMakeLists.txt
+++ b/mooncake-store/benchmarks/CMakeLists.txt
@@ -38,6 +38,16 @@ target_link_libraries(
stress_cluster_bench PRIVATE mooncake_store transfer_engine asio_shared
gflags::gflags glog::glog pthread)
+# CFM client benchmark. Simulates vLLM inference requests as KV-cache block
+# accesses and reports both client performance and IO Pattern observability.
+add_executable(cfm_client_bench cfm_client_bench.cpp)
+target_link_libraries(
+ cfm_client_bench PRIVATE mooncake_store transfer_engine asio_shared
+ gflags::gflags glog::glog pthread)
+if(STORE_USE_ETCD)
+ target_link_libraries(cfm_client_bench PRIVATE ${ETCD_WRAPPER_LIB})
+endif()
+
# Benchmark for vLLM Store Connector path
# Triggers: batch_put_from_multi_buffers / batchIsExist /
# batch_get_into_multi_buffers (and setup/register_buffer/tearDownAll).
@@ -75,3 +85,10 @@ if(STORE_USE_ETCD)
oplog_batch_bench PRIVATE mooncake_store glog::glog gflags::gflags
JsonCpp::JsonCpp)
endif()
+
+if(STORE_USE_ETCD)
+ add_executable(vchunk_distributed_bench vchunk_distributed_bench.cpp)
+ target_link_libraries(
+ vchunk_distributed_bench PRIVATE mooncake_store transfer_engine asio_shared
+ gflags::gflags glog::glog pthread)
+endif()
diff --git a/mooncake-store/benchmarks/cfm_client_bench.cpp b/mooncake-store/benchmarks/cfm_client_bench.cpp
new file mode 100644
index 0000000000..810c035ef4
--- /dev/null
+++ b/mooncake-store/benchmarks/cfm_client_bench.cpp
@@ -0,0 +1,1517 @@
+// CFM benchmark that models the vLLM KV-cache call path in the embedded CFM
+// architecture.
+//
+// CFM is a component of every SubMaster; there is no standalone CFM Master and
+// no credential. A reporting client observes keys (KV blocks) and sends metric
+// batches over the SubMaster's regular coro_rpc endpoint. The SubMaster merges
+// reports into its local runtime, then every merged report drives a local
+// analysis -> decision -> execution cycle (eviction/prefetch/promotion/
+// admission through the storage-safe handlers) on the keys it owns.
+//
+// This benchmark exercises that path in the following modes:
+// - embedded (default): an in-process SubMaster runtime plays the owning
+// CFM component. Reports are delivered in-process; the runtime's
+// report-driven worker executes policy per report, and a final manual
+// Execute emulates the production high-watermark trigger, so the benchmark
+// prints report latency and the resulting eviction/prefetch/admission
+// handler activity.
+// - remote (--cfm_endpoint=host:port): reports go over coro_rpc to a single
+// SubMaster CFM receiver.
+// - remote via etcd (--cfm_endpoint=etcd://connstring): resolves the cluster
+// like a Store client, then buckets each key to its owning SubMaster.
+// The receiving side is not observable here, so only client-side latency
+// is reported.
+//
+// Remote policy execution is only observable when the SubMaster actually owns
+// the reported keys: its eviction/promotion/prefetch handlers operate on real
+// replicas, so a report-only run leaves the master-side counters at zero. When
+// --master_server and --num_keys are provided, a real-data seeding stage runs
+// first ("先种子后仿真"): it writes a batch of real KV objects through
+// RealClient (keys share the simulated KvKey naming and tenant) and reads a
+// subset back to simulate access heat, then the simulated vLLM request stream
+// reports on those same keys.
+
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include