[Bug] describe the main problem

[Noctis-lzy]

Bug Type (问题类型)

None

Before submit

• 我已经确认现有的 Issues 与 FAQ 中没有相同 / 重复问题 (I have confirmed and searched that there are no similar problems in the historical issue and documents)

Environment (环境信息)

HugeGraph 1.7.0(Apache Release,2025-11-16 发布，含 PR #2961 之前的代码)
3 节点 PD(raft 8610 / gRPC 8686 / REST 8620)+ 3 节点 store,CentOS 7.8,JDK 11

Expected & Actual behavior (期望与实际表现)

Expected(期望表现)：

1-PD 集群整体重启后，Raft 重新选主、leader 可能落在任意节点——这本身是正常行为；无论 leader 是哪台，follower 都应能将请求转发(redirect)到新 leader,集群保持高可用；
2-/v1/members 应正确显示各成员 Up、grpcUrl 已填充，pdLeader 指向当前 leader;
3-RaftEngine.getMembers() 所依赖的成员查询 RPC 应能正常工作(至少查询本机 localhost 不应失败)。

Actual(实际表现)：

1-(同 #3001)重启后 follower 转发瘫痪：整集群重启且 leader 换节点后，follower PD 持续报错，集群不可用：
[WARN] o.a.h.p.s.ServiceGrpc - redirect to leader with error:
冒号后为空(NPE 的 getMessage() 为 null)。根因是 1.7.0 的 RaftEngine.getLeaderGrpcAddress() 中 .get() 无超时、且对 null 响应直接调 .getGrpcAddress() 产生 NPE(master 已由 #2961 修复，但截至 2026-08 尚无包含该修复的 release)。

2-(本 issue 新增)底层成员查询 bolt RPC 对所有 peer 全部失败，包括 localhost:手工将 #2961 回移植到 1.7.0 后，follower 转发通过降级地址恢复、集群功能正常，但每次调用 /v1/members,leader 日志固定打出 3 对错误(对三个 raft peer 各一次，含 leader 查询自己 10.143.208.94:8610):
2026-08-26 21:27:40 [ERROR] o.a.h.p.r.RaftRpcClient - failed to call rpc to 10.143.208.92:8610. null
2026-08-26 21:27:40 [ERROR] o.a.h.p.r.RaftEngine - failed to getGrpcAddress of 10.143.208.92:8610
localhost 也 NPE 说明并非网络/防火墙问题，而是 RaftRpcClient.getGrpcAddress()(自定义 GetMemberRequest bolt 调用)本身的问题。#2961 只对 getLeaderGrpcAddress 一个调用方做了容错，未修复该 RPC 本身。

3-受影响的唯一未保护调用方 RaftEngine.getMembers():RPC 全败 → 所有成员被标 Offline、grpcUrl/restUrl/dataPath 全空 → /v1/members 返回 stateCountMap:{"Offline":3}、pdLeader:null,而实际 raft 集群状态为 Cluster_OK、选举/快照/日志复制均正常——监控视图与集群真实状态完全背离。

影响面与当前规避：

1-功能层不受影响(store 注册/心跳不调用该 RPC;PDClient 找 leader 用的是 GetMembersResponse.leader 字段，由 leader 本地构造、数据正确)；
2-但 #2961 的降级地址 = leader raft IP + 本机 grpc.port,要求所有 PD 节点 grpc 端口一致，且 /v1/members 的成员列表/pdLeader 不可信；
3-建议对 getMembers() 增加与 #2961 同等的容错，并排查 GetMemberRequest bolt RPC 本身 NPE 的根因(该错误日志 failed to call rpc to {}. {} 只打印 getMessage() 不打印堆栈，NPE 场景下无栈可查，建议补充异常对象输出)。

Vertex/Edge example (问题点 / 边数据举例)

null

Schema [VertexLabel, EdgeLabel, IndexLabel] (元数据结构)

[dosubot]

Hi @Noctis-lzy! I'm Dosu and I'm helping the HugeGraph team.

Your root cause analysis is accurate across all three problems. I've verified it against the current master codebase.

Problem 1: getMembers() has no timeout or null-safety on the bolt RPC

In RaftEngine.getMembers(), the call to future.get() at line 334 is unbounded — no timeout is applied. By contrast, getLeaderGrpcAddress() was hardened in PR #2961 with future.get(config.getRpcTimeout(), TimeUnit.MILLISECONDS) plus fallback handling [1], but getMembers() was not given the same treatment. Additionally, future.isCompletedExceptionally() is checked at line 328 before future.get() at line 334, creating a TOCTOU race — a future that completes exceptionally after the check but before get() will propagate as an unhandled ExecutionException, marking the node Offline.

Problem 2: The truncated failed to call rpc to {}. {} log

In RaftRpcClient.internalCallAsyncWithRpc(), the catch block logs:

log.error("failed to call rpc to {}. {}", endpoint, t.getMessage());

When the exception is an NPE, t.getMessage() returns null, producing the truncated log you observed. The exception object (including stack trace) is never logged. Changing this to log.error("failed to call rpc to {}", endpoint, t) would cause SLF4J to append the full stack automatically.

Problem 3: Why the bolt RPC fails for all peers including localhost

The IpAuthHandler whitelist configured in createRaftRpcServer() is populated from the peer IPs in the Raft configuration. A plausible explanation for the localhost failure is that the Bolt client's source IP is 127.0.0.1 (loopback) while the whitelist only contains the real NIC IP (e.g., 10.143.208.94), causing IpAuthHandler to reject the connection. This would produce an exception that, via FutureClosureAdapter.failure(), completes the future exceptionally — but since the error message is null for an NPE, the log gives no useful context. That said, I wasn't able to fully confirm this theory from the code alone; it may warrant further investigation.

Suggested fixes for getMembers():

1. Add timeout to future.get() — mirror the getLeaderGrpcAddress() pattern: future.get(config.getRpcTimeout(), TimeUnit.MILLISECONDS) with a catch (TimeoutException).
2. Fix the TOCTOU race on future.isCompletedExceptionally() — check after get() in the catch block rather than before.
3. Self-query shortcut — when peerId equals the local node's endpoint, skip the RPC and return getLocalMember() directly. This makes the leader-querying-itself case reliable regardless of whitelist or network config.
4. Improve logging — change the log.error in RaftRpcClient [2] to pass the throwable object directly so full stack traces are captured on NPE.

The self-query shortcut (point 3) is the most targeted fix for the "leader queries itself and fails" scenario. For a 1.7.0 backport, applying the timeout-bounded get() and the local-node shortcut to getMembers() should resolve /v1/members incorrectly showing all Offline and pdLeader: null.

To reply, just mention @dosu.

---

Docs are dead. Just use Dosu.