Skip to content

[server] Support DescribeTabletServers API for safe scaling and upgrades#3743

Open
morazow wants to merge 1 commit into
apache:mainfrom
morazow:describe-tablet-servers-3570
Open

[server] Support DescribeTabletServers API for safe scaling and upgrades#3743
morazow wants to merge 1 commit into
apache:mainfrom
morazow:describe-tablet-servers-3570

Conversation

@morazow

@morazow morazow commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Adds a read-only DescribeTabletServers RPC to the Coordinator, following the pattern of GetClusterHealth ([server] Support Cluster Health API for safe rolling upgrades #3399 / [server] Add Cluster Health API implementation #3400).
  • New Admin.describeTabletServers() (@PublicEvolving) returns, per tablet server, the same four counters getClusterHealth() reports cluster-wide, scoped to that server: numReplicas, inSyncReplicas, numLeaderReplicas, activeLeaderReplicas.
  • Counters are computed from in-memory CoordinatorContext state on the coordinator event thread (via AccessContextEvent); tablet servers forward the request to the coordinator over the internal listener.
  • Live and shutting-down servers are always reported (even with zero replicas), and dead servers still holding assigned replicas remain visible, so a scale-in gate cannot treat them as drained.
  • This backs the Kubernetes Operator FIP's serverGreen(serverId) rolling-upgrade gate, the scale-in safety gate (numReplicas == 0), and per-server tablet load reporting.

Fixes #3570

Test Plan

  • New unit test DescribeTabletServersTest (fluss-server) covering: empty cluster, full ISR, out-of-ISR replicas, inactive leaders, leaderless buckets, evacuated servers, dead-but-assigned servers, partitioned tables, and reconciliation of per-server sums with computeClusterHealth.
  • New ITCase FlussAdminITCase#testDescribeTabletServersDuringRollingUpgrade: healthy cluster is green and sums match cluster health; a stopped server stays reported, non-empty and non-green; after restart all servers become green again.
  • mvn spotless:check clean on fluss-rpc, fluss-server, fluss-client, fluss-flink-common.

🤖 AI-assisted changes - reviewed by human developer

@morazow
morazow force-pushed the describe-tablet-servers-3570 branch 2 times, most recently from 8df1402 to b3d64f5 Compare July 23, 2026 06:40
…upgrades

Add a read-only DescribeTabletServers RPC to the Coordinator, following
the pattern of GetClusterHealth (apache#3399). It returns, per tablet server,
the same four counters getClusterHealth() reports cluster-wide, scoped
to that server: numReplicas, inSyncReplicas, numLeaderReplicas and
activeLeaderReplicas.

This lets operational tooling (e.g. the Fluss Kubernetes Operator):
- gate scale-in on a server being empty (numReplicas == 0),
- gate rolling upgrades on a per-server green predicate
  (inSyncReplicas == numReplicas && activeLeaderReplicas == numLeaderReplicas),
- report per-server tablet load in cluster status.

The counters are computed from in-memory CoordinatorContext state on
the coordinator event thread via AccessContextEvent. Tablet servers
forward the request to the coordinator. Live and shutting-down servers
are always reported, even with zero replicas, and dead servers that
still hold assigned replicas remain visible so a scale-in gate cannot
treat them as drained.

Closes apache#3570
@morazow
morazow force-pushed the describe-tablet-servers-3570 branch from b3d64f5 to 2aad05f Compare July 23, 2026 12:20
@morazow
morazow marked this pull request as ready for review July 23, 2026 14:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[server] Support DescribeTabletServers API for safe scaling and rolling upgrades

1 participant