Skip to content

[KYUUBI #7712] Support IPv6 host:port parsing in Kyuubi service discovery - #7713

Open
maomaodev wants to merge 3 commits into
apache:masterfrom
maomaodev:kyuubi-7712
Open

[KYUUBI #7712] Support IPv6 host:port parsing in Kyuubi service discovery#7713
maomaodev wants to merge 3 commits into
apache:masterfrom
maomaodev:kyuubi-7712

Conversation

@maomaodev

@maomaodev maomaodev commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

Why are the changes needed?

Fix #7712. Kyuubi currently cannot discover or connect to server/engine endpoints in IPv6-only clusters. Both the server side (DiscoveryClient.parseInstanceHostPort) and the standalone JDBC client side (ZooKeeperHiveClientHelper) parse a host:port string by naively splitting on ":". This breaks for IPv6 because IPv6 addresses themselves contain :.

The correct parsing behavior already exists upstream in Hive (org.apache.hadoop.hive.common.IPStackUtils). This patch mirrors that logic in a new lightweight IPStackUtils utility under kyuubi-util, so it can be shared between the server side and the standalone JDBC client (which only depends on kyuubi-util).

How was this patch tested?

  • Added IPStackUtilsTest (11 test cases) aligned with Hive's own IPStackUtilsTest, covering IPv4, IPv6 with brackets, IPv6 without brackets, hostname, invalid port, missing port, missing host and port range validation.
  • Extended DiscoveryClientSuite with IPv6 cases (with and without brackets) and converted it from an orphan trait to a runnable class.

Was this patch assisted by generative AI tooling?

Assisted-by: DeepSeek-V4-Pro

Comment thread kyuubi-util/src/main/java/org/apache/kyuubi/util/IPStackUtils.java
Comment thread kyuubi-util/src/main/java/org/apache/kyuubi/util/IPStackUtils.java Outdated
@pan3793 pan3793 added this to the v1.13.0 milestone Sep 4, 2026

@pan3793 pan3793 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please prefix the title with [KYUUBI #7712].

A few more places have the same issue and should be fixed in this PR:

  • kyuubi-ha/src/main/scala/org/apache/kyuubi/ha/client/zookeeper/ZookeeperDiscoveryClient.scala L319: when kyuubi.ha.zookeeper.publish.configs=true, an IPv6 instance is split on the first colon and publishes a broken host/port. Please use IPStackUtils.getHostAndPort too.
  • kyuubi-server/src/main/scala/org/apache/kyuubi/server/ui/JettyServer.scala L59: getServerUri (and the advertisedHost branch in KyuubiRestFrontendService.connectionUrl) emits unbracketed IPv6, which breaks the batch kyuubiInstance / InternalRestClient path in IPv6-only clusters. Please fix these with IPStackUtils as well.
  • kyuubi-hive-jdbc/src/main/java/org/apache/kyuubi/jdbc/hive/Utils.java L409 (also L422, L536): authority strings concatenate host without brackets, so the rewritten jdbcUriString is malformed for IPv6. Please use IPStackUtils.formatIPAddressForURL when building them.

IPStackUtils.HostPort hostPort = IPStackUtils.getHostAndPort(dataStr);
connParams.setHost(hostPort.getHostname());
connParams.setPort(hostPort.getPort());
} catch (Exception e) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

getHostAndPort throws IllegalArgumentException only, catch it instead of Exception.

}

host + ":" + actualPort
IPStackUtils.formatIPAddressForURL(host) + ":" + actualPort

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Port Hive's concatHostPort and use it here, it also validates host and port.

@maomaodev maomaodev changed the title Support IPv6 host:port parsing in Kyuubi service discovery [KYUUBI #7712] Support IPv6 host:port parsing in Kyuubi service discovery Sep 4, 2026
@maomaodev

Copy link
Copy Markdown
Contributor Author

@pan3793 Good catch! Fixed all three, and swept the codebase for the same pattern.

Your three:

  • ZookeeperDiscoveryClient.addConfsToPublish: split(":", 2)IPStackUtils.getHostAndPort.
  • JettyServer (start / getServerUri) & KyuubiRestFrontendService.connectionUrl (advertisedHost): → IPStackUtils.concatHostPort.
  • Utils.java L409/L422/L536: authority building → IPStackUtils.concatHostPort (which internally calls formatIPAddressForURL + validates host/port).

Also fixed (same bug elsewhere):

  • KyuubiTrinoFrontendService.connectionUrl (advertisedHost branch).
  • ServiceNodeInfo: the instance string published to service discovery.
  • EngineRef: Data Agent JDBC URL.
  • KyuubiConnection.getServerHttpUrl: HTTP URL construction.
  • KyuubiConfFileParser.addHosts: hosts property fed to beeline JDBC URL builder.

Related:

  • Ported Hive's concatHostPort (validates host/port) into IPStackUtils with UT mirroring Hive.
  • Narrowed catch (Exception) in DiscoveryClient to catch (IllegalArgumentException) per your earlier point.

Intentionally left alone (all non-functional, happy to revisit if you disagree):

  • KyuubiSessionImpl.scala: 5 s"$host:$port" occurrences, all in warn / info / error / logSessionInfo messages — display only, no parsing downstream.
  • KyuubiConnection.java: LOG.warn("Failed to connect to " + host + ":" + port) and logZkDiscoveryMessage("Connected to " + host + ":" + port) — same, log-only.
  • TBinaryFrontendService.scala / KyuubiTHttpFrontendService.scala: s"Failed to initialize frontend service on $serverAddr:$portNum" — exception message string, not consumed by any code path.
  • EmbeddedZookeeper.scala: s"...binding to $host:$clientPort" — exception message, real ZK connect string comes from getConnectString.
  • EngineUIProxyServlet.scala L87: X-Forwarded-Context header value is a path prefix, not a URL authority.
  • HiveSiteHS2ConnectionFileParser.java L215: Hive-side compatibility layer using getHostName() (returns hostname string, not a numeric address).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] Kyuubi service discovery cannot parse IPv6 host:port endpoints

2 participants