[spark] Fix duplicated Hive metastore test port between two Spark ITCases#8824
Merged
JingsongLi merged 1 commit intoJul 23, 2026
Merged
Conversation
…ases SparkDataEvolutionITCase and SparkMultimodalITCase both hardcoded the Hive metastore port 9092. Both run in the same paimon-spark-ut surefire JVM, and TestHiveMetastore.stop() does not await the serve thread's termination, so the first class's socket can still be bound (or in TCP TIME_WAIT) when the second class calls start(9092). The result is an intermittent "java.net.BindException: Address already in use" that fails whichever class starts its metastore second, which is why CI flaked nondeterministically. Every other metastore test in the module already uses a unique fixed port (9083, 9087, 9091, ...); this was the lone duplicate. Give SparkDataEvolutionITCase port 9094, which is referenced nowhere else in the repository, restoring the module's one-fixed-port-per-class convention. SparkMultimodalITCase keeps 9092, now unique. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Contributor
|
+1 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
SparkDataEvolutionITCaseandSparkMultimodalITCaseboth hardcoded the Hive metastore port9092. They run in the samepaimon-spark-utsurefire JVM (reuseForks), andTestHiveMetastore.stop()does not await the serve thread's termination, so the first class's server socket can still be bound (or in TCPTIME_WAIT) when the second class callsstart(9092). Whichever class starts its metastore second then fails withjava.net.BindException: Address already in use (Bind failed). Class ordering within the fork is nondeterministic, so this surfaces as an intermittent red on the "UTCase and ITCase Spark 3.x" leg.The flake was first observed on the CI of an unrelated PR #8821, whose change is confined to
paimon-codegenand touches neither Spark nor Hive. The failing job was https://github.com/apache/paimon/actions/runs/29985121549/job/89135177493, whereSparkMultimodalITCase.startMetastorecould not bind9092.Every other metastore test in the module already picks a unique fixed port (
9083,9087,9091, and so on), and the siblingpaimon-hive-connector-commontests use9082/9084/9085/9086/9088;9092was the only duplicate. This change givesSparkDataEvolutionITCaseport9094, which is referenced nowhere else in the repository, restoring the module's one-fixed-port-per-class convention.SparkMultimodalITCasekeeps9092, now unique.9094was chosen deliberately: the contiguous block9082-9092is fully allocated across the JavaPORTconstants and the per-modulehive-site.xmlresources (9089=spark-3.4,9090=spark-3.5,9091=spark-4.0,9092=spark-4.1), and those modules build concurrently under the CI-T 2Creactor, so the next free value that cannot collide with any of them was picked. OS-assigned ephemeral ports (used by bind-to-zero tests) live in32768-60999, well above9094, so they cannot land on it either.Tests
No new tests. This only reassigns a test-only port constant.
SparkDataEvolutionITCaseandSparkMultimodalITCasecontinue to start their metastores and pass, now on distinct ports so they can no longer collide within the shared surefire JVM.