[KYUUBI #7708][SERVER] Reconcile zero-count metadata updates - #7709
[KYUUBI #7708][SERVER] Reconcile zero-count metadata updates#7709ruanwenjun wants to merge 1 commit into
Conversation
|
the change lgtm, cc @turboFei, who authored this feature, to take a look |
pan3793
left a comment
There was a problem hiding this comment.
Not for this PR: the queued InsertMetadata retry path has a similar amplification pattern. A queued insert that hits a duplicate-key error rethrows to the outer catch and stays at the queue head forever - the foreground path fails fast on unrecoverableDBErr, but the retry loop does not apply the same check. Worth a follow-up issue.
| } | ||
| } | ||
|
|
||
| private def verifyMetadataUpdatePostcondition( |
There was a problem hiding this comment.
Since #7708 says the original cause of the zero update count is still under investigation, please log a warn here when the row already matches, e.g. "update for {} returned 0 but all requested values already match, treating as idempotent success". Otherwise this case becomes invisible, while the old behavior at least surfaced an exception.
| extends MetadataUpdatePostconditionException( | ||
| s"Metadata row $identifier was not found after an update returned 0") | ||
|
|
||
| private[metadata] class MetadataUpdateMismatchException( |
There was a problem hiding this comment.
Nit: consider including expected vs actual values in the message, not just column names. When the discard is logged in MetadataManager, that makes the error actionable without reproducing the DB state.
Why are the changes needed?
A JDBC metadata
UPDATEmay report an update count of zero when the row is missing or when the requested values are already present. Kyuubi currently treats every zero count as a retryable failure. A persistent failure therefore remains in the retry queue and can block batch-session timeout cleanup while it waits for metadata retries to finish.When an
UPDATEreturns zero, this patch verifies only the updated fields on the same JDBC connection:INSERTwithout retrying a permanently missing row forever.Closes #7708.
How was this patch tested?
Added a SQLite trigger-based regression test that forces
executeUpdate()to return zero for both matching and mismatching rows.Added retry coverage showing that a persistently missing row is removed from the retry queue.
Kept the existing FIFO
INSERT-then-UPDATEretry coverage passing.Ran:
build/mvn test -pl kyuubi-server -am \ -Pspark-provided,flink-provided,hive-provided \ -Dtest=none \ -DwildcardSuites=org.apache.kyuubi.server.metadata.jdbc.JDBCMetadataStoreSuite,org.apache.kyuubi.server.metadata.MetadataManagerSuiteAll 13 tests passed.
Was this patch assisted by generative AI tooling?
Assisted-by: OpenAI Codex with GPT-5