Skip to content

perf(bigquery-jdbc): short-circuit disabled connection logs at Gate 1 - #15114

Open
Neenu1995 wants to merge 1 commit into
fix/bq-jdbc-per-conn-log-isolationfrom
perf/bq-jdbc-gate1-log-filtering
Open

Neenu1995 wants to merge 1 commit into
fix/bq-jdbc-per-conn-log-isolationfrom
perf/bq-jdbc-gate1-log-filtering

Conversation

@Neenu1995

Copy link
Copy Markdown
Contributor

What & Why

PR 3 of 3 in the diagnostic logging hardening stack (stacked on PR 2 fix/bq-jdbc-per-conn-log-isolation).

After PR 2 isolated per-connection (LogLevel, LogPath) slots on a shared persistent PerConnectionFileHandler, the JVM-wide com.google.cloud.bigquery root logger runs at the minimum (most verbose) active level across open connections. Without per-connection Gate-1 filtering (Logger.isLoggable(Level) / BigQueryJdbcResultSetLogger.isLoggable(Level)), if Connection A ran at FINEST while Connection B ran at LogLevel=0 (OFF) or LogLevel=3 (INFO), Connection B's hot-path calls (ResultSet getters, Struct/Array accessors, BigQueryJdbcContextProxy, and Supplier<String> / String.format log calls) would still evaluate message suppliers and allocate LogRecord objects before PerConnectionFileHandler.publish() dropped them at Gate 2.

Changes in this PR

  • Zero-allocation external JUL sink tracking (BigQueryJdbcRootLogger): Registers a RootDriverLogger subclass on com.google.cloud.bigquery with synchronized addHandler, removeHandler, and setUseParentHandlers overrides maintaining volatile boolean externalHandlerAttached without allocating Handler[] snapshots on every isLoggable call (plus a pure read-only scanForExternalHandler fallback if an external framework pre-initialized the logger before driver classload, and explicitGlobalLevelValue tracking for BigQueryJdbcRootLogger.setLevel(...)).
  • ConnectionLogContext propagation across BigQueryJdbcMdc & BigQueryJdbcContextProxy:
    • Upgrades BigQueryJdbcMdc from InheritableThreadLocal<String> to InheritableThreadLocal<ConnectionLogContext> (registerContext / registerAndBindConnection / getContext), propagating ConnectionLogContext across MdcThreadPoolExecutor worker tasks.
    • Caches ConnectionLogContext once per BigQueryJdbcContextProxy instance and binds it directly in BigQueryDriver.connect and BigQueryConnection.<init>, avoiding ConcurrentHashMap lookups per JDBC call.
    • Distinguishes unconfigured (INHERIT_VALUE) vs. explicit connection thresholds (OFF_VALUE..ALL_VALUE) in a single volatile int on ConnectionLogContext, and flips closed connections to OFF_VALUE so post-close ResultSet / detached Struct calls also short-circuit at Gate 1.
  • Gate-1 short-circuit in BigQueryJdbcCustomLogger & BigQueryJdbcResultSetLogger:
    • BigQueryJdbcCustomLogger.isLoggable(Level) and BigQueryJdbcResultSetLogger.isLoggable(Level) check the connection's ConnectionLogContext threshold directly (falling back to thread-local MDC for global ResultSet facades) whenever no custom child handler or external JUL sink is attached.
    • Isolates BigQueryJdbcResultSetLogger (isLoggable, warning, and log(LogRecord)) against enclosing-thread MDC cross-talk when two connections execute nested calls on the same thread.

How It Was Tested

  • JAVA_HOME=/usr/lib/jvm/java-17-openjdk-amd64 make lint (fmt-maven-plugin:format).
  • JAVA_HOME=/usr/lib/jvm/java-17-openjdk-amd64 make unittest (Checkstyle, Java 8 animal-sniffer, and the full JUnit 5 unit test suite) and mvn -q -Denforcer.skip=true -Dclirr.skip=true test-compile for integration test compilation.
  • Added focused JUnit 5 tests in BigQueryJdbcCustomLoggerTest covering concurrent FINEST/INFO/OFF connections, global and connection-scoped ResultSet facades, post-close OFF_VALUE short-circuiting under Level.INFO root, nested cross-connection MDC file output and restoration, child logger hasOwnHandlers, MdcThreadPoolExecutor submit/execute propagation, explicitGlobalLevel floor override, setUseParentHandlers(true) external sink visibility, and scanForExternalHandler fallback scanning.
  • Live GCP integration tests (IT*) were not executed against production BigQuery in this step.

Completes PR 3 of 3 in the diagnostic logging hardening stack.

Tracks external JUL handlers and parent-handler delegation on com.google.cloud.bigquery without per-call Handler[] allocations, propagates ConnectionLogContext through BigQueryJdbcMdc and BigQueryJdbcContextProxy, and short-circuits disabled or higher-threshold connection logs at Gate 1 (isLoggable) across BigQueryJdbcCustomLogger and BigQueryJdbcResultSetLogger.
@Neenu1995
Neenu1995 requested review from a team as code owners November 20, 2026 21:25
@Neenu1995
Neenu1995 added this pull request to stack #15115 November 20, 2026 21:26

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors the logging mechanism in the BigQuery JDBC driver to use ConnectionLogContext instead of raw connection IDs for MDC context propagation, improving loggability evaluation and isolation. The reviewer identified a potential issue in BigQueryJdbcResultSetLogger where log records might be dispatched with a null MDC context if threadContext is null but connectionContext is non-null, suggesting a simplified check. Additionally, the reviewer recommended renaming the getter explicitGlobalLevelValue() to getExplicitGlobalLevelValue() to avoid naming conflicts with the static field of the same name, along with updating its usage in BigQueryJdbcCustomLogger.

record.setParameters(newParams);
}
ConnectionLogContext threadContext = BigQueryJdbcMdc.getContext();
if (threadContext != null && threadContext != connectionContext) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

When threadContext is null but connectionContext is non-null, the MDC context is not registered before calling delegate.log(record). This means the log record will be dispatched with a null MDC context, which can cause routing or formatting issues if the downstream handlers or formatters rely on the MDC connection context.

We should simplify the condition to threadContext != connectionContext so that the context is also registered when the current thread has no active MDC context.

Suggested change
if (threadContext != null && threadContext != connectionContext) {
if (threadContext != connectionContext) {

Comment on lines +231 to +233
static int explicitGlobalLevelValue() {
return explicitGlobalLevelValue;
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The static method explicitGlobalLevelValue() has the exact same name as the static field explicitGlobalLevelValue. While this is technically valid in Java, it is highly confusing, violates standard Java getter naming conventions, and reduces code readability and maintainability.

Consider renaming the getter method to getExplicitGlobalLevelValue() to clearly distinguish it from the field.

Suggested change
static int explicitGlobalLevelValue() {
return explicitGlobalLevelValue;
}
static int getExplicitGlobalLevelValue() {
return explicitGlobalLevelValue;
}

if (rawThreshold == ConnectionLogContext.INHERIT_VALUE) {
return true;
}
int threshold = Math.min(rawThreshold, BigQueryJdbcRootLogger.explicitGlobalLevelValue());

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Update the call to use the renamed getter method getExplicitGlobalLevelValue() to maintain consistency and improve readability.

Suggested change
int threshold = Math.min(rawThreshold, BigQueryJdbcRootLogger.explicitGlobalLevelValue());
int threshold = Math.min(rawThreshold, BigQueryJdbcRootLogger.getExplicitGlobalLevelValue());

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant