Skip to content

fix(bigquery-jdbc): redact URL property logs and isolate per-connection logging - #15113

Open
Neenu1995 wants to merge 1 commit into
fix/bq-jdbc-log-file-hardeningfrom
fix/bq-jdbc-per-conn-log-isolation
Open

Neenu1995 wants to merge 1 commit into
fix/bq-jdbc-log-file-hardeningfrom
fix/bq-jdbc-per-conn-log-isolation

Conversation

@Neenu1995

Copy link
Copy Markdown
Contributor

What & Why

PR 2 of 3 in the diagnostic logging hardening stack (stacked on fix/bq-jdbc-log-file-hardening).

Previously, connecting via BigQueryDriver.connect or DataSource.getConnection had two multi-tenant isolation and redaction issues:

  1. Unredacted credential logging prior to MDC binding: BigQueryJdbcUrlUtility.appendPropertiesToURL logged Appending %s with value %s to URL at Level.FINEST without masking sensitive properties (OAuthAccessToken, OAuthRefreshToken, OAuthPvtKey, OAuthClientSecret, ProxyPwd, gcpTelemetryCredentials, etc.) and before any connection ID was bound in BigQueryJdbcMdc. In addition, BYOID_CredentialSource (which can carry JSON HTTP authentication headers in Workload Identity Federation) was listed in BigQueryConnection.SAFE_TO_LOG_PROPERTIES.
  2. JVM-global logger clobbering & handler stripping: Every BigQueryDriver.connect call mutated the JVM-wide com.google.cloud.bigquery logger level and replaced the singleton PerConnectionFileHandler, redirecting concurrent connections' SQL and property logs into the latest caller's LogPath — while any connection with default LogLevel=0 closed and removed all JUL handlers on com.google.cloud.bigquery (including OpenTelemetryJulHandler and active PerConnectionFileHandler instances) and set the global logger level to Level.OFF.

Changes in this PR

  • Shared allowlist secret redaction: Removes BYOID_CredentialSource from SAFE_TO_LOG_PROPERTIES, introduces BigQueryConnection.maskPropertyValue(key, value), and applies it inside BigQueryJdbcUrlUtility.appendPropertiesToURL (with null-safe / non-String-safe Properties handling and lazy Supplier<String> evaluation at FINEST).
  • Per-connection (LogLevel, LogPath) isolation & handler preservation:
    • Allocates connectionId and binds reentrant BigQueryJdbcMdc at the entry of BigQueryDriver.connect, passing connectionId into BigQueryConnection and cleaning up registration on connection setup failure.
    • Attaches a single persistent PerConnectionFileHandler storing per-connectionId (Path logPath, Level level) slots (BQ-JDBC-GLOBAL.log is opened only when JVM-level BIGQUERY_JDBC_LOG_PATH and BIGQUERY_JDBC_LOG_LEVEL environment variables are configured).
    • Never strips OpenTelemetryJulHandler, ConsoleHandler, or external JUL handlers when a connection specifies LogLevel=0, and dynamically computes the root logger threshold as the minimum (most verbose) active level across open connections (plus externally attached JUL handlers).
  • Multi-tenant LogPath guardrail: Adds -Dcom.google.cloud.bigquery.jdbc.allowUrlLogPath (default true; when false, ignores URL/Properties-supplied LogPath and skips the CWD default fallback so only admin-configured BIGQUERY_JDBC_LOG_PATH writes local files).

How It Was Tested

  • JAVA_HOME=/usr/lib/jvm/java-17-openjdk-amd64 make lint (fmt-maven-plugin:format).
  • JAVA_HOME=/usr/lib/jvm/java-17-openjdk-amd64 make unittest (Checkstyle, Java 8 animal-sniffer, and the full JUnit 5 unit test suite) and integration test compilation.
  • Added JUnit 5 regression tests in BigQueryJdbcUrlUtilityTest, BigQueryConnectionTest, BigQueryDriverTest, BigQueryJdbcMdcTest, and PerConnectionFileHandlerTest covering parameterized secret masking, concurrent multi-tenant LogPath and LogLevel isolation, OpenTelemetryJulHandler preservation, external floor restoration, and -Dcom.google.cloud.bigquery.jdbc.allowUrlLogPath=false enforcement.
  • Live GCP integration tests (IT*) were not executed against production BigQuery in this step.

…on logging

Resolves b/584840083 (PR 2 of 3).

Redacts sensitive properties (including BYOID_CredentialSource) via SAFE_TO_LOG_PROPERTIES in BigQueryJdbcUrlUtility.appendPropertiesToURL, isolates per-connection LogLevel and LogPath inside a single persistent PerConnectionFileHandler without stripping OpenTelemetryJulHandler or external JUL handlers on LogLevel=0, makes BigQueryJdbcMdc.registerInstance reentrant, and adds -Dcom.google.cloud.bigquery.jdbc.allowUrlLogPath for multi-tenant deployments.
@Neenu1995
Neenu1995 requested review from a team as code owners November 20, 2026 21:25
@Neenu1995
Neenu1995 added this pull request to stack #15115 November 20, 2026 21:26

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces multi-tenant logging isolation and secure log path configuration for the BigQuery JDBC driver, enabling connection-specific logging levels and paths while masking sensitive properties. Key feedback includes addressing potential issues where external log level configurations are ignored, avoiding disabling parent handler propagation, simplifying a redundant ternary operator in PerConnectionFileHandler, and wrapping system property retrieval in a try-catch block to prevent SecurityException in restricted environments.

Comment on lines +209 to +215
if (hasExternalHandlerAttached()) {
if (current != lastDriverAppliedLevel) {
externalFloorLevel = current;
}
} else {
for (Handler h : logger.getHandlers()) {
h.close();
logger.removeHandler(h);
externalFloorLevel = null;
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

By restricting the detection of external log level changes to only when hasExternalHandlerAttached() is true, any external log level configuration set on the com.google.cloud.bigquery logger (e.g., via logging.properties or Logger.setLevel(...) without directly attaching a custom handler to this specific logger) will be ignored and overwritten by the driver's dynamic level adjustment. In standard JUL usage, handlers are typically attached to the root logger, not package-specific loggers. Consider removing the hasExternalHandlerAttached() guard or finding a way to preserve externally configured levels even when no direct handlers are attached.

}
if (!attached) {
logger.addHandler(persistentFileHandler);
logger.setUseParentHandlers(false);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Setting logger.setUseParentHandlers(false) on the com.google.cloud.bigquery logger will prevent log messages from propagating to parent loggers. In global static utilities shared across multiple concurrent sessions, avoid state changes (such as closing or removing shared handlers) triggered by a single session's configuration if those changes would negatively affect other active sessions.

References
  1. In global static utilities shared across multiple concurrent sessions, avoid state changes (such as closing or removing shared handlers) triggered by a single session's configuration if those changes would negatively affect other active sessions.

Comment on lines +107 to +108
Exception ex = (e instanceof Exception) ? (Exception) e : new IOException(e);
reportError("Failed to initialize env default log file", ex, ErrorManager.OPEN_FAILURE);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Since e is caught as RuntimeException | IOException, both of which are subclasses of Exception, e is guaranteed to be an instance of Exception. The ternary operator (e instanceof Exception) ? (Exception) e : new IOException(e) is redundant and can be simplified by passing e directly to reportError.

Suggested change
Exception ex = (e instanceof Exception) ? (Exception) e : new IOException(e);
reportError("Failed to initialize env default log file", ex, ErrorManager.OPEN_FAILURE);
reportError("Failed to initialize env default log file", e, ErrorManager.OPEN_FAILURE);

Comment on lines +857 to +860
static boolean isUrlLogPathAllowed() {
String prop = System.getProperty(ALLOW_URL_LOG_PATH_SYSTEM_PROPERTY);
return prop == null || !"false".equalsIgnoreCase(prop.trim());
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Calling System.getProperty can throw a SecurityException if run in a restricted environment (such as an application server with a security manager enabled). Consider wrapping this call in a try-catch block to gracefully handle SecurityException and fall back to a default value (e.g., true).

  static boolean isUrlLogPathAllowed() {
    try {
      String prop = System.getProperty(ALLOW_URL_LOG_PATH_SYSTEM_PROPERTY);
      return prop == null || !"false".equalsIgnoreCase(prop.trim());
    } catch (SecurityException e) {
      return true;
    }
  }

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant