Skip to content

fix(bigquery-jdbc): gate Storage Write API on direct VALUES inserts and align columns - #15108

Closed
Neenu1995 wants to merge 1 commit into
fix/swa-proto-3-type-parityfrom
fix/swa-proto-4-direct-insert-gating
Closed

Neenu1995 wants to merge 1 commit into
fix/swa-proto-3-type-parityfrom
fix/swa-proto-4-direct-insert-gating

Conversation

@Neenu1995

Copy link
Copy Markdown
Contributor

Stack: PR 4 of 5

Summary

  • Fix silent SQL expression/UDF bypass (b/587228731) and STRING -> BYTES transform crashes when EnableWriteAPI=true is used with SQL expressions, CASTs, or UDFs inside INSERT ... VALUES (...).
  • Add a zero-regex whitelist parser (parseDirectInsertValues) so only single-row INSERT [INTO] <table> [(<cols>)] VALUES (?, ...) [;] statements with bare ? placeholders use the Storage Write API; all other INSERT forms fall back automatically to BigQuery SQL DML.
  • Align reordered column lists (resolveInsertSchema), reject partial/duplicate/pseudo-column lists so DDL DEFAULT values execute on BigQuery's SQL engine, honor DefaultDataset vs. catalog project matching (resolveTargetProject), skip positional dry runs in Legacy SQL mode, retry unbound dry runs at most once when tables are created after prepareStatement(), and fall back to SQL inside active transactions or _SESSION temp tables.

Testing

  • make lint, mvn -B -ntp dependency:analyze-only -DfailOnWarning=true, mvn -q -Denforcer.skip=true -Dclirr.skip=true test-compile, and make unittest (parameterized SQL whitelist/rejection tests, MASK_UDF(?) b/587228731 & UDF(CAST(? AS STRING), ...) SQL fallback tests, reordered column ProtoRows round-trips, project/session/transaction gating, and persistent dry-run error non-retry tests; ITs require GCP credentials).

…nd align columns

Stack: PR 4 of 5 (base: `fix/swa-proto-3-type-parity`)

### Summary
- Fix silent SQL expression/UDF bypass (`b/587228731`) and `STRING -> BYTES` transform crashes when `EnableWriteAPI=true` is used with SQL expressions, `CAST`s, or UDFs inside `INSERT ... VALUES (...)`.
- Add a zero-regex whitelist parser (`parseDirectInsertValues`) so only single-row `INSERT [INTO] <table> [(<cols>)] VALUES (?, ...) [;]` statements with bare `?` placeholders use the Storage Write API; all other `INSERT` forms fall back automatically to BigQuery SQL DML.
- Align reordered column lists (`resolveInsertSchema`), reject partial/duplicate/pseudo-column lists so DDL `DEFAULT` values execute on BigQuery's SQL engine, honor `DefaultDataset` vs. `catalog` project matching (`resolveTargetProject`), skip positional dry runs in Legacy SQL mode, retry unbound dry runs at most once when tables are created after `prepareStatement()`, and fall back to SQL inside active transactions or `_SESSION` temp tables.

### Testing
- `make lint`, `mvn -B -ntp dependency:analyze-only -DfailOnWarning=true`, `mvn -q -Denforcer.skip=true -Dclirr.skip=true test-compile`, and `make unittest` (parameterized SQL whitelist/rejection tests, `MASK_UDF(?)` `b/587228731` & `UDF(CAST(? AS STRING), ...)` SQL fallback tests, reordered column `ProtoRows` round-trips, project/session/transaction gating, and persistent dry-run error non-retry tests; ITs require GCP credentials).
@Neenu1995
Neenu1995 requested review from a team as code owners November 20, 2026 20:26

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces support for direct single-table INSERT ... VALUES statements using the Storage Write API in BigQueryPreparedStatement. It adds SQL parsing logic to identify eligible queries, resolves target table schemas, and handles transaction and session constraints, backed by comprehensive unit tests. The review feedback highlights a potential bypass in consumeTablePath where _session temporary tables are not fully blocked when using three-segment paths (e.g., project._session.table), and provides a code suggestion to robustly reject any paths containing _session segments.

Comment on lines +393 to +431
private static int consumeTablePath(String sql, int pos) {
Tuple<String, Integer> firstSegment = readTableSegment(sql, pos);
if (firstSegment == null) {
return -1;
}
String firstName = firstSegment.x();
pos = firstSegment.y();
int segmentCount = 1;
while (true) {
int next = skipTrivia(sql, pos);
if (next < 0) {
return -1;
}
if (next < sql.length() && (sql.charAt(next) == '.' || sql.charAt(next) == ':')) {
pos = skipTrivia(sql, next + 1);
if (pos < 0) {
return -1;
}
Tuple<String, Integer> segment = readTableSegment(sql, pos);
if (segment == null) {
return -1;
}
pos = segment.y();
segmentCount++;
} else {
pos = next;
break;
}
}
if (segmentCount == 2 && "_session".equalsIgnoreCase(firstName)) {
return -1;
}
if (segmentCount == 1
&& firstName.length() > 9
&& firstName.regionMatches(true, 0, "_session.", 0, 9)) {
return -1;
}
return pos;
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The current implementation of consumeTablePath only checks if _session is the first segment of a 2-segment path, or if a 1-segment path starts with _session.. However, if a 3-segment path is used (e.g., project._session.table or backticked `project._session.table`), the check is bypassed because segmentCount is 3 and firstName is project. Since the Storage Write API does not support writing to session-specific temporary tables, we should reject any table path where any segment is _session (case-insensitive) or contains _session as a dataset prefix.

  private static int consumeTablePath(String sql, int pos) {
    Tuple<String, Integer> firstSegment = readTableSegment(sql, pos);
    if (firstSegment == null) {
      return -1;
    }
    String firstName = firstSegment.x();
    pos = firstSegment.y();
    List<String> segments = new ArrayList<>();
    segments.add(firstName);
    while (true) {
      int next = skipTrivia(sql, pos);
      if (next < 0) {
        return -1;
      }
      if (next < sql.length() && (sql.charAt(next) == '.' || sql.charAt(next) == ':')) {
        pos = skipTrivia(sql, next + 1);
        if (pos < 0) {
          return -1;
        }
        Tuple<String, Integer> segment = readTableSegment(sql, pos);
        if (segment == null) {
          return -1;
        }
        pos = segment.y();
        segments.add(segment.x());
      } else {
        pos = next;
        break;
      }
    }
    for (String seg : segments) {
      String lower = seg.toLowerCase(Locale.ROOT);
      if ("_session".equals(lower) || lower.startsWith("_session.") || lower.contains("._session.")) {
        return -1;
      }
    }
    return pos;
  }

@Neenu1995
Neenu1995 added this pull request to stack #15111 November 20, 2026 20:31
@Neenu1995
Neenu1995 deleted the branch fix/swa-proto-3-type-parity November 20, 2026 23:27
@Neenu1995 Neenu1995 closed this Nov 20, 2026
@Neenu1995
Neenu1995 deleted the fix/swa-proto-4-direct-insert-gating branch November 20, 2026 23:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant