Repository navigation
Conversation
…nd align columns Stack: PR 4 of 5 (base: `fix/swa-proto-3-type-parity`) ### Summary - Fix silent SQL expression/UDF bypass (`b/587228731`) and `STRING -> BYTES` transform crashes when `EnableWriteAPI=true` is used with SQL expressions, `CAST`s, or UDFs inside `INSERT ... VALUES (...)`. - Add a zero-regex whitelist parser (`parseDirectInsertValues`) so only single-row `INSERT [INTO] <table> [(<cols>)] VALUES (?, ...) [;]` statements with bare `?` placeholders use the Storage Write API; all other `INSERT` forms fall back automatically to BigQuery SQL DML. - Align reordered column lists (`resolveInsertSchema`), reject partial/duplicate/pseudo-column lists so DDL `DEFAULT` values execute on BigQuery's SQL engine, honor `DefaultDataset` vs. `catalog` project matching (`resolveTargetProject`), skip positional dry runs in Legacy SQL mode, retry unbound dry runs at most once when tables are created after `prepareStatement()`, and fall back to SQL inside active transactions or `_SESSION` temp tables. ### Testing - `make lint`, `mvn -B -ntp dependency:analyze-only -DfailOnWarning=true`, `mvn -q -Denforcer.skip=true -Dclirr.skip=true test-compile`, and `make unittest` (parameterized SQL whitelist/rejection tests, `MASK_UDF(?)` `b/587228731` & `UDF(CAST(? AS STRING), ...)` SQL fallback tests, reordered column `ProtoRows` round-trips, project/session/transaction gating, and persistent dry-run error non-retry tests; ITs require GCP credentials).
There was a problem hiding this comment.
Code Review
This pull request introduces support for direct single-table INSERT ... VALUES statements using the Storage Write API in BigQueryPreparedStatement. It adds SQL parsing logic to identify eligible queries, resolves target table schemas, and handles transaction and session constraints, backed by comprehensive unit tests. The review feedback highlights a potential bypass in consumeTablePath where _session temporary tables are not fully blocked when using three-segment paths (e.g., project._session.table), and provides a code suggestion to robustly reject any paths containing _session segments.
| private static int consumeTablePath(String sql, int pos) { | ||
| Tuple<String, Integer> firstSegment = readTableSegment(sql, pos); | ||
| if (firstSegment == null) { | ||
| return -1; | ||
| } | ||
| String firstName = firstSegment.x(); | ||
| pos = firstSegment.y(); | ||
| int segmentCount = 1; | ||
| while (true) { | ||
| int next = skipTrivia(sql, pos); | ||
| if (next < 0) { | ||
| return -1; | ||
| } | ||
| if (next < sql.length() && (sql.charAt(next) == '.' || sql.charAt(next) == ':')) { | ||
| pos = skipTrivia(sql, next + 1); | ||
| if (pos < 0) { | ||
| return -1; | ||
| } | ||
| Tuple<String, Integer> segment = readTableSegment(sql, pos); | ||
| if (segment == null) { | ||
| return -1; | ||
| } | ||
| pos = segment.y(); | ||
| segmentCount++; | ||
| } else { | ||
| pos = next; | ||
| break; | ||
| } | ||
| } | ||
| if (segmentCount == 2 && "_session".equalsIgnoreCase(firstName)) { | ||
| return -1; | ||
| } | ||
| if (segmentCount == 1 | ||
| && firstName.length() > 9 | ||
| && firstName.regionMatches(true, 0, "_session.", 0, 9)) { | ||
| return -1; | ||
| } | ||
| return pos; | ||
| } |
There was a problem hiding this comment.
The current implementation of consumeTablePath only checks if _session is the first segment of a 2-segment path, or if a 1-segment path starts with _session.. However, if a 3-segment path is used (e.g., project._session.table or backticked `project._session.table`), the check is bypassed because segmentCount is 3 and firstName is project. Since the Storage Write API does not support writing to session-specific temporary tables, we should reject any table path where any segment is _session (case-insensitive) or contains _session as a dataset prefix.
private static int consumeTablePath(String sql, int pos) {
Tuple<String, Integer> firstSegment = readTableSegment(sql, pos);
if (firstSegment == null) {
return -1;
}
String firstName = firstSegment.x();
pos = firstSegment.y();
List<String> segments = new ArrayList<>();
segments.add(firstName);
while (true) {
int next = skipTrivia(sql, pos);
if (next < 0) {
return -1;
}
if (next < sql.length() && (sql.charAt(next) == '.' || sql.charAt(next) == ':')) {
pos = skipTrivia(sql, next + 1);
if (pos < 0) {
return -1;
}
Tuple<String, Integer> segment = readTableSegment(sql, pos);
if (segment == null) {
return -1;
}
pos = segment.y();
segments.add(segment.x());
} else {
pos = next;
break;
}
}
for (String seg : segments) {
String lower = seg.toLowerCase(Locale.ROOT);
if ("_session".equals(lower) || lower.startsWith("_session.") || lower.contains("._session.")) {
return -1;
}
}
return pos;
}
Stack: PR 4 of 5
Summary
b/587228731) andSTRING -> BYTEStransform crashes whenEnableWriteAPI=trueis used with SQL expressions,CASTs, or UDFs insideINSERT ... VALUES (...).parseDirectInsertValues) so only single-rowINSERT [INTO] <table> [(<cols>)] VALUES (?, ...) [;]statements with bare?placeholders use the Storage Write API; all otherINSERTforms fall back automatically to BigQuery SQL DML.resolveInsertSchema), reject partial/duplicate/pseudo-column lists so DDLDEFAULTvalues execute on BigQuery's SQL engine, honorDefaultDatasetvs.catalogproject matching (resolveTargetProject), skip positional dry runs in Legacy SQL mode, retry unbound dry runs at most once when tables are created afterprepareStatement(), and fall back to SQL inside active transactions or_SESSIONtemp tables.Testing
make lint,mvn -B -ntp dependency:analyze-only -DfailOnWarning=true,mvn -q -Denforcer.skip=true -Dclirr.skip=true test-compile, andmake unittest(parameterized SQL whitelist/rejection tests,MASK_UDF(?)b/587228731&UDF(CAST(? AS STRING), ...)SQL fallback tests, reordered columnProtoRowsround-trips, project/session/transaction gating, and persistent dry-run error non-retry tests; ITs require GCP credentials).