Skip to content

feat(duckdb): Add transpilation support for STRTOK_TO_ARRAY function#7536

Open
fivetran-amrutabhimsenayachit wants to merge 1 commit intomainfrom
RD-1147775-strtok-to-array
Open

feat(duckdb): Add transpilation support for STRTOK_TO_ARRAY function#7536
fivetran-amrutabhimsenayachit wants to merge 1 commit intomainfrom
RD-1147775-strtok-to-array

Conversation

@fivetran-amrutabhimsenayachit
Copy link
Copy Markdown
Collaborator

Snowflake's STRTOK_TO_ARRAY(str, delim) splits a string on any character in delim (treating it as a character class), skips empty tokens, defaults to space when delimiter is omitted, and returns NULL when either argument is NULL.

DuckDB has no equivalent — a simple rename to STRING_TO_ARRAY fails because DuckDB treats the delimiter as a literal string, keeps empty tokens, errors on a missing delimiter, returns the original string for NULL delimiter, and crashes on an empty delimiter ([] is invalid regex).

@georgesittas
Copy link
Copy Markdown
Collaborator

@fivetran-amrutabhimsenayachit can you please rebase off of main and resolve conflicts?

@github-actions
Copy link
Copy Markdown
Contributor

SQLGlot Integration Test Results

Comparing:

  • this branch (sqlglot:RD-1147775-strtok-to-array, sqlglot version: RD-1147775-strtok-to-array)
  • baseline (main, sqlglot version: 0.0.1.dev1)

By Dialect

dialect main sqlglot:RD-1147775-strtok-to-array transitions links
bigquery -> bigquery 24645/24650 passed (100.0%) 23495/23495 passed (100.0%) No change full result / delta
bigquery -> duckdb 867/1154 passed (75.1%) 0/0 passed (0.0%) Results not found full result / delta
duckdb -> duckdb 5823/5823 passed (100.0%) 5823/5823 passed (100.0%) No change full result / delta
snowflake -> duckdb 1063/1961 passed (54.2%) 0/0 passed (0.0%) Results not found full result / delta
snowflake -> snowflake 65133/65133 passed (100.0%) 63027/63027 passed (100.0%) No change full result / delta
databricks -> databricks 1370/1370 passed (100.0%) 1370/1370 passed (100.0%) No change full result / delta
postgres -> postgres 6042/6042 passed (100.0%) 6042/6042 passed (100.0%) No change full result / delta
redshift -> redshift 7101/7101 passed (100.0%) 7101/7101 passed (100.0%) No change full result / delta

Overall

main: 113234 total, 112044 passed (pass rate: 98.9%), sqlglot version: 0.0.1.dev1

sqlglot:RD-1147775-strtok-to-array: 106858 total, 106858 passed (pass rate: 100.0%), sqlglot version: RD-1147775-strtok-to-array

Transitions:
No change

Dialect pair changes: 0 previous results not found, 2 current results not found

✅ 36 test(s) passed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants