Skip to content

fix(transpile-bug): update core, generator, optimizer, and parser files for select-star-ilike-rename#7720

Open
fivetran-amrutabhimsenayachit wants to merge 1 commit into
mainfrom
RD-1205926-transpile-bug-select-star-ilike-rename
Open

fix(transpile-bug): update core, generator, optimizer, and parser files for select-star-ilike-rename#7720
fivetran-amrutabhimsenayachit wants to merge 1 commit into
mainfrom
RD-1205926-transpile-bug-select-star-ilike-rename

Conversation

@fivetran-amrutabhimsenayachit

Copy link
Copy Markdown
Collaborator

Before:

python3 -c "import sqlglot; print(sqlglot.transpile(\"S
ELECT * ILIKE '%id%' RENAME department_id AS department FROM employee_table\", read='sn
owflake', write='duckdb')[0])"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
    import sqlglot; print(sqlglot.transpile("SELECT * ILIKE '%id%' RENAME department_id AS department FROM employee_table", read='snowflake', write='duckdb')[0])
                          ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/amruta.ayachit/Tobiko/sqlglot/sqlglot/__init__.py", line 196, in transpile
    for expression in parse(sql, read, error_level=error_level)
                      ~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/amruta.ayachit/Tobiko/sqlglot/sqlglot/__init__.py", line 110, in parse
    return Dialect.get_or_raise(read or dialect).parse(sql, **opts)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^
  File "/Users/amruta.ayachit/Tobiko/sqlglot/sqlglot/dialects/dialect.py", line 1124, in parse
    return self.parser(**opts).parse(self.tokenize(sql), sql)
           ~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/amruta.ayachit/Tobiko/sqlglot/sqlglot/parser.py", line 2041, in parse
    return self._parse(
           ~~~~~~~~~~~^
        parse_method=self.__class__._parse_statement, raw_tokens=raw_tokens, sql=sql
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/Users/amruta.ayachit/Tobiko/sqlglot/sqlglot/parser.py", line 2161, in _parse
    return self._parse_batch_statements(parse_method=parse_method, sep_first_statement=False)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/amruta.ayachit/Tobiko/sqlglot/sqlglot/parser.py", line 2131, in _parse_batch_statements
    self.raise_error("Invalid expression / Unexpected token")
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/amruta.ayachit/Tobiko/sqlglot/sqlglot/parser.py", line 1998, in raise_error
    raise error
sqlglot.errors.ParseError: Invalid expression / Unexpected token. Line 1, Col: 42.
  SELECT * ILIKE '%id%' RENAME department_id AS department FROM employee_table
  
  
 python3 -c "import sqlglot; print(repr(sqlglot.parse_one(\"SELECT * ILIKE '%id%' FROM employee_table\", dialect='snowflake')))"

Select(
  expressions=[
    ILike(
      this=Star(),
      expression=Literal(this='%id%', is_string=True))],
  from_=From(
    this=Table(
      this=Identifier(this=employee_table, quoted=False))))

After:

python3 -c "import sqlglot; print(sqlglot.transpile(\"SELECT * ILIKE '%id%' RENAME department_id AS department FROM employee_table\", read='snowflake', write='duckdb')[0])"
SELECT * ILIKE '%id%' RENAME (department_id AS department) FROM employee_table

python3 -c "import sqlglot; print(repr(sqlglot.parse_one(\"SELECT * ILIKE '%id%' FROM employee_table\", dialect='snowflake')))"

Select(
  expressions=[
    Star(
      ilike=Literal(this='%id%', is_string=True))],
  from_=From(
    this=Table(
      this=Identifier(this=employee_table, quoted=False))))

Comment on lines +892 to +893
for table in tables:
ilike_columns[id(table)] = pattern

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we repeat this pattern N times, one for every table? Just one suffices right?

Comment thread sqlglot/generator.py
Comment on lines -3366 to +3368
return f"*{except_}{replace}{rename}"
ilike = self.sql(expression, "ilike")
ilike = f" ILIKE {ilike}" if ilike else ""
return f"*{ilike}{except_}{replace}{rename}"

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What does generation look like when pretty=True? Since we made the effort of pretty-formatting the other clauses (RENAME, etc), let's do it for this one as well so that the output is consistent.

Comment thread sqlglot/parser.py
this.set("unpack", True)
return this

ilike = self._parse_primary() if self._match(TokenType.ILIKE) else None

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are we using _parse_primary() here? Should this be _parse_string()? What if the user supplies a placeholder / parameter, is that handled?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants