refactor: Enable SELECT * optimization when compiling read-table nodes into sqlglot
#2430
Conversation
product-auto-label
bot
added
size: m
chelsea-lin
force-pushed
the
main_chelsealin_selectstart
branch
from
e085050 to
f5a2276
Compare
| nodes.ScanItem( | ||
| identifiers.ColumnId(scan_item.source_id), scan_item.source_id | ||
| ) | ||
| for scan_item in node.scan_list.items |
There was a problem hiding this comment.
I think what we really want is to order by the underlying physical schema?
There was a problem hiding this comment.
When generating SQL, we should respect the original order of the selected columns and avoid reordering them to match the physical schema. While reordering can reveal more SELECT * optimizations when the query involves intermediate CTEs or subqueries, the current logic sorts the scan_list by the algebraic ordering of column names. This approach hides potential SELECT * optimizations, particularly in use cases like bpd.read_table("table_name")
bigframes/core/nodes.py
Outdated
| @property | ||
| def is_star_selection(self) -> bool: | ||
| physical_names = tuple(item.name for item in self.source.table.physical_schema) | ||
| scan_names = tuple(item.source_id for item in self.scan_list.items) |
There was a problem hiding this comment.
the scan list item id must equal source_id as well for this to work as intended
There was a problem hiding this comment.
Good points. It should be addressed in the new commit. Thanks.
chelsea-lin
force-pushed
the
main_chelsealin_selectstart
branch
from
f5a2276 to
15dad78
Compare
3 participants
Fixes internal issue 481740136 🦕