For ParquetVariant (and potentially other types), the array's metadata is required to accurately provide an Arrow Field. The physical layout of the array itself influences the Arrow-side data type.
Parquet solves this in two ways:
- Storing the schema as file metadata.
- Parquet is structured as a tree of types, so Variant columns have children that can be directly inspected to infer the schema correctly.
For ParquetVariant (and potentially other types), the array's metadata is required to accurately provide an Arrow
Field. The physical layout of the array itself influences the Arrow-side data type.Parquet solves this in two ways: