From 0106022c6163c02c313079d257936851e9bb519c Mon Sep 17 00:00:00 2001 From: Kat Batuigas Date: Wed, 13 May 2026 22:14:48 -0700 Subject: [PATCH 1/9] Draft struct support reference --- .../pages/sql/sql-data-types/row.adoc | 146 +++++++++++++++++- 1 file changed, 145 insertions(+), 1 deletion(-) diff --git a/modules/reference/pages/sql/sql-data-types/row.adoc b/modules/reference/pages/sql/sql-data-types/row.adoc index bc55a2dac..453c24dc0 100644 --- a/modules/reference/pages/sql/sql-data-types/row.adoc +++ b/modules/reference/pages/sql/sql-data-types/row.adoc @@ -2,7 +2,7 @@ :description: The ROW data type represents a composite value containing one or more fields of different types. :page-topic-type: reference -The `ROW` data type represents a composite value (also known as a struct or record) containing one or more fields of different types. +The `ROW` data type represents a composite value (also known as a struct or record) containing one or more fields of different types. ROW values support field access, lexicographic comparison, NULL checks, conversion to text, and use in `GROUP BY`, `ORDER BY`, and `JOIN` clauses. == Syntax @@ -75,3 +75,147 @@ SELECT ROW(); () (1 row) ---- + +== Access fields + +=== Access by position + +For anonymous ROW expressions, fields are accessed by the positional names `f1`, `f2`, and so on, in declaration order: + +[source,sql] +---- +SELECT (ROW(1, 'hello', 3.14)).f1, (ROW(1, 'hello', 3.14)).f2; +---- + +[source,sql] +---- + f1 | f2 +----+------- + 1 | hello +(1 row) +---- + +The parentheses around the ROW expression are required when accessing a field. + +=== Access by name + +For composite columns with declared field names — for example, columns mapped from a topic with `struct_mapping_policy = 'COMPOUND'` (see xref:reference:sql/sql-statements/create-table.adoc[CREATE TABLE]) — access fields by their declared names: + +[source,sql] +---- +SELECT (record).customer_id, (record).order_total FROM orders; +---- + +=== Expand all fields with a wildcard + +To project every field of a ROW value as a separate result column, use the `.*` form: + +[source,sql] +---- +SELECT (ROW(1, 'hello', 3.14)).*; +---- + +[source,sql] +---- + f1 | f2 | f3 +----+-------+------ + 1 | hello | 3.14 +(1 row) +---- + +The wildcard form also works inside a `ROW(...)` constructor to copy fields from one composite into another. + +== Compare ROW values + +ROW values support the standard comparison operators `=`, `<>`, `<`, `\<=`, `>`, and `>=`. Comparison is *lexicographic*: fields are compared in order, left to right, and the first differing field determines the result. + +[source,sql] +---- +SELECT ROW(1, 'a') < ROW(1, 'b'); +---- + +[source,sql] +---- + ?column? +---------- + t +(1 row) +---- + +Both ROW values must have the same number of fields, and corresponding fields must have comparable types. + +== Check for NULL + +ROW values support `IS NULL` and `IS NOT NULL`. + +[source,sql] +---- +SELECT ROW(1, 'a') IS NULL; +---- + +[source,sql] +---- + ?column? +---------- + f +(1 row) +---- + +To check whether a specific field within a ROW is NULL, access the field directly and test that. + +== Convert to text + +Cast a ROW value to `text` to produce the standard PostgreSQL composite literal form: + +[source,sql] +---- +SELECT ROW(1, 'hello', 3.14)::text; +---- + +[source,sql] +---- + row +----------------- + (1,"hello",3.14) +(1 row) +---- + +== Use ROW in queries + +ROW values can be used in `GROUP BY`, `ORDER BY`, and `JOIN` clauses with lexicographic comparison semantics. + +=== Group by a ROW field + +[source,sql] +---- +SELECT (customer).region, COUNT(*) +FROM orders +GROUP BY (customer).region; +---- + +=== Order by a whole ROW + +[source,sql] +---- +SELECT * FROM orders ORDER BY customer; +---- + +The rows are sorted lexicographically by the fields of the `customer` composite column, in their declared order. + +=== Join on a multi-column key + +Compare implicit tuples to match multi-column keys without spelling out each field in a `WHERE` clause: + +[source,sql] +---- +SELECT * +FROM table_a a +JOIN table_b b +ON (a.col1, a.col2) = (b.col1, b.col2); +---- + +// TODO: SME — confirm whether nested array-of-struct access (for example, `(arr_of_rows[1]).field_name`) works at GA, and whether wildcard expansion on an empty ROW (`(ROW()).*`) is supported. Both are tracked under OXLA-9444 and OXLA-9431 respectively and remain open as of 2026-05-13. + +== See also + +* xref:reference:sql/sql-statements/create-table.adoc[CREATE TABLE]: maps a Redpanda topic to a SQL table. Use `struct_mapping_policy = 'COMPOUND'` to surface nested topic fields as ROW columns. From 52d1b2cfdb11970e809d5cdf0ee01912ec28cc3a Mon Sep 17 00:00:00 2001 From: Kat Batuigas Date: Wed, 13 May 2026 22:18:16 -0700 Subject: [PATCH 2/9] How to query structs/nested fields --- modules/ROOT/nav.adoc | 1 + .../pages/query-data/query-nested-fields.adoc | 134 ++++++++++++++++++ 2 files changed, 135 insertions(+) create mode 100644 modules/sql/pages/query-data/query-nested-fields.adoc diff --git a/modules/ROOT/nav.adoc b/modules/ROOT/nav.adoc index 4d1521d6f..416cb430a 100644 --- a/modules/ROOT/nav.adoc +++ b/modules/ROOT/nav.adoc @@ -356,6 +356,7 @@ *** xref:sql:query-data/redpanda-catalogs.adoc[Redpanda Catalogs] *** xref:sql:query-data/query-streaming-topics.adoc[Query Streaming Topics] *** xref:sql:query-data/query-iceberg-topics.adoc[Query Iceberg-enabled Topics] +*** xref:sql:query-data/query-nested-fields.adoc[Query Topics with Nested Fields] ** xref:sql:manage/index.adoc[Manage Redpanda SQL] *** xref:sql:manage/manage-access.adoc[Manage access] ** xref:sql:troubleshoot/index.adoc[Troubleshoot] diff --git a/modules/sql/pages/query-data/query-nested-fields.adoc b/modules/sql/pages/query-data/query-nested-fields.adoc new file mode 100644 index 000000000..e5fe753a1 --- /dev/null +++ b/modules/sql/pages/query-data/query-nested-fields.adoc @@ -0,0 +1,134 @@ += Query topics with nested fields +:description: Map a topic with nested protobuf or Avro fields to SQL ROW columns, then query those fields directly. +:page-topic-type: how-to +:personas: app_developer, data_engineer +:learning-objective-1: Map a topic with a nested schema as a SQL table using struct_mapping_policy = 'COMPOUND' +:learning-objective-2: Query nested fields using ROW field-access syntax +:learning-objective-3: Recognize and resolve cyclic-reference errors + +When a topic schema includes nested protobuf or Avro message types, you can map those nested structures as SQL `ROW` columns instead of opaque JSON. This makes nested fields queryable by name, includable in projections, and usable in `WHERE`, `GROUP BY`, and `ORDER BY` clauses, without parsing JSON at query time. + +After completing these steps, you will be able to: + +* [ ] {learning-objective-1} +* [ ] {learning-objective-2} +* [ ] {learning-objective-3} + +== Prerequisites + +Before you query a topic with nested fields: + +* Enable Redpanda SQL on your Redpanda Bring Your Own Cloud (BYOC) cluster. See xref:sql:get-started/deploy-sql-cluster.adoc[Enable Redpanda SQL]. +* Connect to Redpanda SQL with `psql` or another PostgreSQL client. See xref:sql:connect-to-sql/index.adoc[Connect to Redpanda SQL]. +* The topic has a schema (Protobuf or Avro) registered in Schema Registry. The schema includes one or more nested message types. +* You have a Redpanda catalog connection. See xref:reference:sql/sql-statements/create-redpanda-catalog.adoc[CREATE REDPANDA CATALOG]. + +== Map the topic as a SQL table + +Create the SQL table with `struct_mapping_policy = 'COMPOUND'` to surface each nested message as a SQL `ROW` column: + +[source,sql] +---- +CREATE TABLE default_redpanda_catalog=>orders WITH ( + topic = 'orders', + schema_subject = 'orders-value', + struct_mapping_policy = 'COMPOUND' +); +---- + +Replace `orders` with your topic name and `orders-value` with the Schema Registry subject that holds the topic's value schema. + +For a topic schema with this Protobuf definition: + +[source,proto] +---- +message Order { + string order_id = 1; + Customer customer = 2; + double amount = 3; +} + +message Customer { + string customer_id = 1; + string name = 2; + string region = 3; +} +---- + +Redpanda SQL maps the table with three columns: `order_id` (text), `customer` (a `ROW` with fields `customer_id`, `name`, and `region`), and `amount` (double precision). + +TIP: To map nested structures as JSON instead, use `struct_mapping_policy = 'JSON'`. `JSON` is the default, and it is the only option that supports recursive (cyclic) types. See <>. + +== Query nested fields + +Access a nested field by its declared name using the `(column).field` form. The parentheses around the column are required: + +[source,sql] +---- +SELECT order_id, (customer).name, (customer).region, amount +FROM default_redpanda_catalog=>orders +WHERE (customer).region = 'EMEA'; +---- + +To project every field of a nested structure as separate result columns, use the wildcard `.*` form: + +[source,sql] +---- +SELECT order_id, (customer).* +FROM default_redpanda_catalog=>orders +LIMIT 10; +---- + +For schemas with multiple levels of nesting, chain the parenthesized field access: + +[source,sql] +---- +SELECT ((order_metadata).customer).customer_id FROM default_redpanda_catalog=>orders; +---- + +For the full `ROW` reference, including comparison operators, NULL handling, and `::text` casting, see xref:reference:sql/sql-data-types/row.adoc[ROW]. + +[[handle-recursive-cyclic-schemas]] +== Handle recursive (cyclic) schemas + +If your topic schema includes a recursive structure — for example, a `Comment` message that references itself, or two messages that reference each other — mapping the table with `COMPOUND` fails at table-creation time with the following error: + +[source,text] +---- +Cyclic reference at '.' → ''. Cyclic types are not supported in COMPOUND struct mapping policy; use struct_mapping_policy=JSON for recursive types. +---- + +The error message tells you the resolution: re-create the table with `struct_mapping_policy = 'JSON'`. In JSON mode, Redpanda SQL stores each nested structure as a JSON value: + +[source,sql] +---- +CREATE TABLE default_redpanda_catalog=>comments WITH ( + topic = 'comments', + schema_subject = 'comments-value', + struct_mapping_policy = 'JSON' +); +---- + +Query JSON-mapped fields with standard JSON functions instead of ROW field access. See xref:reference:sql/sql-data-types/json.adoc[JSON]. + +== Choose between COMPOUND and JSON + +[cols="<20%,<40%,<40%",options="header"] +|=== +| Policy | Use when | Trade-offs + +| `COMPOUND` +| The topic schema has nested structures that are not recursive, and you want to query nested fields directly by name. +| Typed access; usable in `WHERE`, `GROUP BY`, `ORDER BY`. Required if you also plan to run xref:sql:query-data/query-iceberg-topics.adoc[bridge queries] against an Iceberg catalog, so that nested fields align as typed `ROW` columns on both sides of the union. + +| `JSON` (default) +| The topic schema is recursive, or you prefer flexible access through JSON functions. +| Recursive types supported; fields are untyped until extracted with JSON functions. Bridge queries that compare nested fields across the Kafka topic and the linked Iceberg table do not align cleanly, because Iceberg always exposes nested structures as `ROW` columns. +|=== + +== Next steps + +* xref:sql:query-data/query-streaming-topics.adoc[Query streaming topics]: query a topic without Iceberg history. +* xref:sql:query-data/query-iceberg-topics.adoc[Query Iceberg topics]: query the Iceberg-translated history of a topic. Use `struct_mapping_policy = 'COMPOUND'` so nested fields align between the Redpanda topic and the linked Iceberg table. +* xref:reference:sql/sql-data-types/row.adoc[ROW]: full reference for the `ROW` data type, including comparisons, NULL semantics, and conversion to text. +* xref:reference:sql/sql-statements/create-table.adoc[CREATE TABLE]: complete option list for mapping a Redpanda topic to a SQL table. From b762b86a0971927fe7bc8abd2bd39cd3c31371da Mon Sep 17 00:00:00 2001 From: Kat Batuigas Date: Wed, 13 May 2026 22:26:33 -0700 Subject: [PATCH 3/9] Review pass --- .../pages/query-data/query-nested-fields.adoc | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/modules/sql/pages/query-data/query-nested-fields.adoc b/modules/sql/pages/query-data/query-nested-fields.adoc index e5fe753a1..339d0f8a7 100644 --- a/modules/sql/pages/query-data/query-nested-fields.adoc +++ b/modules/sql/pages/query-data/query-nested-fields.adoc @@ -1,14 +1,14 @@ = Query topics with nested fields -:description: Map a topic with nested protobuf or Avro fields to SQL ROW columns, then query those fields directly. +:description: Map a topic with nested Protobuf or Avro fields to SQL ROW columns, then query those fields directly. :page-topic-type: how-to :personas: app_developer, data_engineer :learning-objective-1: Map a topic with a nested schema as a SQL table using struct_mapping_policy = 'COMPOUND' :learning-objective-2: Query nested fields using ROW field-access syntax :learning-objective-3: Recognize and resolve cyclic-reference errors -When a topic schema includes nested protobuf or Avro message types, you can map those nested structures as SQL `ROW` columns instead of opaque JSON. This makes nested fields queryable by name, includable in projections, and usable in `WHERE`, `GROUP BY`, and `ORDER BY` clauses, without parsing JSON at query time. +When a glossterm:topic[]'s schema includes nested Protobuf or Avro message types, you can map those nested structures as SQL `ROW` columns instead of opaque JSON. This makes nested fields queryable by name, includable in projections, and usable in `WHERE`, `GROUP BY`, and `ORDER BY` clauses, without parsing JSON at query time. -After completing these steps, you will be able to: +After reading this page, you will be able to: * [ ] {learning-objective-1} * [ ] {learning-objective-2} @@ -20,7 +20,7 @@ Before you query a topic with nested fields: * Enable Redpanda SQL on your Redpanda Bring Your Own Cloud (BYOC) cluster. See xref:sql:get-started/deploy-sql-cluster.adoc[Enable Redpanda SQL]. * Connect to Redpanda SQL with `psql` or another PostgreSQL client. See xref:sql:connect-to-sql/index.adoc[Connect to Redpanda SQL]. -* The topic has a schema (Protobuf or Avro) registered in Schema Registry. The schema includes one or more nested message types. +* The topic has a schema (Protobuf or Avro) registered in glossterm:schema-registry[Schema Registry]. The schema includes one or more nested message types. * You have a Redpanda catalog connection. See xref:reference:sql/sql-statements/create-redpanda-catalog.adoc[CREATE REDPANDA CATALOG]. == Map the topic as a SQL table @@ -61,7 +61,7 @@ TIP: To map nested structures as JSON instead, use `struct_mapping_policy = 'JSO == Query nested fields -Access a nested field by its declared name using the `(column).field` form. The parentheses around the column are required: +Access a nested field by its declared name using the `(column).field` form. You must wrap the column in parentheses: [source,sql] ---- @@ -79,11 +79,11 @@ FROM default_redpanda_catalog=>orders LIMIT 10; ---- -For schemas with multiple levels of nesting, chain the parenthesized field access: +For schemas with multiple levels of nesting, chain the parenthesized field access. For example, if `Customer` itself contained a nested `address` message with a `zip_code` field, you would query the zip code as: [source,sql] ---- -SELECT ((order_metadata).customer).customer_id FROM default_redpanda_catalog=>orders; +SELECT ((customer).address).zip_code FROM default_redpanda_catalog=>orders; ---- For the full `ROW` reference, including comparison operators, NULL handling, and `::text` casting, see xref:reference:sql/sql-data-types/row.adoc[ROW]. @@ -91,7 +91,7 @@ For the full `ROW` reference, including comparison operators, NULL handling, and [[handle-recursive-cyclic-schemas]] == Handle recursive (cyclic) schemas -If your topic schema includes a recursive structure — for example, a `Comment` message that references itself, or two messages that reference each other — mapping the table with `COMPOUND` fails at table-creation time with the following error: +Topic schemas can include recursive structures, such as a `Comment` message that references itself or two messages that reference each other. Mapping such a schema with `COMPOUND` fails at table-creation time with the following error: [source,text] ---- From f18cefedf1b46dda19a5bdc5a52de2ce707ddc21 Mon Sep 17 00:00:00 2001 From: Kat Batuigas Date: Sun, 17 May 2026 22:57:35 -0700 Subject: [PATCH 4/9] Apply suggestions from SME review --- .../pages/sql/sql-data-types/row.adoc | 34 +++++++++++++++---- .../sql/sql-statements/create-table.adoc | 6 ++-- .../pages/query-data/query-nested-fields.adoc | 24 ++++++------- 3 files changed, 43 insertions(+), 21 deletions(-) diff --git a/modules/reference/pages/sql/sql-data-types/row.adoc b/modules/reference/pages/sql/sql-data-types/row.adoc index 453c24dc0..a2d5ecb6a 100644 --- a/modules/reference/pages/sql/sql-data-types/row.adoc +++ b/modules/reference/pages/sql/sql-data-types/row.adoc @@ -146,22 +146,44 @@ Both ROW values must have the same number of fields, and corresponding fields mu == Check for NULL -ROW values support `IS NULL` and `IS NOT NULL`. +ROW values support `IS NULL` and `IS NOT NULL`, but with semantics that differ from scalar columns: + +* `expression IS NULL` returns `true` when the expression itself is NULL, *or* when all of the row's fields are NULL. +* `expression IS NOT NULL` returns `true` when the expression itself is non-NULL *and* all of the row's fields are non-NULL. + +Because of this, `IS NULL` and `IS NOT NULL` are not always inverses for ROW values. Both can return `false` for the same input, such as a ROW with a mix of NULL and non-NULL fields. [source,sql] ---- -SELECT ROW(1, 'a') IS NULL; +SELECT ROW(1, 'a') IS NULL AS is_null, + ROW(1, 'a') IS NOT NULL AS is_not_null; ---- [source,sql] ---- - ?column? ----------- - f + is_null | is_not_null +---------+------------- + f | t +(1 row) +---- + +For a ROW with at least one NULL field and at least one non-NULL field, both checks return `false`: + +[source,sql] +---- +SELECT ROW(NULL, 'a') IS NULL AS is_null, + ROW(NULL, 'a') IS NOT NULL AS is_not_null; +---- + +[source,sql] +---- + is_null | is_not_null +---------+------------- + f | f (1 row) ---- -To check whether a specific field within a ROW is NULL, access the field directly and test that. +NOTE: These checks do not recurse into nested ROW values. A nested ROW with all-NULL fields counts as a value (not NULL) at the outer level, so the outer `IS NULL` returns `false`. To check a specific nested field directly, access the field and test that. == Convert to text diff --git a/modules/reference/pages/sql/sql-statements/create-table.adoc b/modules/reference/pages/sql/sql-statements/create-table.adoc index f92fc4d32..3ff8c7ecb 100644 --- a/modules/reference/pages/sql/sql-statements/create-table.adoc +++ b/modules/reference/pages/sql/sql-statements/create-table.adoc @@ -1,8 +1,8 @@ = CREATE TABLE -:description: The CREATE TABLE statement maps a Redpanda topic to a SQL table through a catalog, making topic data queryable with SQL. +:description: The CREATE TABLE statement maps a Redpanda topic to a SQL table through a catalog, making the topic queryable with SQL. :page-topic-type: reference -The `CREATE TABLE` statement maps a Redpanda topic to a SQL table through a catalog. After creating the table, you can query topic data using standard SQL. +The `CREATE TABLE` statement maps a Redpanda topic to a SQL table through a catalog. After creating the table, you can query the topic using standard SQL. NOTE: You must first xref:reference:sql/sql-statements/create-redpanda-catalog.adoc[create a Redpanda catalog connection] before creating tables. `CREATE TABLE` in Redpanda SQL maps Redpanda topics to SQL tables and does not create standalone tables with user-defined schemas. @@ -53,7 +53,7 @@ a|How to handle records that fail deserialization. |No a|How to map nested structures from the topic schema to SQL columns. -* `COMPOUND` (default): Maps each nested structure to a SQL xref:reference:sql/sql-data-types/row.adoc[ROW] value with named fields, queryable using `(column).field_name` syntax. Cyclic types are not supported in `COMPOUND` mode. Use `JSON` for recursive schemas. +* `COMPOUND` (default): Maps each nested structure to a user-defined type with named fields, queryable using `(column).field_name` syntax. Cyclic types are not supported in `COMPOUND` mode. Use `JSON` for recursive schemas. See xref:reference:sql/sql-data-types/row.adoc[ROW] for the field-access syntax. * `JSON`: Stores each nested structure as a JSON value. Required for recursive (cyclic) types. |`output_schema_message_full_name` diff --git a/modules/sql/pages/query-data/query-nested-fields.adoc b/modules/sql/pages/query-data/query-nested-fields.adoc index 339d0f8a7..6a4016b89 100644 --- a/modules/sql/pages/query-data/query-nested-fields.adoc +++ b/modules/sql/pages/query-data/query-nested-fields.adoc @@ -1,14 +1,14 @@ = Query topics with nested fields -:description: Map a topic with nested Protobuf or Avro fields to SQL ROW columns, then query those fields directly. +:description: Map a topic with nested Protobuf, Avro, or JSON fields to SQL ROW columns, then query those fields directly. :page-topic-type: how-to :personas: app_developer, data_engineer :learning-objective-1: Map a topic with a nested schema as a SQL table using struct_mapping_policy = 'COMPOUND' :learning-objective-2: Query nested fields using ROW field-access syntax -:learning-objective-3: Recognize and resolve cyclic-reference errors +:learning-objective-3: Resolve cyclic-reference errors -When a glossterm:topic[]'s schema includes nested Protobuf or Avro message types, you can map those nested structures as SQL `ROW` columns instead of opaque JSON. This makes nested fields queryable by name, includable in projections, and usable in `WHERE`, `GROUP BY`, and `ORDER BY` clauses, without parsing JSON at query time. +When a glossterm:topic[]'s schema includes nested Protobuf, Avro, or JSON message types, you can map those nested structures as user-defined types (UDTs) with named fields, queryable using SQL `ROW` field-access syntax, instead of opaque JSON. This makes nested fields queryable by name, includable in projections, and usable in `WHERE`, `GROUP BY`, and `ORDER BY` clauses, without parsing JSON at query time. -After reading this page, you will be able to: +After completing these steps, you will be able to: * [ ] {learning-objective-1} * [ ] {learning-objective-2} @@ -20,12 +20,12 @@ Before you query a topic with nested fields: * Enable Redpanda SQL on your Redpanda Bring Your Own Cloud (BYOC) cluster. See xref:sql:get-started/deploy-sql-cluster.adoc[Enable Redpanda SQL]. * Connect to Redpanda SQL with `psql` or another PostgreSQL client. See xref:sql:connect-to-sql/index.adoc[Connect to Redpanda SQL]. -* The topic has a schema (Protobuf or Avro) registered in glossterm:schema-registry[Schema Registry]. The schema includes one or more nested message types. +* The topic has a schema registered in glossterm:schema-registry[Schema Registry]. The schema includes one or more nested message types. * You have a Redpanda catalog connection. See xref:reference:sql/sql-statements/create-redpanda-catalog.adoc[CREATE REDPANDA CATALOG]. == Map the topic as a SQL table -Create the SQL table with `struct_mapping_policy = 'COMPOUND'` to surface each nested message as a SQL `ROW` column: +Create the SQL table with `struct_mapping_policy = 'COMPOUND'` to surface each nested message as a user-defined type column: [source,sql] ---- @@ -55,9 +55,9 @@ message Customer { } ---- -Redpanda SQL maps the table with three columns: `order_id` (text), `customer` (a `ROW` with fields `customer_id`, `name`, and `region`), and `amount` (double precision). +Redpanda SQL maps the table with three columns: `order_id` (text), `customer` (a user-defined type with fields `customer_id`, `name`, and `region`), and `amount` (double precision). -TIP: To map nested structures as JSON instead, use `struct_mapping_policy = 'JSON'`. `JSON` is the default, and it is the only option that supports recursive (cyclic) types. See <>. +TIP: `COMPOUND` is the default `struct_mapping_policy`. To map nested structures as opaque JSON instead, use `struct_mapping_policy = 'JSON'`. JSON mapping is the only option that supports recursive (cyclic) types. See <>. == Query nested fields @@ -117,13 +117,13 @@ Query JSON-mapped fields with standard JSON functions instead of ROW field acces |=== | Policy | Use when | Trade-offs -| `COMPOUND` +| `COMPOUND` (default) | The topic schema has nested structures that are not recursive, and you want to query nested fields directly by name. -| Typed access; usable in `WHERE`, `GROUP BY`, `ORDER BY`. Required if you also plan to run xref:sql:query-data/query-iceberg-topics.adoc[bridge queries] against an Iceberg catalog, so that nested fields align as typed `ROW` columns on both sides of the union. +| Typed access; usable in `WHERE`, `GROUP BY`, `ORDER BY`. Required if you xref:sql:query-data/query-iceberg-topics.adoc[query an Iceberg-enabled topic via a linked Redpanda catalog], so that nested fields stay typed across both live and Iceberg-translated records. -| `JSON` (default) +| `JSON` | The topic schema is recursive, or you prefer flexible access through JSON functions. -| Recursive types supported; fields are untyped until extracted with JSON functions. Bridge queries that compare nested fields across the Kafka topic and the linked Iceberg table do not align cleanly, because Iceberg always exposes nested structures as `ROW` columns. +| Recursive types supported; fields are untyped until extracted with JSON functions. Queries that span the Redpanda topic and its linked Iceberg table do not align cleanly, because Iceberg always exposes nested structures as typed columns. |=== == Next steps From b4e76ebfa96230e0023e87b519d519d53e8fdc05 Mon Sep 17 00:00:00 2001 From: Kat Batuigas Date: Wed, 20 May 2026 22:27:40 -0700 Subject: [PATCH 5/9] Apply suggestions from SME review --- .../pages/query-data/query-nested-fields.adoc | 25 +++++++++++-------- 1 file changed, 15 insertions(+), 10 deletions(-) diff --git a/modules/sql/pages/query-data/query-nested-fields.adoc b/modules/sql/pages/query-data/query-nested-fields.adoc index 6a4016b89..3312cb0da 100644 --- a/modules/sql/pages/query-data/query-nested-fields.adoc +++ b/modules/sql/pages/query-data/query-nested-fields.adoc @@ -1,4 +1,4 @@ -= Query topics with nested fields += Query Topics with Nested Fields :description: Map a topic with nested Protobuf, Avro, or JSON fields to SQL ROW columns, then query those fields directly. :page-topic-type: how-to :personas: app_developer, data_engineer @@ -6,9 +6,9 @@ :learning-objective-2: Query nested fields using ROW field-access syntax :learning-objective-3: Resolve cyclic-reference errors -When a glossterm:topic[]'s schema includes nested Protobuf, Avro, or JSON message types, you can map those nested structures as user-defined types (UDTs) with named fields, queryable using SQL `ROW` field-access syntax, instead of opaque JSON. This makes nested fields queryable by name, includable in projections, and usable in `WHERE`, `GROUP BY`, and `ORDER BY` clauses, without parsing JSON at query time. +When a glossterm:topic[]'s schema includes nested Protobuf, Avro, or JSON message types, you can map those nested structures as user-defined types (UDTs) with named fields, queryable using SQL `ROW` field-access syntax instead of opaque JSON. This makes nested fields queryable by name, includable in projections, and usable in `WHERE`, `GROUP BY`, and `ORDER BY` clauses, without parsing JSON at query time. -After completing these steps, you will be able to: +Use this page to: * [ ] {learning-objective-1} * [ ] {learning-objective-2} @@ -20,8 +20,9 @@ Before you query a topic with nested fields: * Enable Redpanda SQL on your Redpanda Bring Your Own Cloud (BYOC) cluster. See xref:sql:get-started/deploy-sql-cluster.adoc[Enable Redpanda SQL]. * Connect to Redpanda SQL with `psql` or another PostgreSQL client. See xref:sql:connect-to-sql/index.adoc[Connect to Redpanda SQL]. -* The topic has a schema registered in glossterm:schema-registry[Schema Registry]. The schema includes one or more nested message types. -* You have a Redpanda catalog connection. See xref:reference:sql/sql-statements/create-redpanda-catalog.adoc[CREATE REDPANDA CATALOG]. +// TODO: Confirm TopicNameStrategy requirement +* Register a schema for the topic in glossterm:schema-registry[Schema Registry], including one or more nested message types. +* The topic's data is reachable through a Redpanda catalog. The `default_redpanda_catalog` is created and linked for you when Redpanda SQL is enabled. == Map the topic as a SQL table @@ -36,6 +37,10 @@ CREATE TABLE default_redpanda_catalog=>orders WITH ( ); ---- +* The `topic` option is required. +* The `struct_mapping_policy` option is optional and defaults to `'COMPOUND'`. +// TODO: Confirm schema_subject required when struct_mapping_policy=COMPOUND + Replace `orders` with your topic name and `orders-value` with the Schema Registry subject that holds the topic's value schema. For a topic schema with this Protobuf definition: @@ -61,7 +66,7 @@ TIP: `COMPOUND` is the default `struct_mapping_policy`. To map nested structures == Query nested fields -Access a nested field by its declared name using the `(column).field` form. You must wrap the column in parentheses: +Access a nested field by its declared name using the `(column).field` form. Wrap the column in parentheses: [source,sql] ---- @@ -91,14 +96,14 @@ For the full `ROW` reference, including comparison operators, NULL handling, and [[handle-recursive-cyclic-schemas]] == Handle recursive (cyclic) schemas -Topic schemas can include recursive structures, such as a `Comment` message that references itself or two messages that reference each other. Mapping such a schema with `COMPOUND` fails at table-creation time with the following error: +Redpanda SQL supports recursive (cyclic) schemas, such as a `Comment` message that references itself or two messages that reference each other, only when you map them to JSON. The `COMPOUND` policy does not support recursive types; trying to map such a schema fails at table-creation time with the following error: [source,text] ---- Cyclic reference at '.' → ''. Cyclic types are not supported in COMPOUND struct mapping policy; use struct_mapping_policy=JSON for recursive types. ---- -The error message tells you the resolution: re-create the table with `struct_mapping_policy = 'JSON'`. In JSON mode, Redpanda SQL stores each nested structure as a JSON value: +Re-create the table with `struct_mapping_policy = 'JSON'`. In JSON mode, Redpanda SQL stores each nested structure as a JSON value: [source,sql] ---- @@ -119,7 +124,7 @@ Query JSON-mapped fields with standard JSON functions instead of ROW field acces | `COMPOUND` (default) | The topic schema has nested structures that are not recursive, and you want to query nested fields directly by name. -| Typed access; usable in `WHERE`, `GROUP BY`, `ORDER BY`. Required if you xref:sql:query-data/query-iceberg-topics.adoc[query an Iceberg-enabled topic via a linked Redpanda catalog], so that nested fields stay typed across both live and Iceberg-translated records. +| Typed access; usable in `WHERE`, `GROUP BY`, `ORDER BY`. Required if you xref:sql:query-data/query-iceberg-topics.adoc[query an Iceberg-enabled topic via a linked Redpanda catalog], so that nested fields stay typed across both live records and the topic's Iceberg history. | `JSON` | The topic schema is recursive, or you prefer flexible access through JSON functions. @@ -129,6 +134,6 @@ Query JSON-mapped fields with standard JSON functions instead of ROW field acces == Next steps * xref:sql:query-data/query-streaming-topics.adoc[Query streaming topics]: query a topic without Iceberg history. -* xref:sql:query-data/query-iceberg-topics.adoc[Query Iceberg topics]: query the Iceberg-translated history of a topic. Use `struct_mapping_policy = 'COMPOUND'` so nested fields align between the Redpanda topic and the linked Iceberg table. +* xref:sql:query-data/query-iceberg-topics.adoc[Query Iceberg-enabled topics]: query a topic with both its live streaming data and Iceberg history. Use `struct_mapping_policy = 'COMPOUND'` so nested fields align between the Redpanda topic and the linked Iceberg table. * xref:reference:sql/sql-data-types/row.adoc[ROW]: full reference for the `ROW` data type, including comparisons, NULL semantics, and conversion to text. * xref:reference:sql/sql-statements/create-table.adoc[CREATE TABLE]: complete option list for mapping a Redpanda topic to a SQL table. From 6df6ff7ec2fb685209431ea54310bbbaa0557ab2 Mon Sep 17 00:00:00 2001 From: Kat Batuigas <36839689+kbatuigas@users.noreply.github.com> Date: Fri, 22 May 2026 12:01:35 -0700 Subject: [PATCH 6/9] Apply suggestions from code review Co-authored-by: Michele Cyran --- .../pages/sql/sql-data-types/row.adoc | 4 +-- .../pages/query-data/query-nested-fields.adoc | 26 +++++++++---------- 2 files changed, 15 insertions(+), 15 deletions(-) diff --git a/modules/reference/pages/sql/sql-data-types/row.adoc b/modules/reference/pages/sql/sql-data-types/row.adoc index a2d5ecb6a..ae6b44c0c 100644 --- a/modules/reference/pages/sql/sql-data-types/row.adoc +++ b/modules/reference/pages/sql/sql-data-types/row.adoc @@ -238,6 +238,6 @@ ON (a.col1, a.col2) = (b.col1, b.col2); // TODO: SME — confirm whether nested array-of-struct access (for example, `(arr_of_rows[1]).field_name`) works at GA, and whether wildcard expansion on an empty ROW (`(ROW()).*`) is supported. Both are tracked under OXLA-9444 and OXLA-9431 respectively and remain open as of 2026-05-13. -== See also +== Related topics -* xref:reference:sql/sql-statements/create-table.adoc[CREATE TABLE]: maps a Redpanda topic to a SQL table. Use `struct_mapping_policy = 'COMPOUND'` to surface nested topic fields as ROW columns. +* xref:reference:sql/sql-statements/create-table.adoc[CREATE TABLE]: Maps a Redpanda topic to a SQL table. Use `struct_mapping_policy = 'COMPOUND'` to surface nested topic fields as ROW columns. diff --git a/modules/sql/pages/query-data/query-nested-fields.adoc b/modules/sql/pages/query-data/query-nested-fields.adoc index 3312cb0da..c7bb67ab7 100644 --- a/modules/sql/pages/query-data/query-nested-fields.adoc +++ b/modules/sql/pages/query-data/query-nested-fields.adoc @@ -18,8 +18,8 @@ Use this page to: Before you query a topic with nested fields: -* Enable Redpanda SQL on your Redpanda Bring Your Own Cloud (BYOC) cluster. See xref:sql:get-started/deploy-sql-cluster.adoc[Enable Redpanda SQL]. -* Connect to Redpanda SQL with `psql` or another PostgreSQL client. See xref:sql:connect-to-sql/index.adoc[Connect to Redpanda SQL]. +* [Enable Redpanda SQL](xref:sql:get-started/deploy-sql-cluster.adoc[Enable Redpanda SQL]) on your Redpanda Bring Your Own Cloud (BYOC) cluster. +* [Connect to Redpanda SQL](xref:sql:connect-to-sql/index.adoc[Connect to Redpanda SQL]) with `psql` or another PostgreSQL client. // TODO: Confirm TopicNameStrategy requirement * Register a schema for the topic in glossterm:schema-registry[Schema Registry], including one or more nested message types. * The topic's data is reachable through a Redpanda catalog. The `default_redpanda_catalog` is created and linked for you when Redpanda SQL is enabled. @@ -31,14 +31,14 @@ Create the SQL table with `struct_mapping_policy = 'COMPOUND'` to surface each n [source,sql] ---- CREATE TABLE default_redpanda_catalog=>orders WITH ( - topic = 'orders', - schema_subject = 'orders-value', - struct_mapping_policy = 'COMPOUND' + topic = 'orders', <1> + schema_subject = 'orders-value', <2> + struct_mapping_policy = 'COMPOUND' <3> ); ---- - -* The `topic` option is required. -* The `struct_mapping_policy` option is optional and defaults to `'COMPOUND'`. +<1> Required. The Redpanda topic to map. +<2> Optional. The Schema Registry subject. Defaults to `-value` when omitted. +<3> Optional. Defaults to `'COMPOUND'`, which surfaces nested structures as user-defined types. // TODO: Confirm schema_subject required when struct_mapping_policy=COMPOUND Replace `orders` with your topic name and `orders-value` with the Schema Registry subject that holds the topic's value schema. @@ -131,9 +131,9 @@ Query JSON-mapped fields with standard JSON functions instead of ROW field acces | Recursive types supported; fields are untyped until extracted with JSON functions. Queries that span the Redpanda topic and its linked Iceberg table do not align cleanly, because Iceberg always exposes nested structures as typed columns. |=== -== Next steps +== Related topics -* xref:sql:query-data/query-streaming-topics.adoc[Query streaming topics]: query a topic without Iceberg history. -* xref:sql:query-data/query-iceberg-topics.adoc[Query Iceberg-enabled topics]: query a topic with both its live streaming data and Iceberg history. Use `struct_mapping_policy = 'COMPOUND'` so nested fields align between the Redpanda topic and the linked Iceberg table. -* xref:reference:sql/sql-data-types/row.adoc[ROW]: full reference for the `ROW` data type, including comparisons, NULL semantics, and conversion to text. -* xref:reference:sql/sql-statements/create-table.adoc[CREATE TABLE]: complete option list for mapping a Redpanda topic to a SQL table. +* xref:sql:query-data/query-streaming-topics.adoc[Query streaming topics]: Query a topic without Iceberg history. +* xref:sql:query-data/query-iceberg-topics.adoc[Query Iceberg-enabled topics]: Query a topic with both its live streaming data and Iceberg history. Use `struct_mapping_policy = 'COMPOUND'` so nested fields align between the Redpanda topic and the linked Iceberg table. +* xref:reference:sql/sql-data-types/row.adoc[ROW]: Full reference for the `ROW` data type, including comparisons, NULL semantics, and conversion to text. +* xref:reference:sql/sql-statements/create-table.adoc[CREATE TABLE]: Complete option list for mapping a Redpanda topic to a SQL table. From f8cb1ad3c66235aaff24bbeb4b84573f6dd8af2c Mon Sep 17 00:00:00 2001 From: Kat Batuigas Date: Fri, 22 May 2026 12:33:41 -0700 Subject: [PATCH 7/9] Review pass --- .../reference/pages/sql/sql-data-types/row.adoc | 6 ++---- .../pages/query-data/query-nested-fields.adoc | 17 +++++++---------- 2 files changed, 9 insertions(+), 14 deletions(-) diff --git a/modules/reference/pages/sql/sql-data-types/row.adoc b/modules/reference/pages/sql/sql-data-types/row.adoc index ae6b44c0c..d07840c9e 100644 --- a/modules/reference/pages/sql/sql-data-types/row.adoc +++ b/modules/reference/pages/sql/sql-data-types/row.adoc @@ -127,7 +127,7 @@ The wildcard form also works inside a `ROW(...)` constructor to copy fields from == Compare ROW values -ROW values support the standard comparison operators `=`, `<>`, `<`, `\<=`, `>`, and `>=`. Comparison is *lexicographic*: fields are compared in order, left to right, and the first differing field determines the result. +ROW values support the standard comparison operators `=`, `<>`, `<`, `<=`, `>`, and `>=`. Comparison is *lexicographic*: fields are compared in order, left to right, and the first differing field determines the result. [source,sql] ---- @@ -236,8 +236,6 @@ JOIN table_b b ON (a.col1, a.col2) = (b.col1, b.col2); ---- -// TODO: SME — confirm whether nested array-of-struct access (for example, `(arr_of_rows[1]).field_name`) works at GA, and whether wildcard expansion on an empty ROW (`(ROW()).*`) is supported. Both are tracked under OXLA-9444 and OXLA-9431 respectively and remain open as of 2026-05-13. - == Related topics -* xref:reference:sql/sql-statements/create-table.adoc[CREATE TABLE]: Maps a Redpanda topic to a SQL table. Use `struct_mapping_policy = 'COMPOUND'` to surface nested topic fields as ROW columns. +* xref:reference:sql/sql-statements/create-table.adoc[CREATE TABLE]: Maps a Redpanda topic to a SQL table. Use `struct_mapping_policy = 'COMPOUND'` to surface nested topic fields as user-defined types accessible with ROW field-access syntax. diff --git a/modules/sql/pages/query-data/query-nested-fields.adoc b/modules/sql/pages/query-data/query-nested-fields.adoc index c7bb67ab7..3478f8a77 100644 --- a/modules/sql/pages/query-data/query-nested-fields.adoc +++ b/modules/sql/pages/query-data/query-nested-fields.adoc @@ -1,12 +1,12 @@ = Query Topics with Nested Fields -:description: Map a topic with nested Protobuf, Avro, or JSON fields to SQL ROW columns, then query those fields directly. +:description: Map a topic's nested fields to typed SQL columns and query them by name. :page-topic-type: how-to :personas: app_developer, data_engineer :learning-objective-1: Map a topic with a nested schema as a SQL table using struct_mapping_policy = 'COMPOUND' :learning-objective-2: Query nested fields using ROW field-access syntax :learning-objective-3: Resolve cyclic-reference errors -When a glossterm:topic[]'s schema includes nested Protobuf, Avro, or JSON message types, you can map those nested structures as user-defined types (UDTs) with named fields, queryable using SQL `ROW` field-access syntax instead of opaque JSON. This makes nested fields queryable by name, includable in projections, and usable in `WHERE`, `GROUP BY`, and `ORDER BY` clauses, without parsing JSON at query time. +When a glossterm:topic[]'s schema includes nested Protobuf, Avro, or JSON message types, you can map those nested structures as user-defined types (UDTs) with named fields. UDT columns are queryable using SQL `ROW` field-access syntax instead of opaque JSON, so nested fields are queryable by name, includable in projections, and usable in `WHERE`, `GROUP BY`, and `ORDER BY` clauses without parsing JSON at query time. Use this page to: @@ -18,9 +18,8 @@ Use this page to: Before you query a topic with nested fields: -* [Enable Redpanda SQL](xref:sql:get-started/deploy-sql-cluster.adoc[Enable Redpanda SQL]) on your Redpanda Bring Your Own Cloud (BYOC) cluster. -* [Connect to Redpanda SQL](xref:sql:connect-to-sql/index.adoc[Connect to Redpanda SQL]) with `psql` or another PostgreSQL client. -// TODO: Confirm TopicNameStrategy requirement +* Enable Redpanda SQL on your Redpanda Bring Your Own Cloud (BYOC) cluster. See xref:sql:get-started/deploy-sql-cluster.adoc[Enable Redpanda SQL]. +* Connect to Redpanda SQL with `psql` or another PostgreSQL client. See xref:sql:connect-to-sql/index.adoc[Connect to Redpanda SQL]. * Register a schema for the topic in glossterm:schema-registry[Schema Registry], including one or more nested message types. * The topic's data is reachable through a Redpanda catalog. The `default_redpanda_catalog` is created and linked for you when Redpanda SQL is enabled. @@ -39,9 +38,8 @@ CREATE TABLE default_redpanda_catalog=>orders WITH ( <1> Required. The Redpanda topic to map. <2> Optional. The Schema Registry subject. Defaults to `-value` when omitted. <3> Optional. Defaults to `'COMPOUND'`, which surfaces nested structures as user-defined types. -// TODO: Confirm schema_subject required when struct_mapping_policy=COMPOUND -Replace `orders` with your topic name and `orders-value` with the Schema Registry subject that holds the topic's value schema. +Replace `orders` with your topic name. Your topic must have a schema registered in Schema Registry. For details on the `schema_subject` option, see xref:reference:sql/sql-statements/create-table.adoc[CREATE TABLE]. For a topic schema with this Protobuf definition: @@ -62,7 +60,7 @@ message Customer { Redpanda SQL maps the table with three columns: `order_id` (text), `customer` (a user-defined type with fields `customer_id`, `name`, and `region`), and `amount` (double precision). -TIP: `COMPOUND` is the default `struct_mapping_policy`. To map nested structures as opaque JSON instead, use `struct_mapping_policy = 'JSON'`. JSON mapping is the only option that supports recursive (cyclic) types. See <>. +TIP: `COMPOUND` is the default `struct_mapping_policy`. To map nested structures as opaque JSON instead, use `struct_mapping_policy = 'JSON'`. Cyclic types require `struct_mapping_policy = 'JSON'`. See <>. == Query nested fields @@ -96,7 +94,7 @@ For the full `ROW` reference, including comparison operators, NULL handling, and [[handle-recursive-cyclic-schemas]] == Handle recursive (cyclic) schemas -Redpanda SQL supports recursive (cyclic) schemas, such as a `Comment` message that references itself or two messages that reference each other, only when you map them to JSON. The `COMPOUND` policy does not support recursive types; trying to map such a schema fails at table-creation time with the following error: +The `COMPOUND` policy does not support recursive (cyclic) schemas, such as a `Comment` message that references itself or two messages that reference each other. Trying to map such a schema with `COMPOUND` fails at table-creation time with the following error: [source,text] ---- @@ -109,7 +107,6 @@ Re-create the table with `struct_mapping_policy = 'JSON'`. In JSON mode, Redpand ---- CREATE TABLE default_redpanda_catalog=>comments WITH ( topic = 'comments', - schema_subject = 'comments-value', struct_mapping_policy = 'JSON' ); ---- From 3a9131abba18172318e953461c15430e5085ea8c Mon Sep 17 00:00:00 2001 From: Kat Batuigas <36839689+kbatuigas@users.noreply.github.com> Date: Fri, 22 May 2026 17:04:15 -0700 Subject: [PATCH 8/9] Apply suggestions from code review Co-authored-by: Michele Cyran --- modules/reference/pages/sql/sql-data-types/row.adoc | 2 +- modules/sql/pages/query-data/query-nested-fields.adoc | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/modules/reference/pages/sql/sql-data-types/row.adoc b/modules/reference/pages/sql/sql-data-types/row.adoc index d07840c9e..450e3ddcb 100644 --- a/modules/reference/pages/sql/sql-data-types/row.adoc +++ b/modules/reference/pages/sql/sql-data-types/row.adoc @@ -2,7 +2,7 @@ :description: The ROW data type represents a composite value containing one or more fields of different types. :page-topic-type: reference -The `ROW` data type represents a composite value (also known as a struct or record) containing one or more fields of different types. ROW values support field access, lexicographic comparison, NULL checks, conversion to text, and use in `GROUP BY`, `ORDER BY`, and `JOIN` clauses. +The `ROW` data type represents a composite value (also known as a struct or record) containing one or more fields of different types. `ROW` values support field access, lexicographic comparison, NULL checks, conversion to text, and use in `GROUP BY`, `ORDER BY`, and `JOIN` clauses. == Syntax diff --git a/modules/sql/pages/query-data/query-nested-fields.adoc b/modules/sql/pages/query-data/query-nested-fields.adoc index 3478f8a77..b29f889ef 100644 --- a/modules/sql/pages/query-data/query-nested-fields.adoc +++ b/modules/sql/pages/query-data/query-nested-fields.adoc @@ -18,8 +18,8 @@ Use this page to: Before you query a topic with nested fields: -* Enable Redpanda SQL on your Redpanda Bring Your Own Cloud (BYOC) cluster. See xref:sql:get-started/deploy-sql-cluster.adoc[Enable Redpanda SQL]. -* Connect to Redpanda SQL with `psql` or another PostgreSQL client. See xref:sql:connect-to-sql/index.adoc[Connect to Redpanda SQL]. +* [Enable Redpanda SQL](xref:sql:get-started/deploy-sql-cluster.adoc[Enable Redpanda SQL]) on your Redpanda Bring Your Own Cloud (BYOC) cluster. +* [Connect to Redpanda SQL](xref:sql:connect-to-sql/index.adoc[Connect to Redpanda SQL]) with `psql` or another PostgreSQL client. * Register a schema for the topic in glossterm:schema-registry[Schema Registry], including one or more nested message types. * The topic's data is reachable through a Redpanda catalog. The `default_redpanda_catalog` is created and linked for you when Redpanda SQL is enabled. From 3b589b0956288405be9d58a266f0f5516eb1c498 Mon Sep 17 00:00:00 2001 From: Kat Batuigas Date: Fri, 22 May 2026 17:16:16 -0700 Subject: [PATCH 9/9] Apply suggestions --- modules/reference/pages/sql/sql-data-types/row.adoc | 4 ++-- modules/sql/pages/query-data/query-nested-fields.adoc | 6 +++--- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/modules/reference/pages/sql/sql-data-types/row.adoc b/modules/reference/pages/sql/sql-data-types/row.adoc index 450e3ddcb..7fa37cd76 100644 --- a/modules/reference/pages/sql/sql-data-types/row.adoc +++ b/modules/reference/pages/sql/sql-data-types/row.adoc @@ -99,7 +99,7 @@ The parentheses around the ROW expression are required when accessing a field. === Access by name -For composite columns with declared field names — for example, columns mapped from a topic with `struct_mapping_policy = 'COMPOUND'` (see xref:reference:sql/sql-statements/create-table.adoc[CREATE TABLE]) — access fields by their declared names: +For composite columns with declared field names, for example, columns mapped from a topic with `struct_mapping_policy = 'COMPOUND'` (see xref:reference:sql/sql-statements/create-table.adoc[CREATE TABLE]), access fields by their declared names: [source,sql] ---- @@ -236,6 +236,6 @@ JOIN table_b b ON (a.col1, a.col2) = (b.col1, b.col2); ---- -== Related topics +== Suggested reading * xref:reference:sql/sql-statements/create-table.adoc[CREATE TABLE]: Maps a Redpanda topic to a SQL table. Use `struct_mapping_policy = 'COMPOUND'` to surface nested topic fields as user-defined types accessible with ROW field-access syntax. diff --git a/modules/sql/pages/query-data/query-nested-fields.adoc b/modules/sql/pages/query-data/query-nested-fields.adoc index b29f889ef..b86bbdf66 100644 --- a/modules/sql/pages/query-data/query-nested-fields.adoc +++ b/modules/sql/pages/query-data/query-nested-fields.adoc @@ -18,8 +18,8 @@ Use this page to: Before you query a topic with nested fields: -* [Enable Redpanda SQL](xref:sql:get-started/deploy-sql-cluster.adoc[Enable Redpanda SQL]) on your Redpanda Bring Your Own Cloud (BYOC) cluster. -* [Connect to Redpanda SQL](xref:sql:connect-to-sql/index.adoc[Connect to Redpanda SQL]) with `psql` or another PostgreSQL client. +* xref:sql:get-started/deploy-sql-cluster.adoc[Enable Redpanda SQL] on your Redpanda Bring Your Own Cloud (BYOC) cluster. +* xref:sql:connect-to-sql/index.adoc[Connect to Redpanda SQL] with `psql` or another PostgreSQL client. * Register a schema for the topic in glossterm:schema-registry[Schema Registry], including one or more nested message types. * The topic's data is reachable through a Redpanda catalog. The `default_redpanda_catalog` is created and linked for you when Redpanda SQL is enabled. @@ -128,7 +128,7 @@ Query JSON-mapped fields with standard JSON functions instead of ROW field acces | Recursive types supported; fields are untyped until extracted with JSON functions. Queries that span the Redpanda topic and its linked Iceberg table do not align cleanly, because Iceberg always exposes nested structures as typed columns. |=== -== Related topics +== Suggested reading * xref:sql:query-data/query-streaming-topics.adoc[Query streaming topics]: Query a topic without Iceberg history. * xref:sql:query-data/query-iceberg-topics.adoc[Query Iceberg-enabled topics]: Query a topic with both its live streaming data and Iceberg history. Use `struct_mapping_policy = 'COMPOUND'` so nested fields align between the Redpanda topic and the linked Iceberg table.