[Enhancement] (http) Expose structured metadata for nested column types in table schema API

[Standing-Man]

Search before asking

• I had searched in the issues and found no similar issues.

Description

The table schema API:

GET /api/{db}/{table}/_schema
GET /api/{catalog}/{db}/{table}/_schema

currently exposes only the top-level primitive type through the type field. The legacy precision and scale fields are provided only when the column itself is a decimal type.

For example:

CREATE TABLE doris_types (
    types_id INT,
    c_decimal ARRAY<DECIMAL(18, 4)>
)
UNIQUE KEY(types_id)
DISTRIBUTED BY HASH(types_id) BUCKETS 1
PROPERTIES (
    "replication_num" = "1"
);

The schema response identifies c_decimal only as:

{
  "name": "c_decimal",
  "type": "ARRAY"
}

Consumers cannot discover that the array element is DECIMAL(18, 4), including its precision and scale. The same limitation applies to nested ARRAY, MAP, and STRUCT combinations.

This affects external connectors that use the Doris schema API for data conversion. For example, RisingWave's Doris sink attempted to retrieve decimal precision and scale from the top-level ARRAY type and panicked because this information was unavailable:

risingwavelabs/risingwave#16176

Although connectors should handle missing metadata without panicking, Doris should expose enough schema information for clients to interpret nested column types reliably.

Solution

Add two backward-compatible fields to every column returned by the table schema API:

• type_sql: Complete SQL representation of the column type.
• type_desc: Recursively structured type metadata.

For ARRAY<DECIMAL(18, 4)>, the response would include:

{
  "name": "c_decimal",
  "type": "ARRAY",
  "type_sql": "array<decimalv3(18,4)>",
  "type_desc": {
    "kind": "ARRAY",
    "sql": "array<decimalv3(18,4)>",
    "contains_null": true,
    "element": {
      "kind": "DECIMAL64",
      "sql": "decimalv3(18,4)",
      "precision": 18,
      "scale": 4
    }
  }
}

The recursive representation should support:

• ARRAY: element type and element nullability.
• MAP: key/value types and their nullability.
• STRUCT: ordered fields, field types, and field nullability.
• Decimal types: precision and scale.
• CHAR and VARCHAR: length.
• DATETIMEV2, TIMEV2, and TIMESTAMPTZ: scale.
• Primitive types: type kind and SQL representation.

Fields that do not apply to a type should be omitted.

Existing fields such as type, precision, and scale should remain unchanged for backward compatibility. The new metadata should be included for both base-table columns and materialized-index columns.

Add two backward-compatible fields to every column returned by the table schema API:

• type_sql: Complete SQL representation of the column type.
• type_desc: Recursively structured type metadata.

For ARRAY<DECIMAL(18, 4)>, the response would include:

{
  "name": "c_decimal",
  "type": "ARRAY",
  "type_sql": "array<decimalv3(18,4)>",
  "type_desc": {
    "kind": "ARRAY",
    "sql": "array<decimalv3(18,4)>",
    "contains_null": true,
    "element": {
      "kind": "DECIMAL64",
      "sql": "decimalv3(18,4)",
      "precision": 18,
      "scale": 4
    }
  }
}

The recursive representation should support:

• ARRAY: element type and element nullability.
• MAP: key/value types and their nullability.
• STRUCT: ordered fields, field types, and field nullability.
• Decimal types: precision and scale.
• CHAR and VARCHAR: length.
• DATETIMEV2, TIMEV2, and TIMESTAMPTZ: scale.
• Primitive types: type kind and SQL representation.

Fields that do not apply to a type should be omitted.

Existing fields such as type, precision, and scale should remain unchanged for backward compatibility. The new metadata should be included for both base-table columns and materialized-index columns.

Acceptance criteria

• Nested decimal precision and scale can be obtained without parsing type strings.
• Arbitrarily nested ARRAY, MAP, and STRUCT types are represented recursively.
• Nullability is preserved at each nested level.
• Existing schema API consumers remain compatible.
• Base-table and materialized-index schemas use the same representation.

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct