Fix table UDF result block splitting

[ColinLeeo]

Description

Problem

A table-model UDF may produce a large result block when processing a partition containing a large device. Previously, TableFunctionOperator returned the generated TsBlock directly without enforcing the configured row-count or byte-size limits.

As a result, a sufficiently large UDF result could eventually exceed the RPC frame limit, for example the 64 MB Thrift frame limit.

Design

TableFunctionOperator now splits generated result blocks before returning them to downstream operators.

For each output fragment, the operator:

1. Limits the candidate row count using max_tsblock_line_number.
2. Checks the candidate block against max_tsblock_size_in_bytes.
3. Uses binary search to find the largest row prefix that fits within the byte-size limit.
4. Retains the remaining rows and returns them in subsequent calls to next().

Binary search was selected because row sizes may vary significantly for variable-width types such as TEXT, STRING, and BLOB. Calculating a fixed row count from the average size of an earlier block would be cheaper, but could allow later blocks containing wider values to exceed the configured byte-size limit.

The size check uses a copied region instead of TsBlock.getRegion(). A zero-copy region keeps the source columns' backing arrays, so its reported size may reflect backing capacity rather than the exact logical contents of the selected rows. Copying the selected region provides an accurate size for variable-width data and prevents a small returned fragment from retaining a large source block.

The splitting logic is applied when results are dequeued, so blocks generated by both process() and finish() follow the same path. This also keeps the splitting behavior independent of pass-through column construction.

Configuration and behavior

This PR introduces no new configuration.

The implementation uses the existing configurations:

• max_tsblock_size_in_bytes, whose default value is 131072.
• max_tsblock_line_number, whose default value is 1000.

The existing configuration validation remains unchanged. The operator defensively returns at least one row per call so that execution always makes progress, even if the configured line limit is invalid or a single row is larger than the byte-size target.

Rows are indivisible. Therefore, if one individual row is larger than max_tsblock_size_in_bytes, that row is returned alone. Preventing a single value larger than the RPC frame limit would require a separate rejection or value-level fragmentation policy.

The following behaviors are preserved:

• Result row ordering is unchanged.
• Empty UDF output still produces no result block.
• All queued and retained fragments are returned before the operator reports that it is finished.
• Retained result blocks are released when the operator is closed.

Alternative designs

An alternative was to calculate a fixed maximum row count from the average row size of the first result block. Although this avoids repeated size checks, it is not reliable when different calls produce values with significantly different widths.

Another alternative was to serialize every candidate block and use its serialized length directly. This would closely match the transport representation but would couple the execution operator to the serialization layer and add repeated serialization overhead. The implemented design follows the existing operator memory-size contract, while the unit test additionally verifies that the serialized result also remains within the configured limit.

Tests

Added testVariableWidthResultsAreSplitByActualSize to cover variable-width output:

• process() first produces a narrow TEXT result.
• finish() then produces a wide TEXT result exceeding both the configured row-count and byte-size limits.
• The test verifies that the result is split into multiple blocks.
• Every returned block respects max_tsblock_line_number.
• Every returned block respects max_tsblock_size_in_bytes.
• The actual serialized size of every returned block is also checked.
• The total row count, row ordering, and value lengths are verified.

The following targeted test was run successfully:

mvn test -pl iotdb-core/datanode -am \
  -Dtest=TableFunctionOperatorTest \
  -DfailIfNoTests=false \
  -Dsurefire.failIfNoSpecifiedTests=false

---

This PR has:

• been self-reviewed.
• added comments explaining the "why" and the intent of the code wherever it would not be obvious to an unfamiliar reader.
• added or updated unit tests to cover the new code paths.
• been tested in a test IoTDB cluster.

---

Key changed/added classes

• TableFunctionOperator

  • Adds row-count and byte-size-aware result splitting.
  • Manages queued and retained result fragments.
  • Copies returned regions to obtain accurate variable-width sizes and release large backing arrays.
  • Updates completion, memory estimation, and cleanup behavior.

• TableFunctionOperatorTest

  • Adds coverage for variable-width UDF results exceeding both row-count and byte-size limits.
  • Verifies logical and serialized result sizes.

[codecov]

Codecov Report

❌ Patch coverage is 93.93939% with 2 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (master@c13be2c). Learn more about missing BASE report.
⚠️ Report is 28 commits behind head on master.

Files with missing lines	Patch %	Lines
...erator/process/function/TableFunctionOperator.java	93.93%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff            @@
##             master   #18310   +/-   ##
=========================================
  Coverage          ?   43.22%           
  Complexity        ?      374           
=========================================
  Files             ?     5363           
  Lines             ?   382084           
  Branches          ?    49638           
=========================================
  Hits              ?   165156           
  Misses            ?   216928           
  Partials          ?        0           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[ColinLeeo]

Superseded by #18333 after renaming the head branch to fix/table-udf-result-chunking to follow the repository branch naming convention.

[sonarqubecloud]

Quality Gate failed

Failed conditions
E Reliability Rating on New Code (required ≥ A)

See analysis details on SonarQube Cloud

Catch issues before they fail your Quality Gate with our IDE extension SonarQube for IDE