Skip to content

lake: add UDF docs - #23544

Draft
awxxxxxx wants to merge 4 commits into
pingcap:release-8.5from
awxxxxxx:lake-publish-udf-docs
Draft

lake: add UDF docs#23544
awxxxxxx wants to merge 4 commits into
pingcap:release-8.5from
awxxxxxx:lake-publish-udf-docs

Conversation

@awxxxxxx

@awxxxxxx awxxxxxx commented Aug 17, 2026

Copy link
Copy Markdown
Collaborator

What is changed, added or deleted? (Required)

This PR publishes TiDB Cloud Lake UDF (user-defined functions) documentation with minimal changes to existing pages:

  • adds a comprehensive guide for choosing among SQL, Python, JavaScript, WebAssembly, aggregate, table, and external UDFs;
  • exposes the External AI Functions guide and External Function CREATE FUNCTION reference in the Lake TOC;
  • preserves the existing External AI and command-page structure while using tidbcloudlake-udf and tidbcloudlake_udf in external Python examples;
  • updates external examples to use a public HTTPS endpoint and documents the tenant UDF server allowlist requirement;
  • fixes External Function management links.

The examples were validated against the published tidbcloudlake-udf v0.2.20 package and a staging Lake warehouse. SQL scalar/table UDFs and Python/JavaScript aggregate UDFs passed. External registration correctly returned Unallowed UDF server address for a hostname not yet added to the tenant allowlist.

Which TiDB version(s) do your changes apply to? (Required)

Tips for choosing the affected version(s):

By default, CHOOSE MASTER ONLY so your changes will be applied to the next TiDB major or minor releases. If your PR involves a product feature behavior change or a compatibility change, CHOOSE THE AFFECTED RELEASE BRANCH(ES) AND MASTER.

For details, see tips for choosing the affected versions.

  • master (the latest development version)
  • v8.5 (TiDB 8.5 versions)
  • v8.4 (TiDB 8.4 versions)
  • v8.3 (TiDB 8.3 versions)
  • v8.2 (TiDB 8.2 versions)
  • v8.1 (TiDB 8.1 versions)
  • v7.5 (TiDB 7.5 versions)
  • v7.1 (TiDB 7.1 versions)
  • v6.5 (TiDB 6.5 versions)

What is the related PR or file link(s)?

AI agent involvement

  • The changes in this PR were primarily made by an AI agent on behalf of the PR author.

Do your changes match any of the following descriptions?

  • Delete files
  • Change aliases
  • Need modification after applied to another branch
  • Might cause conflicts after applied to another branch

@ti-chi-bot ti-chi-bot Bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Aug 17, 2026
@ti-chi-bot

ti-chi-bot Bot commented Aug 17, 2026

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign hfxsd for approval. For more information see the Code Review Process.
Please ensure that each of them provides their approval before proceeding.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot

ti-chi-bot Bot commented Aug 17, 2026

Copy link
Copy Markdown

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@ti-chi-bot ti-chi-bot Bot added the contribution This PR is from a community contributor. label Aug 17, 2026
@coderabbitai

coderabbitai Bot commented Aug 17, 2026

Copy link
Copy Markdown

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 0dc36d08-b6b9-42e0-a9c3-597e4432253c

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@ti-chi-bot ti-chi-bot Bot added missing-translation-status This PR does not have translation status info. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Aug 17, 2026
@ti-chi-bot ti-chi-bot Bot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Aug 17, 2026
@lilin90 lilin90 self-assigned this Aug 17, 2026
@lilin90 lilin90 added the translation/no-need No need to translate this PR. label Aug 17, 2026
@ti-chi-bot ti-chi-bot Bot removed the missing-translation-status This PR does not have translation status info. label Aug 17, 2026
@lilin90 lilin90 added the lake Related to TiDB Cloud Lake docs. label Aug 17, 2026
@lilin90 lilin90 changed the title lake: publish UDF documentation lake: add UDF docs Aug 17, 2026
@lilin90 lilin90 added the for-cloud-release This PR is related to TiDB Cloud release. label Aug 17, 2026
@lilin90
lilin90 requested review from sdojjy and a lite review from Copilot August 17, 2026 08:48

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds and surfaces TiDB Cloud Lake UDF documentation, including a new guide to help users choose among UDF types, and updates the External Function docs and examples to use the tidbcloudlake-udf package and public HTTPS endpoints.

Changes:

  • Adds a new “Choose a User-Defined Function Type” guide and links it from the Lake TOC.
  • Exposes the “External AI Functions” guide in the Lake TOC and updates Python examples to use tidbcloudlake-udf.
  • Updates External Function navigation and examples, including adding the CREATE FUNCTION reference entry.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
TOC-tidb-cloud-lake.md Adds TOC entries for External AI Functions, the UDF type selection guide, and External Function command references.
tidb-cloud-lake/sql/external-function.md Updates the External Function overview table to point at the correct command pages (and should align link text with actual SQL statements).
tidb-cloud-lake/sql/create-function.md Updates the external function end-to-end example to use tidbcloudlake-udf and a public HTTPS ADDRESS.
tidb-cloud-lake/sql/alter-function-sql.md Updates external function ADDRESS examples to a public HTTPS endpoint.
tidb-cloud-lake/guides/external-ai-functions.md Updates the external AI function example to use tidbcloudlake-udf.
tidb-cloud-lake/guides/choose-a-udf-type.md Adds a new guide explaining how to choose between SQL/script/WASM/aggregate/table/external UDF types and operational considerations.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment on lines +14 to +16
| [CREATE EXTERNAL FUNCTION](/tidb-cloud-lake/sql/create-function.md) | Creates a new external function |
| [ALTER EXTERNAL FUNCTION](/tidb-cloud-lake/sql/alter-function-sql.md) | Modifies an existing external function |
| [DROP EXTERNAL FUNCTION](/tidb-cloud-lake/sql/drop-function-sql.md) | Removes an external function |

```bash
pip install databend-udf
pip install tidbcloudlake-udf
Comment on lines 77 to +81
CREATE FUNCTION gcd AS (INT, INT)
RETURNS INT
LANGUAGE python
HANDLER = 'gcd'
ADDRESS = 'http://localhost:8815';
ADDRESS = 'https://udf.example.com';
Comment on lines +18 to +22
- standardizing data cleaning, validation, and business calculations;
- encapsulating a parameterized SQL query;
- implementing a custom aggregation that needs intermediate state;
- calling Python libraries, proprietary logic, or machine learning models;
- scaling specialized compute independently from the warehouse.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

contribution This PR is from a community contributor. do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. for-cloud-release This PR is related to TiDB Cloud release. lake Related to TiDB Cloud Lake docs. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. translation/no-need No need to translate this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants