From 2660d9d2b850f82615f9bef22e50680d85de98fd Mon Sep 17 00:00:00 2001 From: awxxxxxx Date: Mon, 17 Aug 2026 14:17:06 +0800 Subject: [PATCH 1/4] lake: publish UDF documentation --- TOC-tidb-cloud-lake.md | 5 +- tidb-cloud-lake/guides/choose-a-udf-type.md | 354 ++++++++++++++++++ .../guides/external-ai-functions.md | 118 +++--- tidb-cloud-lake/sql/alter-function-sql.md | 55 +-- tidb-cloud-lake/sql/create-function.md | 10 +- tidb-cloud-lake/sql/drop-function-sql.md | 4 +- tidb-cloud-lake/sql/external-function.md | 48 ++- 7 files changed, 505 insertions(+), 89 deletions(-) create mode 100644 tidb-cloud-lake/guides/choose-a-udf-type.md diff --git a/TOC-tidb-cloud-lake.md b/TOC-tidb-cloud-lake.md index 5afecca43656e..f30d8d6813e17 100644 --- a/TOC-tidb-cloud-lake.md +++ b/TOC-tidb-cloud-lake.md @@ -22,6 +22,7 @@ - [Python](/tidb-cloud-lake/guides/connect-using-python.md) - [Rust](/tidb-cloud-lake/guides/connect-using-rust.md) - AI Tools + - [External AI Functions](/tidb-cloud-lake/guides/external-ai-functions.md) - [MCP Client Integration](/tidb-cloud-lake/guides/mcp-client-integration.md) - [MCP Server](/tidb-cloud-lake/guides/mcp-server.md) - Visualization @@ -368,6 +369,7 @@ - [SHOW VIRTUAL COLUMNS](/tidb-cloud-lake/sql/show-virtual-columns.md) - User-Defined Function - [User-Defined Function](/tidb-cloud-lake/sql/user-defined-function.md) + - [Choose a User-Defined Function Type](/tidb-cloud-lake/guides/choose-a-udf-type.md) - [ALTER FUNCTION](/tidb-cloud-lake/sql/alter-function.md) - [CREATE AGGREGATE FUNCTION](/tidb-cloud-lake/sql/create-aggregate-function.md) - [CREATE SCALAR FUNCTION](/tidb-cloud-lake/sql/create-scalar-function.md) @@ -375,9 +377,10 @@ - [DROP FUNCTION](/tidb-cloud-lake/sql/drop-function.md) - [SHOW USER FUNCTIONS](/tidb-cloud-lake/sql/show-user-functions.md) - External Function + - [External Function](/tidb-cloud-lake/sql/external-function.md) + - [CREATE FUNCTION](/tidb-cloud-lake/sql/create-function.md) - [ALTER FUNCTION](/tidb-cloud-lake/sql/alter-function-sql.md) - [DROP FUNCTION](/tidb-cloud-lake/sql/drop-function-sql.md) - - [External Function](/tidb-cloud-lake/sql/external-function.md) - Masking Policy - [Overview](/tidb-cloud-lake/sql/masking-policy-sql.md) - [CREATE MASKING POLICY](/tidb-cloud-lake/sql/create-masking-policy.md) diff --git a/tidb-cloud-lake/guides/choose-a-udf-type.md b/tidb-cloud-lake/guides/choose-a-udf-type.md new file mode 100644 index 0000000000000..f72acaeb88219 --- /dev/null +++ b/tidb-cloud-lake/guides/choose-a-udf-type.md @@ -0,0 +1,354 @@ +--- +title: Choose a User-Defined Function Type +summary: Learn how to choose a SQL, aggregate, table, or external UDF in TiDB Cloud Lake based on return shape, runtime, and operations. +--- + +# Choose a User-Defined Function Type + +User-defined functions (UDFs) let you package reusable logic that is not available as a built-in SQL function. They can standardize business rules, simplify complex queries, implement custom aggregations, expand values into rows, or connect SQL queries to independently hosted Python services. + +Before creating a UDF, check the [SQL Function Reference](/tidb-cloud-lake/sql/sql-function-reference.md). Built-in functions usually provide the simplest implementation, the lowest execution overhead, and the smallest operational burden. + +## Why use UDFs + +As analytical workloads grow, the same transformations often appear across many queries. Copying an expression into every query makes it harder to keep behavior consistent and to roll out changes safely. + +UDFs are useful for the following tasks: + +- standardizing data cleaning, validation, and business calculations; +- encapsulating a parameterized SQL query; +- implementing a custom aggregation that needs intermediate state; +- calling Python libraries, proprietary logic, or machine learning models; +- scaling specialized compute independently from the warehouse. + +A UDF should have one clear responsibility. Data movement, scheduling, streaming state, joins across continuous event streams, and workflow orchestration belong in the corresponding Lake SQL, Stream, Task, or integration feature instead of inside a UDF. + +## Understand the UDF ecosystem + +{{{ .lake }}} provides several UDF execution models. They differ in return shape, language, hosting, and operational responsibility. + +| UDF type | Implementation | Output | Hosting | Typical use | +| --- | --- | --- | --- | --- | +| SQL scalar UDF | SQL expression | One value for each input row | Managed by {{{ .lake }}} | Formatting, calculations, and reusable conditions | +| Script scalar UDF | Python or JavaScript | One value for each input row | Managed by {{{ .lake }}} | Business rules, validation, and structured data processing | +| WebAssembly scalar UDF | WebAssembly module | One value for each input row | Managed by {{{ .lake }}} | Compute-intensive logic compiled to WebAssembly | +| Aggregate UDF | Python or JavaScript | One value for each group | Managed by {{{ .lake }}} | Custom stateful aggregation | +| SQL table UDF | SQL query | A result set | Managed by {{{ .lake }}} | Reusable parameterized queries | +| External scalar UDF | Python UDF Server | One value for each input row | Hosted by you | Libraries, models, and proprietary services | +| External table UDF | Python UDF Server | Multiple columns or rows | Hosted by you | Tokenization, expansion, and record generation | + +An aggregate UDF and a table UDF solve different problems. An aggregate UDF consumes multiple rows and returns one value. A table UDF returns a result set. {{{ .lake }}} does not provide a Python table aggregate UDF that combines both execution models. + +## Choose the simplest execution model + +Use the following order when selecting an implementation: + +1. Use a built-in function when one already provides the required behavior. +2. Use a SQL scalar or table UDF when SQL can express the logic clearly. +3. Use an embedded Python or JavaScript scalar UDF for script logic that should run inside {{{ .lake }}}. +4. Use WebAssembly for compute-intensive scalar logic delivered as a compiled module. +5. Use an aggregate UDF when the calculation requires custom aggregation state. +6. Use an external UDF when the logic depends on remote services, GPUs, or independent scaling. + +The following questions can help narrow the choice: + +| Question | Recommended option | +| --- | --- | +| Can one SQL expression produce the result? | SQL scalar UDF | +| Does row-level logic need Python or JavaScript? | Script scalar UDF | +| Is a compiled, portable module required for compute-intensive row-level logic? | WebAssembly scalar UDF | +| Does a SQL query need to return multiple rows? | SQL table UDF | +| Must multiple input rows be combined into one custom result? | Aggregate UDF | +| Does one input need to produce multiple Python-generated rows? | External table UDF | +| Does the logic require a Python package, model, network call, or separate compute? | External scalar or table UDF | + +## Use SQL scalar UDFs for reusable transformations + +A SQL scalar UDF maps each input row to one value. It is a good fit for calculations, string normalization, and conditional business rules. + +### Normalize phone numbers + +The following function removes formatting characters so that downstream queries use one phone number representation: + +```sql +CREATE FUNCTION normalize_phone(phone VARCHAR) +RETURNS VARCHAR +AS $$ REGEXP_REPLACE(phone, '[^0-9]', '') $$; + +SELECT normalize_phone('+1 (415) 555-0100'); +``` + +### Apply a discount + +The following function centralizes a discount calculation: + +```sql +CREATE FUNCTION apply_discount( + price DECIMAL(10, 2), + rate DECIMAL(5, 2) +) +RETURNS DECIMAL(10, 2) +AS $$ price * (1 - rate) $$; + +SELECT apply_discount(100, 0.15); +``` + +Use [ALTER FUNCTION](/tidb-cloud-lake/sql/alter-function.md) when the shared business rule changes. Queries that call the function then use the new definition without duplicating the expression. + +For complete syntax, see [CREATE SCALAR FUNCTION](/tidb-cloud-lake/sql/create-scalar-function.md). + +## Use Python scalar UDFs for data processing logic + +Python scalar UDFs are useful when the logic needs control flow, the Python standard library, or a package that is awkward to express in SQL. + +The following function standardizes whitespace and capitalization in an address: + +```sql +CREATE FUNCTION normalize_address(value VARCHAR) +RETURNS VARCHAR +LANGUAGE python +HANDLER = 'normalize_address' +AS $$ +def normalize_address(value): + return " ".join(value.strip().upper().split()) +$$; + +SELECT normalize_address(' 123 Main Street '); +``` + +Python UDFs can also use `PACKAGES` for PyPI dependencies and `IMPORTS` for files stored in a stage. Keep dependencies focused so that function environments remain easier to reproduce and maintain. + +## Use JavaScript scalar UDFs for JSON transformations + +JavaScript is a natural fit for object and JSON transformations, especially when the logic already exists in an application codebase. + +The following function normalizes an email address and removes a sensitive field: + +```sql +CREATE FUNCTION clean_profile(value VARIANT) +RETURNS VARIANT +LANGUAGE javascript +HANDLER = 'cleanProfile' +AS $$ +export function cleanProfile(value) { + const result = { ...value }; + if (typeof result.email === 'string') { + result.email = result.email.trim().toLowerCase(); + } + delete result.ssn; + return result; +} +$$; +``` + +Keep the input and return schema stable. A change in the object shape can affect every query that calls the function. + +## Use WebAssembly UDFs for compiled logic + +WebAssembly UDFs package compiled code as a portable module. They are suitable for compute-intensive scalar logic where a compiled implementation is preferable to a script runtime. + +Upload a module that implements the required Arrow UDF interface to a stage, and then register its handler: + +```sql +CREATE FUNCTION fib_wasm(value INT) +RETURNS INT +LANGUAGE wasm +HANDLER = 'fib' +AS $$ @my_wasm_stage/arrow_udf_example.wasm $$; + +SELECT fib_wasm(10); +``` + +The module must export the named handler and use SQL-compatible input and output types. Test the compiled artifact with representative values before publishing the function to other users. + +## Use aggregate UDFs for custom stateful calculations + +An aggregate UDF defines how to: + +1. create an initial aggregation state; +2. add each input row to the state; +3. merge partial states produced by distributed execution; +4. convert the final state into one result. + +The following Python aggregate adds values. Built-in `SUM` is preferable for this specific calculation, but the example shows the lifecycle required by a custom aggregate: + +```sql +CREATE FUNCTION py_total(value BIGINT) +STATE { total BIGINT } +RETURNS BIGINT +LANGUAGE python +AS $$ +class State: + def __init__(self): + self.total = 0 + +def create_state(): + return State() + +def accumulate(state, value): + state.total += value + return state + +def merge(left, right): + left.total += right.total + return left + +def finish(state): + return state.total +$$; + +SELECT py_total(number) FROM numbers(5); +``` + +Aggregate UDFs support Python and JavaScript. Use them only when built-in aggregate functions cannot express the required state transition or finalization logic. For more examples, see [CREATE AGGREGATE FUNCTION](/tidb-cloud-lake/sql/create-aggregate-function.md). + +## Use SQL table UDFs for reusable result sets + +A SQL table UDF encapsulates a SQL query and returns rows and columns. It is useful for reusable filters, small reporting datasets, and parameterized transformations. + +```sql +CREATE FUNCTION small_numbers(max_value INT) +RETURNS TABLE(value UINT64, doubled UINT64) +AS $$ + SELECT number AS value, number * 2 AS doubled + FROM numbers(10) + WHERE number < max_value +$$; + +SELECT * FROM small_numbers(3); +``` + +The function body is a SQL query. It does not accept `LANGUAGE python`. For Python-generated rows, use an external table UDF. + +For complete syntax, see [CREATE TABLE FUNCTION](/tidb-cloud-lake/sql/create-table-function.md). + +## Use external Python UDFs for specialized logic + +The [`tidbcloudlake-udf`](https://pypi.org/project/tidbcloudlake-udf/) package provides an Apache Arrow Flight server for external Python scalar and table UDFs. The Python process runs on your infrastructure, which lets you use custom packages, proprietary code, GPU compute, and independent scaling. + +### Normalize addresses with Python + +Install the SDK: + +```shell +python3 -m pip install tidbcloudlake-udf +``` + +Define a handler and start the server: + +```python +from tidbcloudlake_udf import UDFServer, udf + + +@udf( + input_types=["VARCHAR"], + result_type="VARCHAR", + skip_null=True, +) +def normalize_address(value: str) -> str: + return " ".join(value.strip().upper().split()) + + +if __name__ == "__main__": + server = UDFServer("0.0.0.0:8815") + server.add_function(normalize_address) + server.serve() +``` + +After deploying and allowlisting the server, register the handler: + +```sql +CREATE FUNCTION normalize_address(value VARCHAR) +RETURNS VARCHAR +LANGUAGE python +HANDLER = 'normalize_address' +ADDRESS = 'https://udf.example.com'; +``` + +### Expand text into rows + +An external table UDF uses a list of output columns in `result_type`: + +```python +@udf( + input_types=["VARCHAR"], + result_type=[("token", "VARCHAR")], + skip_null=True, +) +def split_words(value: str): + return [{"token": token} for token in value.split()] +``` + +Register and call the table handler: + +```sql +CREATE FUNCTION split_words(value VARCHAR) +RETURNS TABLE(token VARCHAR) +LANGUAGE python +HANDLER = 'split_words' +ADDRESS = 'https://udf.example.com'; + +SELECT * FROM split_words('external UDF server'); +``` + +For a complete server, deployment, concurrency, and registration workflow, see [CREATE FUNCTION](/tidb-cloud-lake/sql/create-function.md). + +## Deploy external UDFs securely + +External functions communicate through Apache Arrow Flight over gRPC/HTTP2, not through a REST endpoint. + +Before registering an external function: + +- Deploy the UDF Server at a public HTTPS endpoint that supports gRPC over HTTP/2. +- Contact TiDB Cloud Support to add the endpoint hostname to your tenant UDF server allowlist. +- Configure authentication at the gateway, capacity, timeouts, high availability, upgrades, and monitoring. +- Keep credentials in the server deployment environment instead of SQL function definitions. + +The SQL `ADDRESS` must contain the public endpoint. The server process can listen on `0.0.0.0` inside its deployment environment, but `localhost` and `0.0.0.0` are not valid addresses for a Cloud query service to call. + +External UDFs add network latency. Keep latency-sensitive row-by-row calls small, batch work when possible, and avoid calling the same expensive function repeatedly in one query. + +## Compare performance and operations + +Performance depends on function complexity, input size, package startup, warehouse resources, network latency, batch size, and UDF Server capacity. Results measured in another product or deployment do not predict Lake performance. + +| UDF type | Main overhead | Operational responsibility | +| --- | --- | --- | +| SQL scalar UDF | SQL expression evaluation | Managed by {{{ .lake }}} | +| Python or JavaScript scalar UDF | Script runtime and dependency initialization | Managed by {{{ .lake }}} | +| WebAssembly scalar UDF | Module loading and compiled function execution | Managed by {{{ .lake }}} | +| Aggregate UDF | Script runtime and state serialization | Managed by {{{ .lake }}} | +| SQL table UDF | Query execution | Managed by {{{ .lake }}} | +| External UDF | Network transfer and external compute | Shared between {{{ .lake }}} and your UDF Server deployment | + +Benchmark the actual function with representative data and concurrency. Measure query latency, throughput, error handling, cold starts, and external service saturation. + +## Follow UDF best practices + +- Prefer built-in functions and SQL before introducing a script or service. +- Keep each function deterministic and focused when possible. +- Define NULL behavior explicitly and test nullable inputs. +- Use precise input and return types to avoid unnecessary conversions. +- For aggregate UDFs, make `merge` associative so partial states can be combined safely. +- For external UDFs, use `batch_mode` for batch-oriented libraries and `io_threads` for I/O-bound row processing. +- Set `max_concurrency` to protect external dependencies from overload. +- Treat external handler changes like service API changes and deploy them compatibly with registered SQL definitions. +- Monitor errors, latency, saturation, and dependency health for user-hosted servers. +- Remove unused UDF registrations and server handlers together. + +## Get started + +Choose the next step based on the required output: + +- [CREATE SCALAR FUNCTION](/tidb-cloud-lake/sql/create-scalar-function.md) for reusable SQL expressions. +- [CREATE AGGREGATE FUNCTION](/tidb-cloud-lake/sql/create-aggregate-function.md) for custom Python or JavaScript aggregation state. +- [CREATE TABLE FUNCTION](/tidb-cloud-lake/sql/create-table-function.md) for reusable SQL result sets. +- [CREATE FUNCTION](/tidb-cloud-lake/sql/create-function.md) for external Python scalar and table handlers. +- [External AI Functions](/tidb-cloud-lake/guides/external-ai-functions.md) for a model inference example. + +## Related resources + +- [User-Defined Function](/tidb-cloud-lake/sql/user-defined-function.md) +- [External Function](/tidb-cloud-lake/sql/external-function.md) +- [`tidbcloud/lake-udf` on GitHub](https://github.com/tidbcloud/lake-udf) +- [Deep Dive into Databend UDF](https://www.databend.com/blog/category-product/Databend_UDF/), the source article adapted and technically revalidated for this guide diff --git a/tidb-cloud-lake/guides/external-ai-functions.md b/tidb-cloud-lake/guides/external-ai-functions.md index 7f547a31895fe..89f1c766ec603 100644 --- a/tidb-cloud-lake/guides/external-ai-functions.md +++ b/tidb-cloud-lake/guides/external-ai-functions.md @@ -1,78 +1,98 @@ --- title: External AI Functions -summary: Build powerful AI/ML capabilities by connecting {{{ .lake }}} with your own infrastructure. External functions let you deploy custom models, leverage GPU acceleration, and integrate with any ML framework while keeping your data secure. +summary: Learn how to serve a Python embedding model through an external UDF Server and call it from TiDB Cloud Lake SQL queries. --- # External AI Functions -Build powerful AI/ML capabilities by connecting {{{ .lake }}} with your own infrastructure. External functions let you deploy custom models, leverage GPU acceleration, and integrate with any ML framework while keeping your data secure. +External UDFs let {{{ .lake }}} queries call models and Python libraries that run on your infrastructure. You can deploy the UDF Server on CPU or GPU compute and scale it independently from the warehouse. -## Key Capabilities +This example exposes a text embedding model as a scalar function that returns a `VECTOR` value. -| Feature | Benefits | -|---------|----------| -| **Custom Models** | Use any open-source or proprietary AI/ML models | -| **GPU Acceleration** | Deploy on GPU-equipped machines for faster inference | -| **Data Privacy** | Keep your data within your infrastructure | -| **Scalability** | Independent scaling and resource optimization | -| **Flexibility** | Support for any programming language and ML framework | +## Prerequisites -## How It Works +- Python 3.10 or later for the model and UDF Server +- A public HTTPS endpoint that supports gRPC over HTTP/2 +- The endpoint hostname added to your tenant UDF server allowlist by TiDB Cloud Support +- A table containing text to embed -1. **Create AI Server**: Build your AI/ML server using Python and [databend-udf](https://pypi.org/project/databend-udf) -2. **Register Function**: Connect your server to {{{ .lake }}} with `CREATE FUNCTION` -3. **Use in SQL**: Call your custom AI functions directly in SQL queries +## Step 1. Install the dependencies -## Example: Text Embedding Function +```shell +python3 -m venv .venv +source .venv/bin/activate +python3 -m pip install tidbcloudlake-udf sentence-transformers +``` + +## Step 2. Create the embedding handler + +Create `embedding_server.py`: ```python -# Simple embedding UDF server demo -from databend_udf import udf, UDFServer from sentence_transformers import SentenceTransformer +from tidbcloudlake_udf import UDFServer, udf + + +model = SentenceTransformer("sentence-transformers/all-mpnet-base-v2") -# Load pre-trained model -model = SentenceTransformer('all-mpnet-base-v2') # 768-dimensional vectors @udf( - input_types=["STRING"], - result_type="ARRAY(FLOAT)", + input_types=["VARCHAR"], + result_type="VECTOR(768)", + skip_null=True, ) -def ai_embed_768(inputs: list[str], headers) -> list[list[float]]: - """Generate 768-dimensional embeddings for input texts""" - try: - # Process inputs in a single batch - embeddings = model.encode(inputs) - # Convert to list format - return [embedding.tolist() for embedding in embeddings] - except Exception as e: - print(f"Error generating embeddings: {e}") - # Return empty lists in case of error - return [[] for _ in inputs] - -if __name__ == '__main__': - print("Starting embedding UDF server on port 8815...") +def embed_text(value: str) -> list[float]: + embedding = model.encode(value) + return embedding.astype("float32").tolist() + + +if __name__ == "__main__": server = UDFServer("0.0.0.0:8815") - server.add_function(ai_embed_768) + server.add_function(embed_text) server.serve() ``` +Start the server: + +```shell +python3 embedding_server.py +``` + +## Step 3. Deploy and allow the endpoint + +Deploy the Flight server behind a public HTTPS endpoint that preserves gRPC over HTTP/2. Configure authentication, scaling, high availability, and monitoring for the service. + +Contact TiDB Cloud Support to add the endpoint hostname to your tenant UDF server allowlist. The server can bind to `0.0.0.0` inside its deployment, but the SQL `ADDRESS` must use the public hostname. + +## Step 4. Register the function + +```sql +CREATE FUNCTION embed_text(value VARCHAR) +RETURNS VECTOR(768) +LANGUAGE python +HANDLER = 'embed_text' +ADDRESS = 'https://udf.example.com'; +``` + +## Step 5. Use embeddings in a query + ```sql --- Register the external function in {{{ .lake }}} -CREATE OR REPLACE FUNCTION ai_embed_768 (STRING) - RETURNS ARRAY(FLOAT) - LANGUAGE PYTHON - HANDLER = 'ai_embed_768' - ADDRESS = 'https://your-ml-server.example.com'; - --- Use the custom embedding in queries SELECT id, title, - cosine_distance( - ai_embed_768(content), - ai_embed_768('machine learning techniques') - ) AS similarity + COSINE_DISTANCE( + embedding, + embed_text('machine learning techniques') + ) AS distance FROM articles -ORDER BY similarity ASC +ORDER BY distance LIMIT 5; ``` + +External model calls add network and inference latency. Measure representative query concurrency and batch sizes before placing the function in a production query path. + +## Related resources + +- [CREATE FUNCTION](/tidb-cloud-lake/sql/create-function.md) +- [Choose a User-Defined Function Type](/tidb-cloud-lake/guides/choose-a-udf-type.md) +- [`tidbcloudlake-udf` on PyPI](https://pypi.org/project/tidbcloudlake-udf/) diff --git a/tidb-cloud-lake/sql/alter-function-sql.md b/tidb-cloud-lake/sql/alter-function-sql.md index ea830febf5c57..abfa711eb5a6f 100644 --- a/tidb-cloud-lake/sql/alter-function-sql.md +++ b/tidb-cloud-lake/sql/alter-function-sql.md @@ -1,39 +1,46 @@ --- title: ALTER FUNCTION -summary: Alters an external function. +summary: Learn how to change the handler, return schema, description, or HTTPS UDF Server endpoint of an external function in TiDB Cloud Lake. --- # ALTER FUNCTION -Alters an external function. +The `ALTER FUNCTION` statement changes an external function registration. -## Syntax +## Scalar function syntax ```sql -ALTER FUNCTION [ IF NOT EXISTS ] - AS ( ) RETURNS LANGUAGE - HANDLER = '' ADDRESS = '' - [DESC=''] +ALTER FUNCTION [ IF EXISTS ] + ( [] ) + RETURNS + LANGUAGE python + HANDLER = '' + ADDRESS = '' + [ DESC='' ] ``` -| Parameter | Description | -|-----------------------|---------------------------------------------------------------------------------------------------| -| `` | The name of the function. | -| `` | The lambda expression or code snippet defining the function's behavior. | -| `DESC=''` | Description of the UDF.| -| `<`| A list of input parameter names. Separated by comma.| -| `<`| A list of input parameter types. Separated by comma.| -| `` | The return type of the function. | -| `LANGUAGE` | Specifies the language used to write the function. Available values: `python`. | -| `HANDLER = ''` | Specifies the name of the function's handler. | -| `ADDRESS = ''` | Specifies the address of the UDF server. | - -## Examples +## Table function syntax ```sql --- Create an external function -CREATE FUNCTION gcd (INT, INT) RETURNS INT LANGUAGE python HANDLER = 'gcd' ADDRESS = 'http://0.0.0.0:8815'; +ALTER FUNCTION [ IF EXISTS ] + ( [] ) + RETURNS TABLE ( ) + LANGUAGE python + HANDLER = '' + ADDRESS = '' + [ DESC='' ] +``` + +The endpoint hostname must be in your tenant UDF server allowlist. --- Modify the handler of the external function -ALTER FUNCTION gcd (INT, INT) RETURNS INT LANGUAGE python HANDLER = 'gcd_new' ADDRESS = 'http://0.0.0.0:8815'; +## Example + +```sql +ALTER FUNCTION external_add(left INT, right INT) +RETURNS BIGINT +LANGUAGE python +HANDLER = 'add_bigint' +ADDRESS = 'https://udf.example.com'; ``` + +To change a SQL scalar or table UDF, use [ALTER FUNCTION for UDFs](/tidb-cloud-lake/sql/alter-function.md). diff --git a/tidb-cloud-lake/sql/create-function.md b/tidb-cloud-lake/sql/create-function.md index d4fbf70baf49a..a3d7d1d9bafd1 100644 --- a/tidb-cloud-lake/sql/create-function.md +++ b/tidb-cloud-lake/sql/create-function.md @@ -38,16 +38,16 @@ This example walks through a complete end-to-end setup for an external function ### Step 1: Set Up the Python UDF Server -Install the `databend-udf` package: +Install the `tidbcloudlake-udf` package: ```bash -pip install databend-udf +pip install tidbcloudlake-udf ``` Create a file `udf_server.py` with the following content: ```python -from databend_udf import udf, UDFServer +from tidbcloudlake_udf import udf, UDFServer @udf( input_types=["INT", "INT"], @@ -71,6 +71,8 @@ Start the server: python udf_server.py ``` +Deploy the UDF Server behind a public HTTPS endpoint that supports Apache Arrow Flight over gRPC/HTTP2. Contact TiDB Cloud Support to add the endpoint hostname to your tenant UDF server allowlist. + ### Step 2: Register the Function in {{{ .lake }}} ```sql @@ -78,7 +80,7 @@ CREATE FUNCTION gcd AS (INT, INT) RETURNS INT LANGUAGE python HANDLER = 'gcd' - ADDRESS = 'http://localhost:8815'; + ADDRESS = 'https://udf.example.com'; ``` ### Step 3: Call the Function diff --git a/tidb-cloud-lake/sql/drop-function-sql.md b/tidb-cloud-lake/sql/drop-function-sql.md index 56cf360a59c07..9b48c6f1d7fda 100644 --- a/tidb-cloud-lake/sql/drop-function-sql.md +++ b/tidb-cloud-lake/sql/drop-function-sql.md @@ -1,11 +1,11 @@ --- title: DROP FUNCTION -summary: Drops an external function. +summary: Learn how to remove an external scalar or table function registration from TiDB Cloud Lake and verify that it is no longer callable. --- # DROP FUNCTION -Drops an external function. +Removes an external scalar or table function registration. This statement does not stop or delete the external UDF Server. ## Syntax diff --git a/tidb-cloud-lake/sql/external-function.md b/tidb-cloud-lake/sql/external-function.md index 96cd38d496388..f9f41a0e03be5 100644 --- a/tidb-cloud-lake/sql/external-function.md +++ b/tidb-cloud-lake/sql/external-function.md @@ -1,19 +1,49 @@ --- title: External Function -summary: This page provides a comprehensive overview of External Function operations in {{{ .lake }}}, organized by functionality for easy reference. +summary: Learn how TiDB Cloud Lake calls independently hosted Python scalar and table UDFs through Apache Arrow Flight over gRPC and HTTP/2. --- # External Function -This page provides a comprehensive overview of External Function operations in {{{ .lake }}}, organized by functionality for easy reference. +External functions let SQL queries call Python logic that runs on infrastructure you operate. The [`tidbcloudlake-udf`](https://pypi.org/project/tidbcloudlake-udf/) package provides a UDF Server based on Apache Arrow Flight. -## External Function Management +External functions are suitable for Python libraries, model inference, proprietary business logic, GPU workloads, and compute that must scale independently from a warehouse. + +## How external functions work + +1. You define scalar or table handlers with the `tidbcloudlake_udf.udf` decorator. +2. A `UDFServer` exposes the handlers through Arrow Flight over gRPC/HTTP2. +3. You deploy the server behind a public HTTPS endpoint. +4. TiDB Cloud Support adds the endpoint hostname to your tenant UDF server allowlist. +5. You register each handler with `CREATE FUNCTION` and call it from SQL. + +The UDF Server endpoint is not configured in the Lake DSN. The SQL `ADDRESS` identifies the server that the query service calls. + +## Supported functions + +The Python SDK supports: + +- scalar UDFs that return one value for each input row; +- table UDFs that return multiple columns or rows; +- scalar and complex SQL data types, including arrays, maps, tuples, variants, and vectors; +- NULL handling, batch processing, I/O threads, request cancellation, and per-function concurrency limits. + +The SDK does not implement aggregate UDF state. Use [CREATE AGGREGATE FUNCTION](/tidb-cloud-lake/sql/create-aggregate-function.md) for a custom aggregation managed by {{{ .lake }}}. + +## Network and operational requirements + +- The public endpoint must use HTTPS and support gRPC over HTTP/2. +- Contact TiDB Cloud Support to add the endpoint hostname to the tenant UDF server allowlist before running `CREATE FUNCTION`. +- The server process can bind to `0.0.0.0` inside its deployment environment, but SQL `ADDRESS` must use a hostname that {{{ .lake }}} can reach. +- You are responsible for endpoint authentication, capacity, high availability, monitoring, upgrades, and the Python dependencies used by handlers. + +## Management commands | Command | Description | -|---------|-------------| -| [ALTER EXTERNAL FUNCTION](/tidb-cloud-lake/sql/alter-function.md) | Modifies an existing external function | -| [DROP EXTERNAL FUNCTION](/tidb-cloud-lake/sql/drop-function.md) | Removes an external function | +| --- | --- | +| [CREATE FUNCTION](/tidb-cloud-lake/sql/create-function.md) | Registers an external scalar or table handler. | +| [ALTER FUNCTION](/tidb-cloud-lake/sql/alter-function-sql.md) | Changes an external function registration. | +| [DROP FUNCTION](/tidb-cloud-lake/sql/drop-function-sql.md) | Removes an external function registration. | +| [SHOW USER FUNCTIONS](/tidb-cloud-lake/sql/show-user-functions.md) | Lists registered functions. | -> **Note:** -> -> External Functions in {{{ .lake }}} allow you to extend functionality by integrating with external services through HTTP/HTTPS endpoints, enabling you to leverage external processing capabilities. +For an AI inference example, see [External AI Functions](/tidb-cloud-lake/guides/external-ai-functions.md). From 2e5731e9519f25c43e7d972da8d61e95e47e5a73 Mon Sep 17 00:00:00 2001 From: awxxxxxx Date: Mon, 17 Aug 2026 14:39:43 +0800 Subject: [PATCH 2/4] lake: minimize existing UDF doc changes --- .../guides/external-ai-functions.md | 118 ++++++++---------- tidb-cloud-lake/sql/alter-function-sql.md | 55 ++++---- tidb-cloud-lake/sql/drop-function-sql.md | 4 +- tidb-cloud-lake/sql/external-function.md | 51 ++------ 4 files changed, 87 insertions(+), 141 deletions(-) diff --git a/tidb-cloud-lake/guides/external-ai-functions.md b/tidb-cloud-lake/guides/external-ai-functions.md index 89f1c766ec603..8af27bf85b987 100644 --- a/tidb-cloud-lake/guides/external-ai-functions.md +++ b/tidb-cloud-lake/guides/external-ai-functions.md @@ -1,98 +1,80 @@ --- title: External AI Functions -summary: Learn how to serve a Python embedding model through an external UDF Server and call it from TiDB Cloud Lake SQL queries. +summary: Build powerful AI/ML capabilities by connecting {{{ .lake }}} with your own infrastructure. External functions let you deploy custom models, leverage GPU acceleration, and integrate with any ML framework while keeping your data secure. --- # External AI Functions -External UDFs let {{{ .lake }}} queries call models and Python libraries that run on your infrastructure. You can deploy the UDF Server on CPU or GPU compute and scale it independently from the warehouse. +Build powerful AI/ML capabilities by connecting {{{ .lake }}} with your own infrastructure. External functions let you deploy custom models, leverage GPU acceleration, and integrate with any ML framework while keeping your data secure. -This example exposes a text embedding model as a scalar function that returns a `VECTOR` value. +## Key Capabilities -## Prerequisites +| Feature | Benefits | +|---------|----------| +| **Custom Models** | Use any open-source or proprietary AI/ML models | +| **GPU Acceleration** | Deploy on GPU-equipped machines for faster inference | +| **Data Privacy** | Keep your data within your infrastructure | +| **Scalability** | Independent scaling and resource optimization | +| **Flexibility** | Support for any programming language and ML framework | -- Python 3.10 or later for the model and UDF Server -- A public HTTPS endpoint that supports gRPC over HTTP/2 -- The endpoint hostname added to your tenant UDF server allowlist by TiDB Cloud Support -- A table containing text to embed +## How It Works -## Step 1. Install the dependencies +1. **Create AI Server**: Build your AI/ML server using Python and [`tidbcloudlake-udf`](https://pypi.org/project/tidbcloudlake-udf/) +2. **Register Function**: Connect your server to {{{ .lake }}} with `CREATE FUNCTION` +3. **Use in SQL**: Call your custom AI functions directly in SQL queries -```shell -python3 -m venv .venv -source .venv/bin/activate -python3 -m pip install tidbcloudlake-udf sentence-transformers -``` - -## Step 2. Create the embedding handler +External UDF Servers communicate with {{{ .lake }}} through Apache Arrow Flight over gRPC/HTTP2. Deploy the server behind a public HTTPS endpoint, and contact TiDB Cloud Support to add the endpoint hostname to your tenant UDF server allowlist. -Create `embedding_server.py`: +## Example: Text Embedding Function ```python +# Simple embedding UDF server demo +from tidbcloudlake_udf import udf, UDFServer from sentence_transformers import SentenceTransformer -from tidbcloudlake_udf import UDFServer, udf - - -model = SentenceTransformer("sentence-transformers/all-mpnet-base-v2") +# Load pre-trained model +model = SentenceTransformer('all-mpnet-base-v2') # 768-dimensional vectors @udf( - input_types=["VARCHAR"], - result_type="VECTOR(768)", - skip_null=True, + input_types=["STRING"], + result_type="ARRAY(FLOAT)", ) -def embed_text(value: str) -> list[float]: - embedding = model.encode(value) - return embedding.astype("float32").tolist() - - -if __name__ == "__main__": +def ai_embed_768(inputs: list[str], headers) -> list[list[float]]: + """Generate 768-dimensional embeddings for input texts""" + try: + # Process inputs in a single batch + embeddings = model.encode(inputs) + # Convert to list format + return [embedding.tolist() for embedding in embeddings] + except Exception as e: + print(f"Error generating embeddings: {e}") + # Return empty lists in case of error + return [[] for _ in inputs] + +if __name__ == '__main__': + print("Starting embedding UDF server on port 8815...") server = UDFServer("0.0.0.0:8815") - server.add_function(embed_text) + server.add_function(ai_embed_768) server.serve() ``` -Start the server: - -```shell -python3 embedding_server.py -``` - -## Step 3. Deploy and allow the endpoint - -Deploy the Flight server behind a public HTTPS endpoint that preserves gRPC over HTTP/2. Configure authentication, scaling, high availability, and monitoring for the service. - -Contact TiDB Cloud Support to add the endpoint hostname to your tenant UDF server allowlist. The server can bind to `0.0.0.0` inside its deployment, but the SQL `ADDRESS` must use the public hostname. - -## Step 4. Register the function - -```sql -CREATE FUNCTION embed_text(value VARCHAR) -RETURNS VECTOR(768) -LANGUAGE python -HANDLER = 'embed_text' -ADDRESS = 'https://udf.example.com'; -``` - -## Step 5. Use embeddings in a query - ```sql +-- Register the external function in {{{ .lake }}} +CREATE OR REPLACE FUNCTION ai_embed_768 (STRING) + RETURNS ARRAY(FLOAT) + LANGUAGE PYTHON + HANDLER = 'ai_embed_768' + ADDRESS = 'https://your-ml-server.example.com'; + +-- Use the custom embedding in queries SELECT id, title, - COSINE_DISTANCE( - embedding, - embed_text('machine learning techniques') - ) AS distance + cosine_distance( + ai_embed_768(content), + ai_embed_768('machine learning techniques') + ) AS similarity FROM articles -ORDER BY distance +ORDER BY similarity ASC LIMIT 5; ``` - -External model calls add network and inference latency. Measure representative query concurrency and batch sizes before placing the function in a production query path. - -## Related resources - -- [CREATE FUNCTION](/tidb-cloud-lake/sql/create-function.md) -- [Choose a User-Defined Function Type](/tidb-cloud-lake/guides/choose-a-udf-type.md) -- [`tidbcloudlake-udf` on PyPI](https://pypi.org/project/tidbcloudlake-udf/) diff --git a/tidb-cloud-lake/sql/alter-function-sql.md b/tidb-cloud-lake/sql/alter-function-sql.md index abfa711eb5a6f..232c68bde3908 100644 --- a/tidb-cloud-lake/sql/alter-function-sql.md +++ b/tidb-cloud-lake/sql/alter-function-sql.md @@ -1,46 +1,39 @@ --- title: ALTER FUNCTION -summary: Learn how to change the handler, return schema, description, or HTTPS UDF Server endpoint of an external function in TiDB Cloud Lake. +summary: Alters an external function. --- # ALTER FUNCTION -The `ALTER FUNCTION` statement changes an external function registration. +Alters an external function. -## Scalar function syntax +## Syntax ```sql -ALTER FUNCTION [ IF EXISTS ] - ( [] ) - RETURNS - LANGUAGE python - HANDLER = '' - ADDRESS = '' - [ DESC='' ] +ALTER FUNCTION [ IF NOT EXISTS ] + AS ( ) RETURNS LANGUAGE + HANDLER = '' ADDRESS = '' + [DESC=''] ``` -## Table function syntax +| Parameter | Description | +|-----------------------|---------------------------------------------------------------------------------------------------| +| `` | The name of the function. | +| `` | The lambda expression or code snippet defining the function's behavior. | +| `DESC=''` | Description of the UDF.| +| `<`| A list of input parameter names. Separated by comma.| +| `<`| A list of input parameter types. Separated by comma.| +| `` | The return type of the function. | +| `LANGUAGE` | Specifies the language used to write the function. Available values: `python`. | +| `HANDLER = ''` | Specifies the name of the function's handler. | +| `ADDRESS = ''` | Specifies the address of the UDF server. | -```sql -ALTER FUNCTION [ IF EXISTS ] - ( [] ) - RETURNS TABLE ( ) - LANGUAGE python - HANDLER = '' - ADDRESS = '' - [ DESC='' ] -``` - -The endpoint hostname must be in your tenant UDF server allowlist. - -## Example +## Examples ```sql -ALTER FUNCTION external_add(left INT, right INT) -RETURNS BIGINT -LANGUAGE python -HANDLER = 'add_bigint' -ADDRESS = 'https://udf.example.com'; -``` +-- Create an external function +CREATE FUNCTION gcd (INT, INT) RETURNS INT LANGUAGE python HANDLER = 'gcd' ADDRESS = 'https://udf.example.com'; -To change a SQL scalar or table UDF, use [ALTER FUNCTION for UDFs](/tidb-cloud-lake/sql/alter-function.md). +-- Modify the handler of the external function +ALTER FUNCTION gcd (INT, INT) RETURNS INT LANGUAGE python HANDLER = 'gcd_new' ADDRESS = 'https://udf.example.com'; +``` diff --git a/tidb-cloud-lake/sql/drop-function-sql.md b/tidb-cloud-lake/sql/drop-function-sql.md index 9b48c6f1d7fda..56cf360a59c07 100644 --- a/tidb-cloud-lake/sql/drop-function-sql.md +++ b/tidb-cloud-lake/sql/drop-function-sql.md @@ -1,11 +1,11 @@ --- title: DROP FUNCTION -summary: Learn how to remove an external scalar or table function registration from TiDB Cloud Lake and verify that it is no longer callable. +summary: Drops an external function. --- # DROP FUNCTION -Removes an external scalar or table function registration. This statement does not stop or delete the external UDF Server. +Drops an external function. ## Syntax diff --git a/tidb-cloud-lake/sql/external-function.md b/tidb-cloud-lake/sql/external-function.md index f9f41a0e03be5..8b06a88f92d7b 100644 --- a/tidb-cloud-lake/sql/external-function.md +++ b/tidb-cloud-lake/sql/external-function.md @@ -1,49 +1,20 @@ --- title: External Function -summary: Learn how TiDB Cloud Lake calls independently hosted Python scalar and table UDFs through Apache Arrow Flight over gRPC and HTTP/2. +summary: This page provides a comprehensive overview of External Function operations in {{{ .lake }}}, organized by functionality for easy reference. --- # External Function -External functions let SQL queries call Python logic that runs on infrastructure you operate. The [`tidbcloudlake-udf`](https://pypi.org/project/tidbcloudlake-udf/) package provides a UDF Server based on Apache Arrow Flight. +This page provides a comprehensive overview of External Function operations in {{{ .lake }}}, organized by functionality for easy reference. -External functions are suitable for Python libraries, model inference, proprietary business logic, GPU workloads, and compute that must scale independently from a warehouse. - -## How external functions work - -1. You define scalar or table handlers with the `tidbcloudlake_udf.udf` decorator. -2. A `UDFServer` exposes the handlers through Arrow Flight over gRPC/HTTP2. -3. You deploy the server behind a public HTTPS endpoint. -4. TiDB Cloud Support adds the endpoint hostname to your tenant UDF server allowlist. -5. You register each handler with `CREATE FUNCTION` and call it from SQL. - -The UDF Server endpoint is not configured in the Lake DSN. The SQL `ADDRESS` identifies the server that the query service calls. - -## Supported functions - -The Python SDK supports: - -- scalar UDFs that return one value for each input row; -- table UDFs that return multiple columns or rows; -- scalar and complex SQL data types, including arrays, maps, tuples, variants, and vectors; -- NULL handling, batch processing, I/O threads, request cancellation, and per-function concurrency limits. - -The SDK does not implement aggregate UDF state. Use [CREATE AGGREGATE FUNCTION](/tidb-cloud-lake/sql/create-aggregate-function.md) for a custom aggregation managed by {{{ .lake }}}. - -## Network and operational requirements - -- The public endpoint must use HTTPS and support gRPC over HTTP/2. -- Contact TiDB Cloud Support to add the endpoint hostname to the tenant UDF server allowlist before running `CREATE FUNCTION`. -- The server process can bind to `0.0.0.0` inside its deployment environment, but SQL `ADDRESS` must use a hostname that {{{ .lake }}} can reach. -- You are responsible for endpoint authentication, capacity, high availability, monitoring, upgrades, and the Python dependencies used by handlers. - -## Management commands +## External Function Management | Command | Description | -| --- | --- | -| [CREATE FUNCTION](/tidb-cloud-lake/sql/create-function.md) | Registers an external scalar or table handler. | -| [ALTER FUNCTION](/tidb-cloud-lake/sql/alter-function-sql.md) | Changes an external function registration. | -| [DROP FUNCTION](/tidb-cloud-lake/sql/drop-function-sql.md) | Removes an external function registration. | -| [SHOW USER FUNCTIONS](/tidb-cloud-lake/sql/show-user-functions.md) | Lists registered functions. | - -For an AI inference example, see [External AI Functions](/tidb-cloud-lake/guides/external-ai-functions.md). +|---------|-------------| +| [CREATE EXTERNAL FUNCTION](/tidb-cloud-lake/sql/create-function.md) | Creates a new external function | +| [ALTER EXTERNAL FUNCTION](/tidb-cloud-lake/sql/alter-function-sql.md) | Modifies an existing external function | +| [DROP EXTERNAL FUNCTION](/tidb-cloud-lake/sql/drop-function-sql.md) | Removes an external function | + +> **Note:** +> +> External Functions in {{{ .lake }}} allow you to extend functionality by integrating with external services through Apache Arrow Flight over gRPC/HTTP2. From 8d9cbe24a3885d4c8ffc13843b44e576d960fa02 Mon Sep 17 00:00:00 2001 From: awxxxxxx Date: Mon, 17 Aug 2026 14:48:38 +0800 Subject: [PATCH 3/4] lake: remove Databend references --- tidb-cloud-lake/guides/choose-a-udf-type.md | 1 - 1 file changed, 1 deletion(-) diff --git a/tidb-cloud-lake/guides/choose-a-udf-type.md b/tidb-cloud-lake/guides/choose-a-udf-type.md index f72acaeb88219..0ed8bfa248f0a 100644 --- a/tidb-cloud-lake/guides/choose-a-udf-type.md +++ b/tidb-cloud-lake/guides/choose-a-udf-type.md @@ -351,4 +351,3 @@ Choose the next step based on the required output: - [User-Defined Function](/tidb-cloud-lake/sql/user-defined-function.md) - [External Function](/tidb-cloud-lake/sql/external-function.md) - [`tidbcloud/lake-udf` on GitHub](https://github.com/tidbcloud/lake-udf) -- [Deep Dive into Databend UDF](https://www.databend.com/blog/category-product/Databend_UDF/), the source article adapted and technically revalidated for this guide From 5151ebea48ee6d17cc3eb79446b8bca0dda28143 Mon Sep 17 00:00:00 2001 From: awxxxxxx Date: Mon, 17 Aug 2026 14:57:43 +0800 Subject: [PATCH 4/4] lake: keep UDF endpoint docs minimal --- tidb-cloud-lake/guides/choose-a-udf-type.md | 6 ++---- tidb-cloud-lake/guides/external-ai-functions.md | 2 -- tidb-cloud-lake/sql/create-function.md | 2 -- tidb-cloud-lake/sql/external-function.md | 2 +- 4 files changed, 3 insertions(+), 9 deletions(-) diff --git a/tidb-cloud-lake/guides/choose-a-udf-type.md b/tidb-cloud-lake/guides/choose-a-udf-type.md index 0ed8bfa248f0a..64e61317a3004 100644 --- a/tidb-cloud-lake/guides/choose-a-udf-type.md +++ b/tidb-cloud-lake/guides/choose-a-udf-type.md @@ -224,7 +224,7 @@ For complete syntax, see [CREATE TABLE FUNCTION](/tidb-cloud-lake/sql/create-tab ## Use external Python UDFs for specialized logic -The [`tidbcloudlake-udf`](https://pypi.org/project/tidbcloudlake-udf/) package provides an Apache Arrow Flight server for external Python scalar and table UDFs. The Python process runs on your infrastructure, which lets you use custom packages, proprietary code, GPU compute, and independent scaling. +The [`tidbcloudlake-udf`](https://pypi.org/project/tidbcloudlake-udf/) package provides a Python UDF Server for external scalar and table UDFs. The Python process runs on your infrastructure, which lets you use custom packages, proprietary code, GPU compute, and independent scaling. ### Normalize addresses with Python @@ -295,11 +295,9 @@ For a complete server, deployment, concurrency, and registration workflow, see [ ## Deploy external UDFs securely -External functions communicate through Apache Arrow Flight over gRPC/HTTP2, not through a REST endpoint. - Before registering an external function: -- Deploy the UDF Server at a public HTTPS endpoint that supports gRPC over HTTP/2. +- Deploy the UDF Server at a public HTTPS endpoint. - Contact TiDB Cloud Support to add the endpoint hostname to your tenant UDF server allowlist. - Configure authentication at the gateway, capacity, timeouts, high availability, upgrades, and monitoring. - Keep credentials in the server deployment environment instead of SQL function definitions. diff --git a/tidb-cloud-lake/guides/external-ai-functions.md b/tidb-cloud-lake/guides/external-ai-functions.md index 8af27bf85b987..70329a75336bb 100644 --- a/tidb-cloud-lake/guides/external-ai-functions.md +++ b/tidb-cloud-lake/guides/external-ai-functions.md @@ -23,8 +23,6 @@ Build powerful AI/ML capabilities by connecting {{{ .lake }}} with your own infr 2. **Register Function**: Connect your server to {{{ .lake }}} with `CREATE FUNCTION` 3. **Use in SQL**: Call your custom AI functions directly in SQL queries -External UDF Servers communicate with {{{ .lake }}} through Apache Arrow Flight over gRPC/HTTP2. Deploy the server behind a public HTTPS endpoint, and contact TiDB Cloud Support to add the endpoint hostname to your tenant UDF server allowlist. - ## Example: Text Embedding Function ```python diff --git a/tidb-cloud-lake/sql/create-function.md b/tidb-cloud-lake/sql/create-function.md index a3d7d1d9bafd1..052f601303c86 100644 --- a/tidb-cloud-lake/sql/create-function.md +++ b/tidb-cloud-lake/sql/create-function.md @@ -71,8 +71,6 @@ Start the server: python udf_server.py ``` -Deploy the UDF Server behind a public HTTPS endpoint that supports Apache Arrow Flight over gRPC/HTTP2. Contact TiDB Cloud Support to add the endpoint hostname to your tenant UDF server allowlist. - ### Step 2: Register the Function in {{{ .lake }}} ```sql diff --git a/tidb-cloud-lake/sql/external-function.md b/tidb-cloud-lake/sql/external-function.md index 8b06a88f92d7b..18c011b261e3e 100644 --- a/tidb-cloud-lake/sql/external-function.md +++ b/tidb-cloud-lake/sql/external-function.md @@ -17,4 +17,4 @@ This page provides a comprehensive overview of External Function operations in { > **Note:** > -> External Functions in {{{ .lake }}} allow you to extend functionality by integrating with external services through Apache Arrow Flight over gRPC/HTTP2. +> External Functions in {{{ .lake }}} allow you to extend functionality by integrating with external services through HTTP/HTTPS endpoints, enabling you to leverage external processing capabilities.