Skip to content

[python] Support OSS blob presigned URLs - #10040

Draft
XiaoHongbo-Hope wants to merge 6 commits into
apache:masterfrom
XiaoHongbo-Hope:codex/python-blob-presigned-url
Draft

XiaoHongbo-Hope wants to merge 6 commits into
apache:masterfrom
XiaoHongbo-Hope:codex/python-blob-presigned-url

Conversation

@XiaoHongbo-Hope

@XiaoHongbo-Hope XiaoHongbo-Hope commented Sep 21, 2026

Copy link
Copy Markdown
Contributor

Purpose

Add the PyPaimon counterpart of the Java OSS BLOB presigning API introduced in #8832.

This change:

  • adds Blob.to_presigned_url(file_io, table_root, validity) and a FileIO extension point;
  • materializes the exact descriptor byte range in OSS with the Java-compatible object key, content type, and descriptor fingerprint metadata;
  • reuses valid materialized objects and creates a fresh V4-signed GET URL;
  • preserves configured OSS server-side encryption for empty and multipart materialized objects, and rejects cached objects with incompatible encryption metadata;
  • supports both standard OSS URIs and legacy credential URIs without treating the legacy bucket path segment as part of the object key;
  • supports static credentials and STS credentials supplied by DLF REST catalogs;
  • forwards the operation through resolving, caching, and REST token FileIO wrappers;
  • validates table ownership, ranges, validity, HTTPS endpoints, and the generated URL target;
  • adds a copy-paste DLF quick start covering catalog configuration, table creation, MP4 BLOB writes, descriptor reads, presigning, ECS roles, and a Model Studio video_url request.

Tests

  • 422 passed, 15 skipped, 61 subtests passed across the new tests and related BLOB/FileIO suites.
  • AES256 and KMS unit coverage verifies multipart initialization, zero-length uploads, cache rejection/reuse, and legacy URI object keys.
  • Flake8 and git diff --check passed for all changed Python files.
  • Docusaurus production build passed, including REST OpenAPI validation.
  • Ran an AES256 integration test against a real DLF catalog: created a table, wrote a 1 MiB BLOB, fetched the signed URL with HTTP 200, verified the returned bytes and x-oss-server-side-encryption: AES256, and removed the table.
  • Previously ran the Python block extracted from the quick start against a real DLF catalog: created the table, wrote a 1.1 MB MP4, read its descriptor, fetched the signed URL with HTTP 200, and received HTTP 200 from DashScope qwen3.8-flash.
  • Removed the integration-test DLF tables, materialized OSS objects, and local files.

@XiaoHongbo-Hope
XiaoHongbo-Hope marked this pull request as draft September 21, 2026 04:33
@XiaoHongbo-Hope
XiaoHongbo-Hope force-pushed the codex/python-blob-presigned-url branch from 00bdad1 to 8532201 Compare September 21, 2026 04:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant