Skip to content

[doc] documentation for gravitino integration#3669

Open
VladBanar wants to merge 2 commits into
apache:mainfrom
VladBanar:doc-for-gravitino
Open

[doc] documentation for gravitino integration#3669
VladBanar wants to merge 2 commits into
apache:mainfrom
VladBanar:doc-for-gravitino

Conversation

@VladBanar

@VladBanar VladBanar commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Generated-by: Claude Code (Sonnet 4.5) following the guidelines

Purpose

Linked issue: close #3512

Adds a quick-start guide for Apache Gravitino as an Iceberg REST catalog for Fluss lakehouse tiering.

Brief change log

New page catalogs/gravitino.md:

Tests

Documentation-only. Built the Docusaurus site locally (npm run start) — no broken links.

@VladBanar

Copy link
Copy Markdown
Contributor Author

@MehulBatra Hi, please take a look. Sorry for the delay, it was my first time configuring tiered storage

@VladBanar VladBanar changed the title doc for gravitino integration [doc] documentation for gravitino integration Jul 16, 2026
SET 'execution.runtime-mode' = 'batch';

-- Union read: combines fresh data in Fluss with historical data in Iceberg
SELECT COUNT(*) FROM orders;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add example of how we can assure data is tiered to lake, by only querying the data from lake?

}'
```

> **NOTE**: The `credential-providers: s3-secret-key` property enables credential vending. Gravitino will provide temporary S3 credentials to Fluss instead of requiring static credentials in Fluss configuration.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The doc currently states that s3-secret-key provides temporary S3 credentials, retrieves credentials dynamically, and therefore requires no static credentials. That isn't how this provider works.

s3-secret-key vends the static access key and secret configured on the catalog. In the Gravitino implementation, S3SecretKeyCredential.expireTimeInMs() returns 0 (and asserts the expiry is 0), so the credentials are not temporary. The benefit is that Fluss doesn't need the credentials configured locally—they are retrieved from Gravitino over the REST channel—but they are still long-lived static credentials.

Only s3-token (STS AssumeRole) vends short-lived, scoped credentials.

I'd suggest either:

  • Switch the example to s3-token if the goal is to demonstrate temporary credentials.
  • Keep s3-secret-key, but reword it to something like: "Gravitino returns the S3 credentials to Fluss, so they don't need to be configured in Fluss." It would also be helpful to add a note that s3-token is the recommended option when temporary, scoped credentials are required.

I think this distinction is important, especially since readers may interpret the current wording as a security property that s3-secret-key doesn't actually provide.


```yaml
# Optional: pass additional REST catalog properties
datalake.iceberg.header.Authorization: Bearer <token>

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The example config points to http://<host>:9001/iceberg/, and authentication is only mentioned as an optional Authorization: Bearer header.

It would be helpful to explicitly call out the default security posture. Out of the box, the Iceberg REST service does not enforce authentication, so the endpoint is accessible unless server-side authentication is configured. A short note (or a link to the relevant documentation) on securing the endpoint would help prevent users from unintentionally exposing it.

This is separate from credential vending, but the two are easy to conflate, so making the distinction explicit would improve the documentation.


```properties
gravitino.iceberg-rest.catalog-config-provider = dynamic-config-provider
gravitino.iceberg-rest.gravitino-uri = http://localhost:8090

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The doc correctly uses :8090 for the Gravitino server API (where the metalake and catalog are created) and :9001 for the Iceberg REST service (which Fluss connects to). It also correctly relies on the metalake being configured server-side in gravitino.conf rather than in Fluss.

That said, the flow isn't immediately obvious because these details are introduced in different places. It would help to add a brief sentence up front, for example:

"You create the catalog through the Gravitino server API on :8090. Fluss then communicates only with the Iceberg REST service on :9001, which resolves the catalog using the metalake configured in gravitino.conf."

A small clarification like this would make the overall request flow much easier to follow before readers get into the configuration examples.

@MehulBatra

Copy link
Copy Markdown
Contributor

@VladBanar did you get a chance to run the entire setup end-to-endstarting Gravitino with the Iceberg REST Catalog, tiering data from Fluss, and then reading it back through Flink SQL?
If not, I'd strongly encourage doing that before we merge. It will give us much higher confidence that the documented flow works as expected and help catch any integration issues early.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[docs] Add Apache Gravitino/Iceberg Rest catalog quick-start guide for Fluss-Iceberg integration

2 participants