[Bug](lance) REST catalog against Gravitino: namespace-vended credentials are dropped, and the root database is not listable

[FANNG1]

Search before asking

• I had searched in the issues and found no similar issues.

Version

branch-4.1 @ 0e53b31f586. The FE Lance catalog currently exists only on branch-4.1 — master has no fe/fe-core/src/main/java/org/apache/doris/datasource/lance/. LanceStorageOptions came in with e3289c1 (#65730); the REST properties were reshaped by 8375559 (#66581).

Server side: Apache Gravitino 1.3.0, native lance-rest auxiliary service (gravitino.auxService.names = lance-rest, gravitino.lance-rest.httpPort = 9101, namespace-backend = gravitino), base path /lance.

Namespace library versions differ across the wire but are not the cause of either problem below: Gravitino bundles lance-namespace-core 0.4.5, while Doris FE pulls lance-namespace-apache-client 0.7.7 transitively from org.lance:lance-core:9.1.0-beta.3.

What's Wrong?

I ran the Lance REST catalog against a real Lance Namespace server (Gravitino) rather than the Python stub in docker/thirdparties/docker-compose/iceberg/scripts/lance_rest_server.py, and hit two independent problems.

The protocol itself lines up — endpoint paths, the $ delimiter, pagination, managed_versioning, and the table_uri/location fallback all match, and once both problems are worked around every read path passes (see Anything else?). These are integration defects, not a protocol mismatch.

---

Problem 1 — namespace-vended storage credentials are silently dropped (blocking)

Credential vending is the main reason to prefer a REST catalog over a filesystem one, and Doris explicitly asks for it — LanceExternalCatalog.describeTable sends .vendCredentials(true) for REST catalogs.

Gravitino answers with unprefixed object-store option names (it stores them as lance.storage.<key> and strips that prefix in LancePropertiesUtils.resolveLanceStorageOptions):

"storage_options": {
  "access_key_id":     "minioadmin",
  "secret_access_key": "minioadmin",
  "endpoint":          "http://127.0.0.1:9000",
  "region":            "us-east-1",
  "allow_http":        "true"
}

But LanceStorageOptions.forBackend only ever looks up five aws_-prefixed names:

// fe/fe-core/src/main/java/org/apache/doris/datasource/lance/LanceStorageOptions.java
S3_KEYS.put("AWS_ACCESS_KEY", "aws_access_key_id");
S3_KEYS.put("AWS_SECRET_KEY", "aws_secret_access_key");
S3_KEYS.put("AWS_TOKEN",      "aws_session_token");
S3_KEYS.put("AWS_ENDPOINT",   "aws_endpoint");
S3_KEYS.put("AWS_REGION",     "aws_region");
...
S3_KEYS.forEach((dorisKey, lanceKey) -> putIfNotEmpty(result, dorisKey,
        lanceStorageOptions.get(lanceKey)));

Nothing matches, so every vended credential is discarded before it reaches the BE.

The resulting failure is split across FE and BE, which makes it hard to read. The FE merges the vended map in raw and hands it to the Lance Java SDK, whose object-store layer accepts the unprefixed spelling — so all metadata operations succeed. Only the BE, which goes through forBackend, ends up with nothing:

DESC   lance_novend.doris_probe.rest_probe;   -- OK, 5 columns
SELECT count(*) FROM lance_novend.doris_probe.rest_probe;

ERROR 1105 (HY000): errCode = 2, detailMessage = (127.0.0.1)[INTERNAL_ERROR]
open Lance dataset failed: LanceError(IO): Generic N/A error: ...
Failed to get AWS credentials: CredentialsNotLoaded(CredentialsNotLoaded {
  source: Some("no providers in chain provided credentials") }),
.../lance-io/src/object_store/providers/aws.rs:401:21

Positive control isolating the key spelling. I created a second Gravitino catalog whose properties spell the same values lance.storage.aws_access_key_id, lance.storage.aws_secret_access_key, lance.storage.aws_endpoint, lance.storage.aws_region, pointed it at the same physical dataset, and read it through the same Doris configuration (still no static credentials). The scan returned all 1024 rows. Key spelling was the only variable.

Both spellings are valid for the Rust object_store config that ultimately consumes them, and the FE's success is direct evidence that the unprefixed form works — only Doris's own BE-side converter insists on one of the two.

---

Problem 2 — the parent namespace is published as a database that cannot be listed

LanceExternalCatalog.listDatabaseNames unconditionally publishes the configured root database:

// The configured root database represents the empty relative Lance namespace.
LinkedHashSet<String> databases = new LinkedHashSet<>();
databases.add(rootDatabase);

so default always appears in SHOW DATABASES. It maps to the parent namespace itself, which is one level deep — but Gravitino's ListTables requires an identifier of exactly two levels (catalog + schema), so listing it fails:

SHOW TABLES FROM lance_gvt.`default`;
ERROR 1105 (HY000): errCode = 2, detailMessage = Invalid input: Expected 2-level namespace but got: 1

The same collision makes "test_connection" = "true" unusable, because the probe lists tables directly under the parent:

testNamespace.listTables(new ListTablesRequest().id(parent).limit(1));
testNamespace.listNamespaces(new ListNamespacesRequest().id(parent).limit(1));

With test_connection = true, CREATE CATALOG fails outright with the same message. Leaving it at its default of false is the workaround.

This is not a data-correctness problem — Doris loads databases lazily, so default is just an entry that must be routed around — but it does mean SHOW DATABASES advertises something unusable, and it silently rules out the connectivity check.

What You Expected?

1. Vended storage_options should reach the BE regardless of which accepted alias the namespace server uses for a key. A server that vends access_key_id should work exactly like one that vends aws_access_key_id — as it already does on the FE side.
2. A database returned by SHOW DATABASES should be listable, and test_connection = true should not require the parent namespace itself to be table-listable.

How to Reproduce?

1. Gravitino 1.3.0 with the Lance REST service, and a lakehouse-generic catalog carrying unprefixed storage options:

# conf/gravitino.conf
gravitino.auxService.names            = lance-rest
gravitino.lance-rest.classpath        = lance-rest-server/libs
gravitino.lance-rest.httpPort         = 9101
gravitino.lance-rest.namespace-backend = gravitino
gravitino.lance-rest.gravitino-uri    = http://127.0.0.1:8090
gravitino.lance-rest.gravitino-metalake = test

curl -X POST http://127.0.0.1:8090/api/metalakes/test/catalogs \
  -H 'Accept: application/vnd.gravitino.v1+json' -H 'Content-Type: application/json' \
  -d '{"name":"lance_catalog","type":"relational","provider":"lakehouse-generic","properties":{
       "location":"s3://contacts/raw/lance",
       "lance.storage.access_key_id":"minioadmin",
       "lance.storage.secret_access_key":"minioadmin",
       "lance.storage.endpoint":"http://127.0.0.1:9000",
       "lance.storage.region":"us-east-1",
       "lance.storage.allow_http":"true"}}'

curl -X POST 'http://127.0.0.1:9101/lance/v1/namespace/lance_catalog%24doris_probe/create?delimiter=%24' \
  -H 'Content-Type: application/json' \
  -d '{"id":["lance_catalog","doris_probe"],"mode":"CREATE","properties":{}}'

2. Write a dataset and register it (pylance 7.0.0):

import lance, numpy as np, pyarrow as pa
URI = "s3://contacts/raw/lance/doris_probe/rest_probe/"
SO = {"access_key_id": "minioadmin", "secret_access_key": "minioadmin",
      "endpoint": "http://127.0.0.1:9000", "region": "us-east-1",
      "allow_http": "true", "virtual_hosted_style_request": "false"}
rid = np.arange(1, 1025, dtype=np.int64)
emb = (rid[:, None] - 1).astype(np.float32) + np.arange(16, dtype=np.float32)[None, :]
t = pa.table({
    "row_id": pa.array(rid, type=pa.int64()),
    "int32_value": pa.array((rid % 100).astype(np.int32), type=pa.int32()),
    "embedding": pa.FixedSizeListArray.from_arrays(
        pa.array(emb.reshape(-1), type=pa.float32()), 16)})
lance.write_dataset(t, URI, mode="overwrite", storage_options=SO, max_rows_per_file=512)
lance.dataset(URI, storage_options=SO).create_index(
    "embedding", index_type="IVF_FLAT", metric="l2", num_partitions=4)

curl -X POST 'http://127.0.0.1:9101/lance/v1/table/lance_catalog%24doris_probe%24rest_probe/register?delimiter=%24' \
  -H 'Content-Type: application/json' \
  -d '{"id":["lance_catalog","doris_probe","rest_probe"],
       "location":"s3://contacts/raw/lance/doris_probe/rest_probe/"}'

3. Problem 1 — a catalog with no static credentials, relying entirely on vending:

CREATE CATALOG lance_novend PROPERTIES (
    "type"                   = "lance",
    "lance.catalog.type"     = "rest",
    "lance.rest.uri"         = "http://127.0.0.1:9101/lance",
    "lance.namespace.parent" = "lance_catalog"
);

DESC lance_novend.doris_probe.rest_probe;                    -- OK
SELECT count(*) FROM lance_novend.doris_probe.rest_probe;    -- CredentialsNotLoaded

4. Problem 2 — against the same catalog:

SHOW DATABASES FROM lance_novend;              -- lists `default`
SHOW TABLES FROM lance_novend.`default`;       -- Expected 2-level namespace but got: 1

and adding "test_connection" = "true" to the properties above makes CREATE CATALOG itself fail.

Anything Else?

Everything else works. With both problems worked around — static s3.access_key/s3.secret_key, and test_connection left at its default — the REST catalog reads Gravitino-managed Lance tables correctly. Verified against a deterministic fixture (1024 rows in 2 fragments, 16-dim float32 embedding where row n is [n-1, n, …, n+14], so the exact squared L2 distance between rows n and r is 16·(n−r)², plus an IVF_FLAT index over 4 partitions), cross-checked row-for-row against pylance:

Layer	Result
CREATE CATALOG, database/table discovery	ok
Type mapping (int64→bigint, string→text, int32→int, double→double, fixed_size_list<float,16>→array<float>)	ok
Full scan — 1024 rows, sum(row_id) = 524800	ok
Substrait predicate pushdown (lancePushdownPredicate), non-convertible expression left as BE residual	ok
Projection pruning	ok
IVF_FLAT vector search — full probe reproduces flat search row-for-row on the exact 16·d² ladder	ok
Global Top-K across fragments (row 1024 lives in the second fragment)	ok
FOR VERSION AS OF — after appending 10 rows: latest 1034, v2 1024, v1 1024	ok

Working configuration, for reference:

CREATE CATALOG lance_gvt PROPERTIES (
    "type"                   = "lance",
    "lance.catalog.type"     = "rest",
    "lance.rest.uri"         = "http://127.0.0.1:9101/lance",  -- Gravitino's /lance base path
    "lance.namespace.parent" = "lance_catalog",                -- Doris database == Gravitino schema
    "s3.endpoint"            = "http://127.0.0.1:9000",
    "s3.access_key"          = "minioadmin",
    "s3.secret_key"          = "minioadmin",
    "s3.region"              = "us-east-1",
    "use_path_style"         = "true"
);

Two notes for whoever picks this up:

1. lance.rest.uri has to carry the server's base path (/lance here). LanceRestMetastoreProperties.validateRestUri already permits a path component, and the client appends /v1/… to it, so this works — worth a line in the docs, since the in-repo stub is mounted at the root and never exercises it.
2. Test coverage has a hole shaped exactly like Problem 1. test_lance_rest_catalog.groovy does cover credential vending, but the stub it talks to vends aws_-prefixed keys (lance_rest_server.py returns aws_access_key_id, aws_secret_access_key, aws_region), so the unprefixed alias path — which is what a real server emits — is never exercised. Teaching the stub to vend the unprefixed spelling in one case would pin this.

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[Doris-Breakwater]

Breakwater-GitHub-Analysis-Slot: slot_13a9e214821c

This content is generated by AI for reference only.

Initial assessment: both reported behaviors are supported by the code at branch-4.1 commit 0e53b31f586. Problem 1 is a confirmed Doris credential-conversion defect and blocks REST credential vending. Problem 2 is a confirmed Gravitino interoperability defect, although its fix needs to preserve REST servers that legitimately allow tables directly under the configured parent namespace. This is not evidence of data corruption or a protocol-version mismatch.

The issue is currently open with no labels, assignee, milestone, or prior comments. It is unusually complete: the version, server version/configuration, REST response, error, reproduction, positive control, and workarounds are sufficient to start a fix. I did not run the full Doris/Gravitino deployment locally; the conclusions below come from the exact Doris source, the public Gravitino v1.3.0 source, and the reporter's controlled reproduction.

Verified facts

1. Vended credentials are lost only on the FE-to-BE conversion path.

   • LanceExternalCatalog.describeTable() requests credentials for REST catalogs.
   • loadTableMetadata() merges DescribeTableResponse.storage_options verbatim into the Java SDK options, explaining why FE metadata reads work.
   • The same method calls LanceStorageOptions.forBackend(), which recognizes only aws_access_key_id, aws_secret_access_key, aws_session_token, aws_endpoint, aws_region, and aws_virtual_hosted_style_request.
   • Gravitino v1.3.0's LancePropertiesUtils deliberately strips lance.storage. and therefore returns the configured unprefixed keys such as access_key_id, secret_access_key, endpoint, and region.
   • Unmatched vended values are absent from LanceTableMetadata; LanceScanNode.getLocationProperties() then sends only that reduced map to the BE. The BE lance_reader.cpp reconstructs Lance options only from normalized AWS_*/use_path_style properties, so an empty converted map reaches the credential provider. This exactly matches the reported FE-success/BE-failure split and the key-spelling positive control.

2. The unusable root database and failing connectivity check follow directly from the current namespace mapping.

   • listDatabaseNames() always publishes rootDatabase; that name decodes to an empty relative namespace and therefore maps to the configured parent itself.
   • listTableNamesFromRemote() lists tables against that parent. With lance.namespace.parent = lance_catalog, Doris sends a one-level identifier.
   • Gravitino v1.3.0 accepts a one-level identifier for listNamespaces() (catalog -> schemas) but requires exactly two levels for listTables() (catalog + schema), producing the reported error.
   • checkWhenCreating() calls listTables(parent) before listNamespaces(parent), so test_connection=true fails before performing the valid namespace probe.
   • The in-repo REST fixture does not expose this incompatibility: it permits tables at the root and vends only aws_*-prefixed credentials.

What remains a design decision

• The parent namespace is table-listable on the existing fixture and may be table-listable on other Lance Namespace implementations, so globally removing the root database would be a compatibility regression. Doris should either detect parent table-list capability and omit only an unsupported root, or introduce an explicit mapping policy. Whichever approach is chosen, SHOW DATABASES should not advertise an entry that immediately fails SHOW TABLES.
• The connectivity test should first use the operation valid at the configured parent (listNamespaces(parent)). If table-endpoint validation is retained, it should target a discovered child namespace when one exists rather than requiring tables to be legal directly under the parent.

Recommended next steps

1. Accept this as a branch-4.1 Lance REST catalog bug and add the repository's bug/component labels; the Lance FE code is not present on master at the reported/current revisions.
2. Normalize both accepted S3 key families in one place before building BE properties. Preserve deterministic precedence when both spellings are returned, keep vended values ahead of static catalog credentials, and cover the full temporary-credential set (including the session token) plus endpoint/region/path-style aliases supported by the pinned Lance object-store layer.
3. Add focused unit tests for LanceStorageOptions.forBackend() covering unprefixed keys, prefixed keys, conflicts, empty values, static fallback, and temporary credentials.
4. Extend the REST regression fixture with an unprefixed vending response and prove a scan succeeds with no static access/secret key. Add a Gravitino-shaped namespace case where listNamespaces([catalog]) succeeds, listTables([catalog]) is invalid, and listTables([catalog, schema]) succeeds; verify both catalog creation with test_connection=true and that every displayed database is listable. Keep a root-table-listable case to prevent compatibility regressions.
5. Document that lance.rest.uri may include the service base path (for example /lance). This is a documentation follow-up, not a blocker for the two code fixes.

No additional reporter logs or profile are required to begin implementation. The reporter has offered to submit a PR; maintainers mainly need to choose the root-database compatibility policy above and review the alias precedence/test matrix.

[Doris-Breakwater]

Breakwater-GitHub-Analysis-Slot: slot_13a9e214821c

Initial triage

Judgment: valid, high-confidence branch-4.1 Lance REST integration bugs. There are two independent FE-side compatibility defects: credential alias normalization drops Gravitino-vended options on the BE path, and Doris assumes that the configured parent namespace is itself table-listable. Problem 1 blocks scans that rely on credential vending; Problem 2 creates an unusable advertised database and rejects an otherwise reachable catalog when test_connection=true. The issue currently has no labels, assignee, milestone, or earlier comments, and the reporter is willing to submit a PR.

I checked the exact reported Doris commit (0e53b31f58674a4716f12cdab4c46f3cb607ed05) and the Apache Gravitino 1.3.0 tag (40fdf6ab96ac87b47e6d3e14e7c4dc0d815e68f0). I did not rerun the full MinIO/Gravitino fixture locally; the runtime A/B evidence below is reporter-supplied, while the failure mechanisms are source-confirmed.

Verified facts

1. Vended credentials are lost only on the FE-to-BE conversion path

• Doris explicitly requests credentials in describeTable. It merges the returned storage_options unchanged into the Java SDK options, but separately calls LanceStorageOptions.forBackend(...) for the scan path (lines 288-313). This explains why FE metadata loading can succeed while the BE scan fails.
• forBackend recognizes only aws_access_key_id, aws_secret_access_key, aws_session_token, aws_endpoint, aws_region, and aws_virtual_hosted_style_request. With no static credentials, Gravitino's unprefixed access_key_id, secret_access_key, endpoint, and region therefore contribute no BE properties.
• The BE does not receive the raw vended map. LanceScanNode.getLocationProperties() returns the converted map, and LanceTableReader::_storage_options reads only Doris's normalized AWS_*/use_path_style keys. There is no later opportunity for the unprefixed values to recover.
• Gravitino intentionally strips lance.storage. from catalog/table properties in LancePropertiesUtils, and its table-description path returns that resolved map as storage_options (source). Its tests explicitly expect access_key_id, endpoint, and region without aws_ prefixes.
• The reporter's positive control—same Doris catalog and physical dataset, changing only the vended key spelling to aws_*, after which all 1024 rows scan—matches this exact branch in the code. This is not explained by the Java namespace library version difference.

2. The root database and connectivity probe use an invalid Gravitino namespace depth

• listDatabaseNames unconditionally adds rootDatabase. The root name maps back to an empty relative namespace, so listTableNamesFromRemote calls ListTables on the configured parent itself.
• For lance.namespace.parent=lance_catalog, that request has one level. Gravitino's native backend accepts a one-level identifier for ListNamespaces and returns its schemas (source), but requires exactly two levels for ListTables (source). The reported error is therefore deterministic for this mapping.
• checkWhenCreating calls ListTables(parent) before ListNamespaces(parent), so test_connection=true fails even though the namespace endpoint and the schema-level table endpoints are reachable.

3. Existing tests model only the successful spellings and a permissive root

• The regression stub vends only aws_* keys (source), while the scan regression deliberately has no static access/secret key. Thus the test exercises vending but not Gravitino's spelling.
• The same stub allows ListTables at any namespace depth and places its default table directly at the empty/root namespace (source, list implementation). It therefore cannot reveal the fixed-depth Gravitino behavior.
• There is no focused FE unit test for LanceStorageOptions.forBackend; current coverage tests only the static Doris-to-Java direction.

Missing information / remaining decision

No additional version, log, profile, or reproduction information is required to accept and prioritize these two defects. The supplied versions, REST response, SQL reproduction, BE error, workaround, and spelling-only positive control are sufficient. A maintainer-side end-to-end run is still useful validation, but it is not a reporter blocker.

One implementation contract needs an explicit maintainer decision: generic Lance namespaces and the existing filesystem/stub behavior can have tables directly under the configured parent, while Gravitino's catalog/schema model cannot. Removing the root database unconditionally would regress the former; continuing to add it unconditionally leaves a phantom database for the latter. The fix should preserve both models rather than infer support solely from namespace depth.

Recommended next steps

1. Normalize vended aliases centrally. Extend forBackend (or a shared canonicalizer) to accept the unprefixed keys actually emitted by Gravitino as well as the existing aws_* forms, preserve the current precedence of vended values over static catalog defaults, and define deterministic precedence if both aliases are present. Cover endpoint/HTTP and addressing-style behavior, not only access/secret keys. Do not log credential values.
2. Do not require the configured parent to be table-listable for connectivity. Probe ListNamespaces(parent) first; if a child namespace exists, ListTables can be tested against that table-capable child. An empty catalog should still pass a namespace connectivity check. Do not catch and suppress arbitrary table-list failures, since that would hide authentication and service errors.
3. Make root exposure capability-aware or explicitly configurable. Include the root database only when tables directly under the configured parent are supported, or add a documented way to suppress it for catalog/schema backends. Keep the existing root behavior for filesystem/permissive namespaces. Whichever approach is chosen, every database returned by SHOW DATABASES must be listable.
4. Add interoperability coverage. Add a unit test for forBackend using Gravitino's unprefixed map and alias precedence; add a regression-stub mode that vends unprefixed keys with no static credentials; and add a fixed-depth mode where ListNamespaces([catalog]) succeeds but ListTables([catalog]) fails while ListTables([catalog,schema]) succeeds. Retain a root-table case to prevent regressions in the generic model.
5. Add a REST URI base-path case (for example /lance) to tests/docs, since the current URI validation already permits it but the root-mounted stub does not exercise it.

Suggested triage: add the repository's bug and Lance/external-catalog ownership labels, and let the willing reporter prepare the PR once the root-namespace contract is agreed. The code is currently branch-4.1-specific, so the target branch should follow the maintainers' branch policy.

[FANNG1]

Correction to the fix guidance above

Digging further into how Lance actually consumes storage_options, the suggestion in the issue body — teach LanceStorageOptions.forBackend to accept the unprefixed aliases — is necessary but not sufficient, and one of its premises was wrong. Everything below is verified against the exact Lance revision that lance-c v0.1.6 pins (f69324596), i.e. what the BE on branch-4.1 actually compiles, not a newer release.

1. The outbound spelling is wrong too, so an inbound-only fix still breaks

The BE emits aws_-prefixed option names (kStorageKeys in be/src/format_v2/table/lance_reader.cpp). Those silently stop working the moment Lance takes its OpenDAL path:

// rust/lance-io/src/object_store/providers/aws.rs  —  build_opendal_s3_store
let mut config_map: HashMap<String, String> = storage_options.0.clone();
config_map.insert("bucket".to_string(), bucket);
let operator = Operator::from_iter::<S3>(config_map)

There is no alias normalization on this path — the map goes straight into OpenDAL's S3Config, whose field names are unprefixed, and OpenDAL's deserializer ignores unknown keys without error.

	object_store path (default)	OpenDAL path (use_opendal=true)
access_key_id	works	works
aws_access_key_id	works	silently dropped

To be precise about the current blast radius: use_opendal is itself read from the storage options (aws.rs:139-143), and the same five-key filter drops it, so this path is not reachable through Doris today — the prefixed spelling is a latent hazard rather than a live bug. It becomes live exactly when the filter is opened up, which is what any real fix to this issue does. That is why it belongs in the same change: widening the inbound aliases while continuing to emit the prefixed form outbound would make use_opendal=true reachable and silently credential-less at the same moment.

The unprefixed short names are the only spelling both paths accept, so forJavaSdk and the BE's kStorageKeys are the place to converge.

2. Both spellings have first-party sources — the real principle is that the consumer must be permissive

The issue body implied Gravitino's unprefixed spelling is the more standard one. That is not accurate, and the correct argument is stronger.

Lance's own credential vendor emits the prefixed form:

// rust/lance-namespace-impls/src/credentials/aws.rs
storage_options.insert("aws_access_key_id".to_string(), access_key_id);
storage_options.insert("aws_session_token".to_string(), session_token);

while Lance's own consumer accepts both, prefixed first:

// rust/lance-io/src/object_store/dynamic_credentials.rs:172-186
.get("aws_access_key_id").or_else(|| credentials.0.get("access_key_id"))
.get("aws_session_token")
    .or_else(|| credentials.0.get("session_token"))
    .or_else(|| credentials.0.get("token"))

So both spellings legitimately occur in the wild from first-party sources, and Lance's answer is: vendors emit one, consumers accept all of them. Doris is a consumer that accepts one. That is the defect, and it holds regardless of which namespace server is on the other end — this is not a Gravitino-specific quirk.

Worth noting the spec is explicit that this field is opaque: "Configuration options to be used to access storage… These will be passed directly to Lance to initialize storage access." The namespace protocol deliberately does not define the key vocabulary, so a client cannot assume one.

3. The bigger loss: the filter also discards Lance's credential auto-refresh (corrected)

Corrected 2026-08-16 — the claim originally in this section was wrong. I wrote that
expires_at_millis drives a credential refresh that the five-key filter silently opts Doris out
of. It does not, on the path the BE actually uses.

Lance does have that machinery (LanceNamespaceStorageOptionsProvider / StorageOptionsAccessor
in rust/lance-io/src/object_store/storage_options.rs), but it is engaged by installing a
provider through DatasetBuilder::with_storage_options_provider. lance-c only ever calls
with_storage_options:

// lance-c v0.1.6, src/dataset.rs
builder = builder.with_storage_options(opts);

which builds a static accessor with no provider (rust/lance/src/dataset/builder.rs:305-317), so
nothing re-vends. as_s3_options then drops expires_at_millis outright, since it parses as
neither an S3 nor a client config key.

So: passing the key through is forward-looking, not load-bearing, and this is not an additional
loss caused by the filter. The underlying limitation is real but belongs elsewhere — short-lived
vended credentials do expire mid-scan on the BE, and fixing that needs lance-c to expose the
provider API, not a change to this filter.

The rest of this comment stands; items 1, 2 and 4 are unaffected.

4. The filter also locks the Lance catalog to S3

S3_KEYS (FE) and kStorageKeys (BE) are both S3-only, five entries each. Lance vends Azure and GCP credentials as well (azure_storage_sas_token, google_storage_token — credentials/azure.rs, credentials/gcp.rs). Against a namespace serving Lance tables on Azure or GCS, 100% of the vended credentials are dropped — not a spelling mismatch, simply no entry in the table. As written, the Lance catalog cannot support a non-S3 backend through credential vending at all.

5. Revised suggestion

Rather than widening the lookup table, stop translating server-supplied options at all:

• FE — pass DescribeTable's storage_options through untranslated.
• BE — hand them to lance-c as-is and drop the second translation. Note today's round trip is access_key_id → AWS_ACCESS_KEY → aws_access_key_id: Lance vocabulary out, Lance vocabulary back in, with a lossy filter in between and no information gained.
• Static catalog properties (s3.access_key and friends) keep their conversion — they genuinely are in Doris vocabulary — but should emit the unprefixed spelling.

On the safety concern that presumably motivated an allowlist in the first place: Lance's own model for exactly this problem is a denylist of location-controlling keys, not an allowlist of credential keys.

// rust/lance-io/src/object_store/dynamic_opendal.rs
protected_keys: Vec<&'static str>,   // stripped from server-supplied options

That defends the thing actually worth defending — a namespace server redirecting where data is read from — while scaling to other clouds and to keys added later, which an allowlist cannot.

If a smaller change is preferred for now, widening forBackend's aliases still fixes the immediate breakage and is worth doing on its own; it just leaves items 1, 3 and 4 open.

Testing note

The coverage gap noted in the issue body is a little wider than described: lance_rest_server.py vends only aws_-prefixed keys, so the unprefixed alias path is not exercised anywhere today.