Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
eb8909f
Add hostmetadata package with caching resolver skeleton
simonfaltum Apr 17, 2026
fa9b174
Fix Task 1 spec deviations
simonfaltum Apr 17, 2026
9856973
Isolate hostmetadata test from user cache directory
simonfaltum Apr 17, 2026
2678fe3
Add hostmetadata cache hit/miss tests
simonfaltum Apr 17, 2026
b71f298
Add hostmetadata negative-cache and host-isolation tests
simonfaltum Apr 17, 2026
a8fc7d1
Attach hostmetadata cache to root workspace/account clients
simonfaltum Apr 17, 2026
6d9981c
Isolate CLI cache dir in test cleanup environment
simonfaltum Apr 17, 2026
b23299a
Refactor hostmetadata.Attach to drop ctx requirement
simonfaltum Apr 17, 2026
609f044
Attach hostmetadata cache to bundle workspace config
simonfaltum Apr 17, 2026
62e71f2
Attach hostmetadata cache at remaining config construction sites
simonfaltum Apr 17, 2026
50a27ca
Replace sync.Once with eager cache init in hostmetadata.Attach
simonfaltum Apr 17, 2026
4937738
Attach hostmetadata cache at two missing login construction sites
simonfaltum Apr 17, 2026
2c9b1f3
Add guardrail test for hostmetadata.Attach injection sites
simonfaltum Apr 17, 2026
1a179c5
Add end-to-end acceptance test for host metadata caching
simonfaltum Apr 17, 2026
28127e5
Regenerate acceptance test expectations for hostmetadata caching
simonfaltum Apr 17, 2026
0ce5f13
Expose NewResolver to decouple caching from *config.Config
simonfaltum Apr 17, 2026
ec2bd69
Probe positive cache first, skip caching transient errors
simonfaltum Apr 17, 2026
8ce5516
Match sanitized dev version in test replacements
simonfaltum Apr 17, 2026
21fe664
Skip stat-not-found debug log on cache miss
simonfaltum Apr 17, 2026
2ac9b94
Merge branch 'main' into simonfaltum/hostmetadata-cache
simonfaltum Apr 21, 2026
fae41c6
Collapse per-site Attach wiring into a single factory registration
simonfaltum Apr 21, 2026
9f32fc8
Drop redundant factory-registration test
simonfaltum Apr 21, 2026
1ec8a58
Address codex review: cleaner negative-cache probe and no raw error t…
simonfaltum Apr 21, 2026
29d7389
Trim comment noise and a tautological test
simonfaltum Apr 21, 2026
f1c51d0
Merge branch 'main' into simonfaltum/hostmetadata-cache
simonfaltum Apr 21, 2026
83d7aa7
Merge remote-tracking branch 'origin/main' into simonfaltum/hostmetad…
simonfaltum Apr 22, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions NEXT_CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@

* Added `--limit` flag to all paginated list commands for client-side result capping ([#4984](https://github.com/databricks/cli/pull/4984)).
* Accept `yes` in addition to `y` for confirmation prompts, and show `[y/N]` to indicate that no is the default.
* Cache `/.well-known/databricks-config` lookups under `~/.cache/databricks/<version>/host-metadata/` so repeat CLI invocations against the same host skip the ~700ms discovery round trip.
* Deprecated `auth env`. The command is hidden from help listings and prints a deprecation warning to stderr; it will be removed in a future release.

### Bundles
Expand Down
2 changes: 0 additions & 2 deletions acceptance/auth/bundle_and_profile/output.txt
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,6 @@

=== Inside the bundle, profile flag not matching bundle host. Should use profile from the flag and not the bundle.
>>> errcode [CLI] current-user me -p profile_name
Warn: Failed to resolve host metadata: (redacted). Falling back to user config.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why did we lose the warning?

Is it because this is cached across different test runs?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

to answer my q: I see below that we just started hiding the warnings for this, so it's deliberate decision to always hide it (and won't cause flaky tests).

Still q remains about caching -- is it new cache for each test?

Error: Get "https://non.existing.subdomain.databricks.com/api/2.0/preview/scim/v2/Me": (redacted)

Exit code: 1
Expand Down Expand Up @@ -73,7 +72,6 @@ Validation OK!

=== Bundle commands load bundle configuration with -t and -p flag, validation not OK (profile host don't match bundle host)
>>> errcode [CLI] bundle validate -t prod -p DEFAULT
Warn: Failed to resolve host metadata: (redacted). Falling back to user config.
Error: cannot resolve bundle auth configuration: the host in the profile ([DATABRICKS_TARGET]) doesn’t match the host configured in the bundle (https://bar.com)

Name: test-auth
Expand Down
9 changes: 0 additions & 9 deletions acceptance/auth/credentials/unified-host/out.requests.txt
Original file line number Diff line number Diff line change
Expand Up @@ -22,15 +22,6 @@
"method": "GET",
"path": "/api/2.0/preview/scim/v2/Me"
}
{
"headers": {
"User-Agent": [
"cli/[DEV_VERSION] databricks-sdk-go/[SDK_VERSION] go/[GO_VERSION] os/[OS]"
]
},
"method": "GET",
"path": "/.well-known/databricks-config"
}
{
"headers": {
"Authorization": [
Expand Down
5 changes: 5 additions & 0 deletions acceptance/auth/host-metadata-cache/out.test.toml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

32 changes: 32 additions & 0 deletions acceptance/auth/host-metadata-cache/output.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@

=== First invocation populates the cache
{
"profiles": [
{
"name":"cached",
"host":"[DATABRICKS_URL]",
"cloud":"aws",
"auth_type":"",
"valid":false
}
]
}

=== Second invocation should read from the cache
{
"profiles": [
{
"name":"cached",
"host":"[DATABRICKS_URL]",
"cloud":"aws",
"auth_type":"",
"valid":false
}
]
}

=== Only one /.well-known/databricks-config request recorded
{
"method": "GET",
"path": "/.well-known/databricks-config"
}
19 changes: 19 additions & 0 deletions acceptance/auth/host-metadata-cache/script
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
sethome "./home"
export DATABRICKS_CACHE_DIR="$TEST_TMP_DIR/cache"

# Point a profile at the mock server so auth profiles triggers a host metadata
# fetch. Without a profile the command does nothing and the cache is never read.
cat > "./home/.databrickscfg" <<EOF
[cached]
host = ${DATABRICKS_HOST}
token = test-token
EOF

title "First invocation populates the cache\n"
$CLI auth profiles --skip-validate --output json || true
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what is || true for? if it fails it should be prefixed with musterr.


title "Second invocation should read from the cache\n"
$CLI auth profiles --skip-validate --output json || true

title "Only one /.well-known/databricks-config request recorded\n"
print_requests.py //.well-known/databricks-config --get --sort
5 changes: 5 additions & 0 deletions acceptance/auth/host-metadata-cache/test.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
Ignore = [
"home",
"cache",
]
RecordRequests = true
Original file line number Diff line number Diff line change
@@ -1,15 +1,7 @@
{
"method": "GET",
"path": "/.well-known/databricks-config"
}
{
"method": "GET",
"path": "/api/2.1/unity-catalog/volumes/main.myschema.myvolume"
}
{
"method": "GET",
"path": "/.well-known/databricks-config"
}
{
"method": "GET",
"path": "/api/2.1/unity-catalog/volumes/main.myschema.mynewvolume"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,10 +16,6 @@
"path": "/oidc/v1/token",
"raw_body": "grant_type=client_credentials\u0026scope=all-apis"
}
{
"method": "GET",
"path": "/.well-known/databricks-config"
}
{
"method": "GET",
"path": "/oidc/.well-known/oauth-authorization-server"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,6 @@
"method": "GET",
"path": "/.well-known/databricks-config"
}
{
"method": "GET",
"path": "/.well-known/databricks-config"
}
{
"headers": {
"Authorization": [
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,10 +16,6 @@
"path": "/oidc/v1/token",
"raw_body": "grant_type=client_credentials\u0026scope=all-apis"
}
{
"method": "GET",
"path": "/.well-known/databricks-config"
}
{
"method": "GET",
"path": "/oidc/.well-known/oauth-authorization-server"
Expand Down
10 changes: 8 additions & 2 deletions acceptance/cache/clear/output.txt
Original file line number Diff line number Diff line change
@@ -1,19 +1,25 @@

=== First call in a session is expected to be a cache miss:
[DEBUG_TIMESTAMP] Debug: [Local Cache] using cache key: [SHA256_HASH]
[DEBUG_TIMESTAMP] Debug: [Local Cache] failed to stat cache file: (redacted)
[DEBUG_TIMESTAMP] Debug: [Local Cache] cache miss, computing
[DEBUG_TIMESTAMP] Debug: [Local Cache] computed and stored result
[DEBUG_TIMESTAMP] Debug: [Local Cache] using cache key: [SHA256_HASH]
[DEBUG_TIMESTAMP] Debug: [Local Cache] cache miss, computing
[DEBUG_TIMESTAMP] Debug: [Local Cache] computed and stored result

=== Second call in a session is expected to be a cache hit
[DEBUG_TIMESTAMP] Debug: [Local Cache] using cache key: [SHA256_HASH]
[DEBUG_TIMESTAMP] Debug: [Local Cache] cache hit
[DEBUG_TIMESTAMP] Debug: [Local Cache] using cache key: [SHA256_HASH]
[DEBUG_TIMESTAMP] Debug: [Local Cache] cache hit

>>> [CLI] cache clear
Cache cleared successfully from [TEST_TMP_DIR]/.cache

=== First call after a clear is expected to be a cache miss:
[DEBUG_TIMESTAMP] Debug: [Local Cache] using cache key: [SHA256_HASH]
[DEBUG_TIMESTAMP] Debug: [Local Cache] failed to stat cache file: (redacted)
[DEBUG_TIMESTAMP] Debug: [Local Cache] cache miss, computing
[DEBUG_TIMESTAMP] Debug: [Local Cache] computed and stored result
[DEBUG_TIMESTAMP] Debug: [Local Cache] using cache key: [SHA256_HASH]
[DEBUG_TIMESTAMP] Debug: [Local Cache] cache miss, computing
[DEBUG_TIMESTAMP] Debug: [Local Cache] computed and stored result
6 changes: 5 additions & 1 deletion acceptance/cache/simple/output.txt
Original file line number Diff line number Diff line change
@@ -1,13 +1,17 @@

=== First call in a session is expected to be a cache miss:
[DEBUG_TIMESTAMP] Debug: [Local Cache] using cache key: [SHA256_HASH]
[DEBUG_TIMESTAMP] Debug: [Local Cache] failed to stat cache file: (redacted)
[DEBUG_TIMESTAMP] Debug: [Local Cache] cache miss, computing
[DEBUG_TIMESTAMP] Debug: [Local Cache] computed and stored result
[DEBUG_TIMESTAMP] Debug: [Local Cache] using cache key: [SHA256_HASH]
[DEBUG_TIMESTAMP] Debug: [Local Cache] cache miss, computing
[DEBUG_TIMESTAMP] Debug: [Local Cache] computed and stored result

=== Second call in a session is expected to be a cache hit
[DEBUG_TIMESTAMP] Debug: [Local Cache] using cache key: [SHA256_HASH]
[DEBUG_TIMESTAMP] Debug: [Local Cache] cache hit
[DEBUG_TIMESTAMP] Debug: [Local Cache] using cache key: [SHA256_HASH]
[DEBUG_TIMESTAMP] Debug: [Local Cache] cache hit

=== Bundle deploy should send telemetry values

Expand Down
1 change: 0 additions & 1 deletion acceptance/cmd/auth/profiles/output.txt
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@

=== Profiles with workspace_id (JSON output)
Warn: Failed to resolve host metadata: fetching host metadata from "https://test.cloud.databricks.com/.well-known/databricks-config": Get "https://test.cloud.databricks.com/.well-known/databricks-config": dial tcp: lookup test.cloud.databricks.com: no such host. Falling back to user config.
{
"profiles": [
{
Expand Down
4 changes: 0 additions & 4 deletions acceptance/cmd/workspace/apps/out.requests.txt
Original file line number Diff line number Diff line change
Expand Up @@ -25,10 +25,6 @@
"method": "GET",
"path": "/api/2.0/apps/test-name"
}
{
"method": "GET",
"path": "/.well-known/databricks-config"
}
{
"method": "PATCH",
"path": "/api/2.0/apps/test-name",
Expand Down
3 changes: 3 additions & 0 deletions acceptance/telemetry/failure/output.txt
Original file line number Diff line number Diff line change
@@ -1,12 +1,15 @@

>>> [CLI] selftest send-telemetry --debug
HH:MM:SS Info: start pid=PID version=[DEV_VERSION] args="[CLI], selftest, send-telemetry, --debug"
HH:MM:SS Debug: [Local Cache] using cache key: [SHA256_HASH] pid=PID
HH:MM:SS Debug: [Local Cache] cache miss, computing pid=PID
HH:MM:SS Debug: GET /.well-known/databricks-config
< HTTP/1.1 200 OK
< {
< "oidc_endpoint": "[DATABRICKS_URL]/oidc",
< "workspace_id": "[NUMID]"
< } pid=PID sdk=true
HH:MM:SS Debug: [Local Cache] computed and stored result pid=PID
HH:MM:SS Debug: Resolved workspace_id from host metadata: "[NUMID]" pid=PID sdk=true
HH:MM:SS Debug: Resolved cloud from hostname: "AWS" pid=PID sdk=true
HH:MM:SS Debug: Resolved discovery_url from host metadata: "[DATABRICKS_URL]/oidc/.well-known/oauth-authorization-server" pid=PID sdk=true
Expand Down
3 changes: 3 additions & 0 deletions acceptance/telemetry/partial-success/output.txt
Original file line number Diff line number Diff line change
@@ -1,12 +1,15 @@

>>> [CLI] selftest send-telemetry --debug
HH:MM:SS Info: start pid=PID version=[DEV_VERSION] args="[CLI], selftest, send-telemetry, --debug"
HH:MM:SS Debug: [Local Cache] using cache key: [SHA256_HASH] pid=PID
HH:MM:SS Debug: [Local Cache] cache miss, computing pid=PID
HH:MM:SS Debug: GET /.well-known/databricks-config
< HTTP/1.1 200 OK
< {
< "oidc_endpoint": "[DATABRICKS_URL]/oidc",
< "workspace_id": "[NUMID]"
< } pid=PID sdk=true
HH:MM:SS Debug: [Local Cache] computed and stored result pid=PID
HH:MM:SS Debug: Resolved workspace_id from host metadata: "[NUMID]" pid=PID sdk=true
HH:MM:SS Debug: Resolved cloud from hostname: "AWS" pid=PID sdk=true
HH:MM:SS Debug: Resolved discovery_url from host metadata: "[DATABRICKS_URL]/oidc/.well-known/oauth-authorization-server" pid=PID sdk=true
Expand Down
3 changes: 3 additions & 0 deletions acceptance/telemetry/skipped/output.txt
Original file line number Diff line number Diff line change
@@ -1,12 +1,15 @@

>>> [CLI] selftest send-telemetry --debug
HH:MM:SS Info: start pid=PID version=[DEV_VERSION] args="[CLI], selftest, send-telemetry, --debug"
HH:MM:SS Debug: [Local Cache] using cache key: [SHA256_HASH] pid=PID
HH:MM:SS Debug: [Local Cache] cache miss, computing pid=PID
HH:MM:SS Debug: GET /.well-known/databricks-config
< HTTP/1.1 200 OK
< {
< "oidc_endpoint": "[DATABRICKS_URL]/oidc",
< "workspace_id": "[NUMID]"
< } pid=PID sdk=true
HH:MM:SS Debug: [Local Cache] computed and stored result pid=PID
HH:MM:SS Debug: Resolved workspace_id from host metadata: "[NUMID]" pid=PID sdk=true
HH:MM:SS Debug: Resolved cloud from hostname: "AWS" pid=PID sdk=true
HH:MM:SS Debug: Resolved discovery_url from host metadata: "[DATABRICKS_URL]/oidc/.well-known/oauth-authorization-server" pid=PID sdk=true
Expand Down
3 changes: 3 additions & 0 deletions acceptance/telemetry/success/output.txt
Original file line number Diff line number Diff line change
@@ -1,12 +1,15 @@

>>> [CLI] selftest send-telemetry --debug
HH:MM:SS Info: start pid=PID version=[DEV_VERSION] args="[CLI], selftest, send-telemetry, --debug"
HH:MM:SS Debug: [Local Cache] using cache key: [SHA256_HASH] pid=PID
HH:MM:SS Debug: [Local Cache] cache miss, computing pid=PID
HH:MM:SS Debug: GET /.well-known/databricks-config
< HTTP/1.1 200 OK
< {
< "oidc_endpoint": "[DATABRICKS_URL]/oidc",
< "workspace_id": "[NUMID]"
< } pid=PID sdk=true
HH:MM:SS Debug: [Local Cache] computed and stored result pid=PID
HH:MM:SS Debug: Resolved workspace_id from host metadata: "[NUMID]" pid=PID sdk=true
HH:MM:SS Debug: Resolved cloud from hostname: "AWS" pid=PID sdk=true
HH:MM:SS Debug: Resolved discovery_url from host metadata: "[DATABRICKS_URL]/oidc/.well-known/oauth-authorization-server" pid=PID sdk=true
Expand Down
8 changes: 8 additions & 0 deletions acceptance/telemetry/test.toml
Original file line number Diff line number Diff line change
Expand Up @@ -36,3 +36,11 @@ New = "pid=PID"
[[Repls]]
Old = "\\([0-9]+ more bytes\\)"
New = "(N more bytes)"

# Host metadata cache keys vary per-test because the mock server URL changes.
# Normalize them so the golden output is stable across runs.
# Order=1 so it runs before the parent's `\d{14,}` → `[NUMID]` replacement.
[[Repls]]
Old = '[a-f0-9]{64}'
New = "[SHA256_HASH]"
Order = 1
3 changes: 3 additions & 0 deletions acceptance/telemetry/timeout/output.txt
Original file line number Diff line number Diff line change
@@ -1,12 +1,15 @@

>>> [CLI] selftest send-telemetry --debug
HH:MM:SS Info: start pid=PID version=[DEV_VERSION] args="[CLI], selftest, send-telemetry, --debug"
HH:MM:SS Debug: [Local Cache] using cache key: [SHA256_HASH] pid=PID
HH:MM:SS Debug: [Local Cache] cache miss, computing pid=PID
HH:MM:SS Debug: GET /.well-known/databricks-config
< HTTP/1.1 200 OK
< {
< "oidc_endpoint": "[DATABRICKS_URL]/oidc",
< "workspace_id": "[NUMID]"
< } pid=PID sdk=true
HH:MM:SS Debug: [Local Cache] computed and stored result pid=PID
HH:MM:SS Debug: Resolved workspace_id from host metadata: "[NUMID]" pid=PID sdk=true
HH:MM:SS Debug: Resolved cloud from hostname: "AWS" pid=PID sdk=true
HH:MM:SS Debug: Resolved discovery_url from host metadata: "[DATABRICKS_URL]/oidc/.well-known/oauth-authorization-server" pid=PID sdk=true
Expand Down
8 changes: 0 additions & 8 deletions acceptance/workspace/lakeview/publish/out.requests.txt
Original file line number Diff line number Diff line change
Expand Up @@ -9,10 +9,6 @@
"path": "/Users/[USERNAME]"
}
}
{
"method": "GET",
"path": "/.well-known/databricks-config"
}
{
"method": "POST",
"path": "/api/2.0/lakeview/dashboards",
Expand All @@ -22,10 +18,6 @@
"warehouse_id": "test-warehouse"
}
}
{
"method": "GET",
"path": "/.well-known/databricks-config"
}
{
"method": "POST",
"path": "/api/2.0/lakeview/dashboards/[DASHBOARD_ID]/published",
Expand Down
12 changes: 0 additions & 12 deletions acceptance/workspace/repos/create_with_provider/out.requests.txt
Original file line number Diff line number Diff line change
Expand Up @@ -11,18 +11,10 @@
"url": "https://github.com/databricks/databricks-empty-ide-project.git"
}
}
{
"method": "GET",
"path": "/.well-known/databricks-config"
}
{
"method": "GET",
"path": "/api/2.0/repos/[NUMID]"
}
{
"method": "GET",
"path": "/.well-known/databricks-config"
}
{
"method": "GET",
"path": "/api/2.0/workspace/get-status",
Expand All @@ -34,10 +26,6 @@
"method": "GET",
"path": "/api/2.0/repos/[NUMID]"
}
{
"method": "GET",
"path": "/.well-known/databricks-config"
}
{
"method": "DELETE",
"path": "/api/2.0/repos/[NUMID]"
Expand Down
12 changes: 0 additions & 12 deletions acceptance/workspace/repos/delete_by_path/out.requests.txt
Original file line number Diff line number Diff line change
Expand Up @@ -11,10 +11,6 @@
"url": "https://github.com/databricks/databricks-empty-ide-project.git"
}
}
{
"method": "GET",
"path": "/.well-known/databricks-config"
}
{
"method": "GET",
"path": "/api/2.0/workspace/get-status",
Expand All @@ -26,10 +22,6 @@
"method": "GET",
"path": "/api/2.0/repos/[NUMID]"
}
{
"method": "GET",
"path": "/.well-known/databricks-config"
}
{
"method": "GET",
"path": "/api/2.0/workspace/get-status",
Expand All @@ -41,10 +33,6 @@
"method": "DELETE",
"path": "/api/2.0/repos/[NUMID]"
}
{
"method": "GET",
"path": "/.well-known/databricks-config"
}
{
"method": "GET",
"path": "/api/2.0/workspace/get-status",
Expand Down
Loading