-
Notifications
You must be signed in to change notification settings - Fork 161
Cache /.well-known/databricks-config lookups in the CLI #5011
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
simonfaltum
wants to merge
26
commits into
main
Choose a base branch
from
simonfaltum/hostmetadata-cache
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+354
−98
Open
Changes from all commits
Commits
Show all changes
26 commits
Select commit
Hold shift + click to select a range
eb8909f
Add hostmetadata package with caching resolver skeleton
simonfaltum fa9b174
Fix Task 1 spec deviations
simonfaltum 9856973
Isolate hostmetadata test from user cache directory
simonfaltum 2678fe3
Add hostmetadata cache hit/miss tests
simonfaltum b71f298
Add hostmetadata negative-cache and host-isolation tests
simonfaltum a8fc7d1
Attach hostmetadata cache to root workspace/account clients
simonfaltum 6d9981c
Isolate CLI cache dir in test cleanup environment
simonfaltum b23299a
Refactor hostmetadata.Attach to drop ctx requirement
simonfaltum 609f044
Attach hostmetadata cache to bundle workspace config
simonfaltum 62e71f2
Attach hostmetadata cache at remaining config construction sites
simonfaltum 50a27ca
Replace sync.Once with eager cache init in hostmetadata.Attach
simonfaltum 4937738
Attach hostmetadata cache at two missing login construction sites
simonfaltum 2c9b1f3
Add guardrail test for hostmetadata.Attach injection sites
simonfaltum 1a179c5
Add end-to-end acceptance test for host metadata caching
simonfaltum 28127e5
Regenerate acceptance test expectations for hostmetadata caching
simonfaltum 0ce5f13
Expose NewResolver to decouple caching from *config.Config
simonfaltum ec2bd69
Probe positive cache first, skip caching transient errors
simonfaltum 8ce5516
Match sanitized dev version in test replacements
simonfaltum 21fe664
Skip stat-not-found debug log on cache miss
simonfaltum 2ac9b94
Merge branch 'main' into simonfaltum/hostmetadata-cache
simonfaltum fae41c6
Collapse per-site Attach wiring into a single factory registration
simonfaltum 9f32fc8
Drop redundant factory-registration test
simonfaltum 1ec8a58
Address codex review: cleaner negative-cache probe and no raw error t…
simonfaltum 29d7389
Trim comment noise and a tautological test
simonfaltum f1c51d0
Merge branch 'main' into simonfaltum/hostmetadata-cache
simonfaltum 83d7aa7
Merge remote-tracking branch 'origin/main' into simonfaltum/hostmetad…
simonfaltum File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Oops, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,32 @@ | ||
|
|
||
| === First invocation populates the cache | ||
| { | ||
| "profiles": [ | ||
| { | ||
| "name":"cached", | ||
| "host":"[DATABRICKS_URL]", | ||
| "cloud":"aws", | ||
| "auth_type":"", | ||
| "valid":false | ||
| } | ||
| ] | ||
| } | ||
|
|
||
| === Second invocation should read from the cache | ||
| { | ||
| "profiles": [ | ||
| { | ||
| "name":"cached", | ||
| "host":"[DATABRICKS_URL]", | ||
| "cloud":"aws", | ||
| "auth_type":"", | ||
| "valid":false | ||
| } | ||
| ] | ||
| } | ||
|
|
||
| === Only one /.well-known/databricks-config request recorded | ||
| { | ||
| "method": "GET", | ||
| "path": "/.well-known/databricks-config" | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,19 @@ | ||
| sethome "./home" | ||
| export DATABRICKS_CACHE_DIR="$TEST_TMP_DIR/cache" | ||
|
|
||
| # Point a profile at the mock server so auth profiles triggers a host metadata | ||
| # fetch. Without a profile the command does nothing and the cache is never read. | ||
| cat > "./home/.databrickscfg" <<EOF | ||
| [cached] | ||
| host = ${DATABRICKS_HOST} | ||
| token = test-token | ||
| EOF | ||
|
|
||
| title "First invocation populates the cache\n" | ||
| $CLI auth profiles --skip-validate --output json || true | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. what is |
||
|
|
||
| title "Second invocation should read from the cache\n" | ||
| $CLI auth profiles --skip-validate --output json || true | ||
|
|
||
| title "Only one /.well-known/databricks-config request recorded\n" | ||
| print_requests.py //.well-known/databricks-config --get --sort | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| Ignore = [ | ||
| "home", | ||
| "cache", | ||
| ] | ||
| RecordRequests = true |
8 changes: 0 additions & 8 deletions
8
acceptance/bundle/resources/volumes/change-schema-name/out.requests.txt
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,19 +1,25 @@ | ||
|
|
||
| === First call in a session is expected to be a cache miss: | ||
| [DEBUG_TIMESTAMP] Debug: [Local Cache] using cache key: [SHA256_HASH] | ||
| [DEBUG_TIMESTAMP] Debug: [Local Cache] failed to stat cache file: (redacted) | ||
| [DEBUG_TIMESTAMP] Debug: [Local Cache] cache miss, computing | ||
| [DEBUG_TIMESTAMP] Debug: [Local Cache] computed and stored result | ||
| [DEBUG_TIMESTAMP] Debug: [Local Cache] using cache key: [SHA256_HASH] | ||
| [DEBUG_TIMESTAMP] Debug: [Local Cache] cache miss, computing | ||
| [DEBUG_TIMESTAMP] Debug: [Local Cache] computed and stored result | ||
|
|
||
| === Second call in a session is expected to be a cache hit | ||
| [DEBUG_TIMESTAMP] Debug: [Local Cache] using cache key: [SHA256_HASH] | ||
| [DEBUG_TIMESTAMP] Debug: [Local Cache] cache hit | ||
| [DEBUG_TIMESTAMP] Debug: [Local Cache] using cache key: [SHA256_HASH] | ||
| [DEBUG_TIMESTAMP] Debug: [Local Cache] cache hit | ||
|
|
||
| >>> [CLI] cache clear | ||
| Cache cleared successfully from [TEST_TMP_DIR]/.cache | ||
|
|
||
| === First call after a clear is expected to be a cache miss: | ||
| [DEBUG_TIMESTAMP] Debug: [Local Cache] using cache key: [SHA256_HASH] | ||
| [DEBUG_TIMESTAMP] Debug: [Local Cache] failed to stat cache file: (redacted) | ||
| [DEBUG_TIMESTAMP] Debug: [Local Cache] cache miss, computing | ||
| [DEBUG_TIMESTAMP] Debug: [Local Cache] computed and stored result | ||
| [DEBUG_TIMESTAMP] Debug: [Local Cache] using cache key: [SHA256_HASH] | ||
| [DEBUG_TIMESTAMP] Debug: [Local Cache] cache miss, computing | ||
| [DEBUG_TIMESTAMP] Debug: [Local Cache] computed and stored result |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why did we lose the warning?
Is it because this is cached across different test runs?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
to answer my q: I see below that we just started hiding the warnings for this, so it's deliberate decision to always hide it (and won't cause flaky tests).
Still q remains about caching -- is it new cache for each test?