Fix vector search index test failures by janniklasrose · Pull Request #5454 · databricks/cli

janniklasrose · 2026-06-05T13:53:09Z

Changes

Add index_subtype as backend_default for vector search indexes and teach testserver to return a default value
Add --unique to print_requests.py so tests can choose to collapse subsequent GET requests (e.g. encountered during polling)

Why

Grants test was failing with recreate

acceptance_test.go:939: Diff:
          --- bundle/resources/vector_search_indexes/grants/select/output.txt
          +++ /tmp/TestAcceptbundleresourcesvector_search_indexesgrantsselect1239696916/001/output.txt
          @@ -13,8 +13,11 @@
           Deployment complete!
           
           >>> [CLI] bundle plan
          -Plan: 0 to add, 0 to change, 0 to delete, 3 unchanged
          +recreate vector_search_indexes.my_index
          +update vector_search_indexes.my_index.grants
           
          +Plan: 1 to add, 1 to change, 1 to delete, 1 unchanged
          +
           >>> [CLI] grants get table main.default.vs_index_[UNIQUE_NAME]
           {
             "privilege_assignments": [

bundle/resources/vector_search_indexes/recreate/embedding_dimension/out.requests.create.direct.json had an extra GET

--- bundle/resources/vector_search_indexes/recreate/embedding_dimension/out.requests.create.direct.json
          +++ /tmp/TestAcceptbundleresourcesvector_search_indexesrecreateembed3063347120/001/out.requests.create.direct.json
          @@ -21,3 +21,555 @@
             "method": "GET",
             "path": "/api/2.0/vector-search/indexes/main.default.vs_index_[UNIQUE_NAME]"
           }
          +{
          +  "method": "GET",
          +  "path": "/api/2.0/vector-search/indexes/main.default.vs_index_[UNIQUE_NAME]"
          +}

Tests

Reverted the resources.yml change and reran grants/select:

>>> [CLI] bundle plan
-Plan: 0 to add, 0 to change, 0 to delete, 3 unchanged
+recreate vector_search_indexes.my_index
+update vector_search_indexes.my_index.grants
+
+Plan: 1 to add, 1 to change, 1 to delete, 1 unchanged

github-actions · 2026-06-05T13:53:39Z

Approval status: pending

`/acceptance/bundle/` - needs approval

Files: acceptance/bundle/resources/vector_search_indexes/recreate/embedding_dimension/output.txt, acceptance/bundle/resources/vector_search_indexes/recreate/embedding_dimension/script, acceptance/bundle/resources/vector_search_indexes/recreate/with_endpoint/output.txt
Suggested: @denik
Also eligible: @pietern, @andrewnester, @anton-107, @shreyas-goenka, @lennartkats-db

`/bundle/` - needs approval

Files: bundle/direct/dresources/resources.yml
Suggested: @denik
Also eligible: @pietern, @andrewnester, @anton-107, @shreyas-goenka, @lennartkats-db

General files (require maintainer)

Files: acceptance/bin/print_requests.py, libs/testserver/vector_search_indexes.go
Based on git history:

@denik -- recent work in bundle/direct/dresources/, acceptance/bin/

_{Any maintainer (@andrewnester, @anton-107, @denik, @pietern, @shreyas-goenka, @simonfaltum, @renaudhartert-db) can approve all areas.

See OWNERS for ownership rules.}

eng-dev-ecosystem-bot · 2026-06-05T14:35:27Z

Commit: 3b9afac

Run: 27019007846

Ignoring .gh-logs/27019007846/update-check-action/action.yml: cannot extract env
Ignoring .gh-logs/27019007846/gh-report-action/gh_pr_comment.py: cannot extract env
Ignoring .gh-logs/27019007846/gh-report-action/action.yml: cannot extract env

	Env	🟨KNOWN	🔄flaky	💚RECOVERED	🙈SKIP	✅pass	🙈skip	Time
🔄	aws linux		4	7	15	257	923	6:23
🟨	aws windows	7			15	263	921	12:16
💚	aws-ucws linux			7	15	357	837	6:11
💚	aws-ucws windows			7	15	359	835	9:12
💚	azure linux			1	17	264	921	6:08
💚	azure windows			1	17	266	919	9:06
💚	azure-ucws linux			1	17	362	833	7:41
💚	azure-ucws windows			1	17	364	831	10:23
💚	gcp linux			1	17	260	924	6:40
💚	gcp windows			1	17	262	922	11:20

26 interesting tests: 15 SKIP, 7 KNOWN, 4 flaky

	Test Name	aws linux	aws windows	aws-ucws linux	aws-ucws windows	azure linux	azure windows	azure-ucws linux	azure-ucws windows	gcp linux	gcp windows
🟨	TestAccept	💚R	🟨K	💚R	💚R	💚R	💚R	💚R	💚R	💚R	💚R
🔄	TestAccept/bundle/destroy/jobs-and-pipeline	🔄f	✅p	✅p	✅p	✅p	✅p	✅p	✅p	✅p	✅p
🔄	TestAccept/bundle/destroy/jobs-and-pipeline/DATABRICKS_BUNDLE_ENGINE=terraform	🔄f	✅p	✅p	✅p	✅p	✅p	✅p	✅p	✅p	✅p
🙈	TestAccept/bundle/invariant/no_drift	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
🙈	TestAccept/bundle/resources/permissions	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
🟨	TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions	💚R	🟨K	💚R	💚R	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
🟨	TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions/DATABRICKS_BUNDLE_ENGINE=direct	💚R	🟨K	💚R	💚R
🟨	TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions/DATABRICKS_BUNDLE_ENGINE=terraform	💚R	🟨K	💚R	💚R
🟨	TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions	💚R	🟨K	💚R	💚R	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
🟨	TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions/DATABRICKS_BUNDLE_ENGINE=direct	💚R	🟨K	💚R	💚R
🟨	TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions/DATABRICKS_BUNDLE_ENGINE=terraform	💚R	🟨K	💚R	💚R
🙈	TestAccept/bundle/resources/postgres_branches/basic	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
🙈	TestAccept/bundle/resources/postgres_branches/recreate	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
🙈	TestAccept/bundle/resources/postgres_branches/replace_existing	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
🙈	TestAccept/bundle/resources/postgres_branches/update_protected	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
🙈	TestAccept/bundle/resources/postgres_branches/without_branch_id	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
🙈	TestAccept/bundle/resources/postgres_endpoints/basic	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
🙈	TestAccept/bundle/resources/postgres_endpoints/recreate	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
🙈	TestAccept/bundle/resources/postgres_projects/update_display_name	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
🙈	TestAccept/bundle/resources/synced_database_tables/basic	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
🙈	TestAccept/bundle/resources/vector_search_endpoints/drift/recreated_same_name	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
🙈	TestAccept/bundle/resources/vector_search_indexes/basic	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
🙈	TestAccept/bundle/resources/vector_search_indexes/grants/select	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
🔄	TestAccept/selftest/record_cloud/pipeline-crud	🔄f	✅p	✅p	✅p	✅p	✅p	✅p	✅p	✅p	✅p
🔄	TestAccept/selftest/record_cloud/pipeline-crud/DATABRICKS_BUNDLE_ENGINE=terraform	🔄f	✅p	✅p	✅p	✅p	✅p	✅p	✅p	✅p	✅p
🙈	TestAccept/ssh/connection	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S

Top 27 slowest tests (at least 2 minutes):

duration	env	testname
5:08	gcp linux	TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
5:02	gcp linux	TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
4:46	gcp windows	TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
4:33	gcp windows	TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
4:12	azure-ucws linux	TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
4:03	azure-ucws windows	TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
4:02	azure windows	TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
3:34	azure linux	TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
3:34	azure windows	TestAccept
3:33	aws-ucws windows	TestAccept
3:29	azure-ucws windows	TestAccept
3:25	aws windows	TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
3:24	gcp windows	TestAccept
3:12	aws-ucws windows	TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
3:03	aws windows	TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
3:03	azure windows	TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
3:02	azure-ucws windows	TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:52	aws-ucws windows	TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:50	aws-ucws linux	TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:44	azure-ucws linux	TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:38	aws-ucws linux	TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:38	aws linux	TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:36	azure linux	TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:32	azure-ucws windows	TestAccept/bundle/resources/volumes/recreate/DATABRICKS_BUNDLE_ENGINE=terraform
2:29	aws linux	TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:12	azure windows	TestAccept/bundle/deploy/files/no-snapshot-sync/DATABRICKS_BUNDLE_ENGINE=terraform
2:08	azure-ucws windows	TestAccept/bundle/resources/jobs/check-metadata/DATABRICKS_BUNDLE_ENGINE=terraform

denik · 2026-06-05T14:45:00Z

Fix vector search index test failures

It's not just fixing test failures, right? It fixes real bug where we got undesirable recreate? So we need the title to reflect that + NEXT_CHANGELOG

janniklasrose added 2 commits June 5, 2026 15:49

Add index_subtype backend_default for vector search indexes

0e40f2d

Add --unique to print_requests

3b9afac

janniklasrose requested a review from denik June 5, 2026 13:53

janniklasrose temporarily deployed to test-trigger-is June 5, 2026 13:53 — with GitHub Actions Inactive

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix vector search index test failures#5454

Fix vector search index test failures#5454
janniklasrose wants to merge 2 commits into
mainfrom
janniklasrose/vector-search-index-grants-acceptance

janniklasrose commented Jun 5, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Jun 5, 2026

Uh oh!

eng-dev-ecosystem-bot commented Jun 5, 2026

Uh oh!

denik commented Jun 5, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

janniklasrose commented Jun 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

Why

Tests

Uh oh!

github-actions Bot commented Jun 5, 2026

Approval status: pending

/acceptance/bundle/ - needs approval

/bundle/ - needs approval

General files (require maintainer)

Uh oh!

eng-dev-ecosystem-bot commented Jun 5, 2026

Uh oh!

denik commented Jun 5, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

janniklasrose commented Jun 5, 2026 •

edited

Loading

`/acceptance/bundle/` - needs approval

`/bundle/` - needs approval