Skip to content

discoveryengine: fix TestAccDiscoveryEngineDataStore_discoveryengineDatastoreKmsKeyNameExample - #18829

Draft
SirGitsalot wants to merge 1 commit into
GoogleCloudPlatform:mainfrom
SirGitsalot:fix-discoveryengine-datastore-kms-key-name
Draft

discoveryengine: fix TestAccDiscoveryEngineDataStore_discoveryengineDatastoreKmsKeyNameExample#18829
SirGitsalot wants to merge 1 commit into
GoogleCloudPlatform:mainfrom
SirGitsalot:fix-discoveryengine-datastore-kms-key-name

Conversation

@SirGitsalot

@SirGitsalot SirGitsalot commented Aug 28, 2026

Copy link
Copy Markdown
Member

Fixes nightly acceptance test failure for TestAccDiscoveryEngineDataStore_discoveryengineDatastoreKmsKeyNameExample.

Root Cause & Fix

  1. In Discovery Engine (Vertex AI Search), creating a google_discovery_engine_data_store with kms_key_name requires an associated CmekConfig in the ACTIVE state for that KMS key. Without an active CmekConfig, the backend fails CreateDataStore with FAILED_PRECONDITION or 404 KMS key not found.
  2. The default bootstrap key tftest-shared-key-1 was previously mapped in the shared test project to a deleted CmekConfig, causing tests using that key to fail with:
    CmekConfig projects/.../locations/us/cmekConfigs/tftest-shared-cmekconfig-1 is in the DELETED state but should be in the ACTIVE state to create DataStores.
  3. When destroying resources, asynchronous/eventually-consistent DataStore deletion can momentarily delay CmekConfig deletion, returning:
    There are currently 1 datastore(s) connected to this CmekConfig. Please delete these datastores before trying to delete the CmekConfig.
    Adding a retry predicate allows Terraform to retry the deletion until the DataStore detachment propagates.

This PR:

  1. Updates mmv1/templates/terraform/samples/services/discoveryengine/discoveryengine_datastore_kms_key_name.tf.tmpl to:
    • Reference the project number via data "google_project" "project" {}.
    • Grant roles/cloudkms.cryptoKeyEncrypterDecrypter on the KMS key to the Discovery Engine service account (service-${data.google_project.project.number}@gcp-sa-discoveryengine.iam.gserviceaccount.com).
    • Provision google_discovery_engine_cmek_config for the KMS key before creating the DataStore (depends_on = [google_discovery_engine_cmek_config.default]).
  2. Updates mmv1/products/discoveryengine/DataStore.yaml to:
    • Add cmek_config_id: 'cmek-config-id' to resource_id_vars.
    • Override kms_key_name to use kms.BootstrapKMSKeyWithPurposeInLocationAndName(t, "ENCRYPT_DECRYPT", "us", "tftest-shared-key-5").CryptoKey.Name (matching the bootstrap key used by the passing CMEK config tests).
  3. Adds IsDiscoveryEngineCmekConfigInUseError retry predicate to mmv1/third_party/terraform/transport/error_retry_predicates.go and registers it in mmv1/products/discoveryengine/CmekConfig.yaml to handle eventual consistency on deletion.

@SirGitsalot

Copy link
Copy Markdown
Member Author

/gcbrun

@modular-magician

modular-magician commented Aug 28, 2026

Copy link
Copy Markdown
Collaborator

Hi there, I'm the Modular magician. I've detected the following information about your changes for commit a4ee698:

Diff report

Your PR generated the following diffs in downstream repositories:

Repository Diff Link Changes
google provider View Diff 2 files changed, 21 insertions(+)
google-beta provider View Diff 2 files changed, 21 insertions(+)
Open in Cloud Shell View Diff 1 file changed, 10 insertions(+)

Test report

Analytics

Total Tests Passed Skipped Affected
53 48 4 1
Affected Service Packages
  • discoveryengine

Learn how VCR tests work


Step 1: Replaying Mode

Action taken

Found 1 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit.

Click here to see the affected tests
  • TestAccDiscoveryEngineDataStore_discoveryengineDatastoreKmsKeyNameExample

View the replaying VCR build log


Step 2: Recording Mode

Recording Mode Replaying Rerun Test Name
❌ Error · Log - TestAccDiscoveryEngineDataStore_discoveryengineDatastoreKmsKeyNameExample

Caution

Issues requiring attention before PR completion

🔴 Initial Recording Failed: Some tests failed during the recording step. See the table above for details.

Please address these issues to complete your PR. If you believe these detections are incorrect or unrelated to your change, please raise the concern with your reviewer.

View the recording VCR build log or the debug logs folder for detailed results.

@SirGitsalot VCR tests complete for a4ee698!

@SirGitsalot
SirGitsalot force-pushed the fix-discoveryengine-datastore-kms-key-name branch from a4ee698 to 997c897 Compare August 28, 2026 23:25
@modular-magician

modular-magician commented Aug 28, 2026

Copy link
Copy Markdown
Collaborator

Hi there, I'm the Modular magician. I've detected the following information about your changes for commit 997c897:

Diff report

Your PR generated the following diffs in downstream repositories:

Repository Diff Link Changes
google provider View Diff 2 files changed, 43 insertions(+), 3 deletions(-)
google-beta provider View Diff 2 files changed, 43 insertions(+), 3 deletions(-)
Open in Cloud Shell View Diff 1 file changed, 19 insertions(+)

Test report

Analytics

Total Tests Passed Skipped Affected
53 48 4 1
Affected Service Packages
  • discoveryengine

Learn how VCR tests work


Step 1: Replaying Mode

Action taken

Found 1 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit.

Click here to see the affected tests
  • TestAccDiscoveryEngineDataStore_discoveryengineDatastoreKmsKeyNameExample

View the replaying VCR build log


Step 2: Recording Mode

Recording Mode Replaying Rerun Test Name
❌ Error · Log - TestAccDiscoveryEngineDataStore_discoveryengineDatastoreKmsKeyNameExample

Caution

Issues requiring attention before PR completion

🔴 Initial Recording Failed: Some tests failed during the recording step. See the table above for details.

Please address these issues to complete your PR. If you believe these detections are incorrect or unrelated to your change, please raise the concern with your reviewer.

View the recording VCR build log or the debug logs folder for detailed results.

@SirGitsalot VCR tests complete for 997c897!

@SirGitsalot
SirGitsalot force-pushed the fix-discoveryengine-datastore-kms-key-name branch from 997c897 to a6d6697 Compare August 29, 2026 00:02
@modular-magician

modular-magician commented Aug 29, 2026

Copy link
Copy Markdown
Collaborator

Hi there, I'm the Modular magician. I've detected the following information about your changes for commit a6d6697:

Diff report

Your PR generated the following diffs in downstream repositories:

Repository Diff Link Changes
google provider View Diff 4 files changed, 90 insertions(+), 33 deletions(-)
google-beta provider View Diff 4 files changed, 90 insertions(+), 33 deletions(-)
terraform-google-conversion View Diff 1 file changed, 14 insertions(+)
Open in Cloud Shell View Diff 1 file changed, 19 insertions(+)

Test report

Analytics

Total Tests Passed Skipped Affected
6933 6219 698 16
Affected Service Packages
  • All service packages are affected

Learn how VCR tests work


Step 1: Replaying Mode

Action taken

Found 16 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit.

Click here to see the affected tests
  • TestAccComputeInstanceFromMachineImage_diffProject
  • TestAccComputeSecurityPolicy_withPreconfiguredWafConfig
  • TestAccContainerCluster_autopilot_privileged_admission
  • TestAccContainerCluster_dataplaneOptimizationMode
  • TestAccContainerCluster_withNodeConfigKubeletConfigSettingsInNodePool
  • TestAccContainerCluster_withSoleTenantGroup
  • TestAccContainerNodePool_withHugepageConfig
  • TestAccContainerNodePool_withSoleTenantConfig
  • TestAccDataformConfig_update
  • TestAccDataprocMetastoreService_dataprocMetastoreServicePrivateServiceConnectExample
  • TestAccDiscoveryEngineDataStore_discoveryengineDatastoreKmsKeyNameExample
  • TestAccManagedKafkaConnector_managedkafkaConnectorBasicExample
  • TestAccProjectIamMemberRemove_memberInMultipleBindings
  • TestAccPubsubSubscription_pubsubSubscriptionTagsExample
  • TestAccVertexAIFeatureOnlineStoreFeatureview_vertexAiFeatureonlinestoreFeatureview_featureRegistry_updated
  • TestAccVertexAISemanticGovernancePolicyEngine_basic

View the replaying VCR build log


Step 2: Recording Mode

Recording Mode Replaying Rerun Test Name
✅ Log TestAccDataformConfig_update
✅ Log ❌ Error · Log TestAccProjectIamMemberRemove_memberInMultipleBindings
❌ Error · Log - TestAccComputeInstanceFromMachineImage_diffProject
❌ Error · Log - TestAccComputeSecurityPolicy_withPreconfiguredWafConfig
❌ Error · Log - TestAccContainerCluster_autopilot_privileged_admission
❌ Error · Log - TestAccContainerCluster_dataplaneOptimizationMode
❌ Error · Log - TestAccContainerCluster_withNodeConfigKubeletConfigSettingsInNodePool
❌ Error · Log - TestAccContainerCluster_withSoleTenantGroup
❌ Error · Log - TestAccContainerNodePool_withHugepageConfig
❌ Error · Log - TestAccContainerNodePool_withSoleTenantConfig
❌ Error · Log - TestAccDataprocMetastoreService_dataprocMetastoreServicePrivateServiceConnectExample
❌ Error · Log - TestAccDiscoveryEngineDataStore_discoveryengineDatastoreKmsKeyNameExample
❌ Error · Log - TestAccManagedKafkaConnector_managedkafkaConnectorBasicExample
❌ Error · Log - TestAccPubsubSubscription_pubsubSubscriptionTagsExample
❌ Error · Log - TestAccVertexAIFeatureOnlineStoreFeatureview_vertexAiFeatureonlinestoreFeatureview_featureRegistry_updated
❌ Error · Log - TestAccVertexAISemanticGovernancePolicyEngine_basic

Caution

Issues requiring attention before PR completion

🔴 Initial Recording Failed: Some tests failed during the recording step. See the table above for details.

🔴 Replaying Rerun Failed: Some tests failed due to non-determinism when VCR replayed the response. See the table above for details.

Please address these issues to complete your PR. If you believe these detections are incorrect or unrelated to your change, please raise the concern with your reviewer.

View the recording VCR build log or the debug logs folder for detailed results.

@SirGitsalot VCR tests complete for a6d6697!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants