diff --git a/test-design/templates/testcase-template.md b/test-design/templates/testcase-template.md index 6dc0482..68e1249 100644 --- a/test-design/templates/testcase-template.md +++ b/test-design/templates/testcase-template.md @@ -20,10 +20,8 @@ See test-design/README.md for complete standards. | **Field** | **Value** | |-----------|-----------| -| **Title** | [Test Case Title] | -| **Feature** | [Feature Name] | | **Pos/Neg** | [Positive/Negative] | -| **Priority** | [Critical/High/Medium/Low] | +| **Priority** | [Tier0/Tier1/Tier2] | | **Status** | [Draft/Deprecated] | | **Automation** | [Automated/Not Automated/Manual Only] | | **Version** | [Version Number] | diff --git a/test-design/testcases/adapter-framework/adapter-framework.md b/test-design/testcases/adapter-framework/adapter-framework.md new file mode 100644 index 0000000..b823c0c --- /dev/null +++ b/test-design/testcases/adapter-framework/adapter-framework.md @@ -0,0 +1,610 @@ +# Feature: Adapter Framework - Customization + +## Table of Contents + +1. [Adapter Framework can work correctly with Minimal Configuration](#test-title-adapter-framework-can-work-correctly-with-minimal-adapter-configuration) +2. [K8S object can be created correctly on the regional cluster without Maestro enable](#test-title-k8s-object-can-be-created-correctly-on-the-regional-cluster-without-maestro-enable) +3. [Adapter framework can work correctly with adapter configuration dependency relationships](#test-title-adapter-framework-can-work-correctly-with-adapter-configuration-dependency-relationships) +4. [Adapter framework can detect and report invalid adapter configuration failures in deploying progress](#test-title-adapter-framework-can-detect-and-report-invalid-adapter-configuration-failures-in-deploying-progress) +5. [Adapter framework can detect and report failures to cluster API endpoints](#test-title-adapter-framework-can-detect-and-report-failures-to-cluster-api-endpoints) +6. [Adapter framework can detect and handle resource timeouts](#test-title-adapter-framework-can-detect-and-handle-resource-timeouts) +7. [K8s object will be created correctly on targeted cluster with Maestro enabled](#test-title-k8s-object-will-be-created-correctly-on-targeted-cluster-with-maestro-enabled) + +--- + +## Test Title: Adapter framework can work correctly with minimal adapter configuration + +### Description + +This test validates the core functionality of the adapter framework operating with minimal configuration settings (without Maestro enabled). It verifies that the adapter core workflow correctly processes a valid adapter configuration, consumes CloudEvents from the HyperFleet API, executes configured cluster operations, and reports status updates back to the API. The test ensures the complete adapter lifecycle functions end-to-end. + +--- + +| **Field** | **Value** | +|-----------|-----------| +| **Pos/Neg** | Positive | +| **Priority** | Tier0 | +| **Status** | Draft | +| **Automation** | Not Automated | +| **Version** | MVP | +| **Created** | 2026-01-29 | +| **Updated** | 2026-01-29 | + + +--- + +### Preconditions + +1. Environment is prepared using [hyperfleet-infra](https://github.com/openshift-hyperfleet/hyperfleet-infra) with all required platform resources +2. HyperFleet API and HyperFleet Sentinel services are deployed and running successfully + +--- + +### Test Steps + +#### Step 1: Deploy adapter with minimal configuration +**Action:** +- Apply minimal adapter configuration (detailed test data will be provided in test implementation) +- Deploy the adapter to the target environment +- Verify adapter deployment status + +**Expected Result:** +- Adapter pods transition to Running state +- Adapter logs indicate successful initialization without errors +- Adapter can communicate with HyperFleet API successfully + +#### Step 2: Create a new cluster via API +**Action:** +- Submit a POST request to create a new cluster: +```bash +curl -X POST ${API_URL}/api/hyperfleet/v1/clusters \ + -H "Content-Type: application/json" \ + -d +``` + +**Expected Result:** +- Response includes the created cluster ID and initial metadata +- Initial cluster conditions have `status: False` for both condition `{"type": "Ready"}` and `{"type": "Available"}` + +#### Step 3: Verify adapter status + +**Action:** +- Retrieve adapter statuses information: +```bash +curl -X GET ${API_URL}/api/hyperfleet/v1/clusters/{cluster_id}/statuses +``` + +**Expected Result:** +- Response returns HTTP 200 (OK) status code +- Adapter status payload contains the following: + +**Condition Types:** + - All required condition types are present: `Applied`, `Available`, `Health` + - Each condition has `status: "True"` when successful + - `reason`: Human-readable summary of the condition state + - `message`: Detailed human-readable description + - `created_time`: Timestamp when the condition was first created + - `last_transition_time`: Timestamp of the last status change + - `last_updated_time`: Timestamp of the most recent update + - `observed_generation`: Set to `1` for the initial cluster generation + +#### Step 4: Verify cluster final status + +**Action:** +- Retrieve cluster status information: +```bash +curl -X GET ${API_URL}/api/hyperfleet/v1/clusters/{cluster_id} +``` +**Expected Result:** +- Final cluster conditions have `status: True` for both condition `{"type": "Ready"}` and `{"type": "Available"}` + +--- + +## Test Title: K8S object can be created correctly on the regional cluster without Maestro enable + +### Description + +This test validates that the adapter framework can correctly manage Kubernetes resources using templated manifests with Go template syntax. It verifies create operations, resource discovery by labels, and handling multiple resource types in a defined sequence. + +--- + +| **Field** | **Value** | +|-----------|-----------| +| **Pos/Neg** | Positive | +| **Priority** | Tier1 | +| **Status** | Draft | +| **Automation** | Not Automated | +| **Version** | MVP | +| **Created** | 2026-01-29 | +| **Updated** | 2026-01-29 | + + +--- + +### Preconditions + +1. Environment is prepared using [hyperfleet-infra](https://github.com/openshift-hyperfleet/hyperfleet-infra) with all required platform resources +2. HyperFleet API and HyperFleet Sentinel services are deployed and running successfully + +--- + +### Test Steps + +#### Step 1: Prepare and deploy adapter with templated resource manifests +**Action:** +- Configure adapter resources section with templated manifests using Go template syntax: (The detailed test data will be added soon) + - K8s Resource(e.g Namespace) created + - Configure resource labels for discovery +- Deploy the test adapter to the target environment +- Verify the test adapter deployment status + +**Expected Result:** +- Adapter pods transition to Running state +- Adapter logs indicate successful initialization without errors +- Adapter can communicate with HyperFleet API successfully + +#### Step 2: Send POST request to create a new cluster +**Action:** +- Execute cluster creation request: +```bash +curl -X POST ${API_URL}/api/hyperfleet/v1/clusters \ + -H "Content-Type: application/json" \ + -d +``` + +**Expected Result:** +- API returns successful response + +#### Step 3: Verify cluster status and adapter execution +**Action:** +- Send GET request to retrieve the specific cluster: + +```bash +curl -X GET ${API_URL}/api/hyperfleet/v1/clusters/{cluster_id}/statuses +``` + +**Expected Result:** +- Response status code is 200 (OK) +- Each condition has `status: "True"` when successful + - `Applied` contains the 'Job resource created' related message + +#### Step 4: Verify Kubernetes Resources Management +**Action:** +- Verify resource created + +**Expected Result:** +- Resource is created successfully with templated values rendered + +#### Step 5: Verify cluster final status + +**Action:** +- Retrieve cluster status information: +```bash +curl -X GET ${API_URL}/api/hyperfleet/v1/clusters/{cluster_id} +``` +**Expected Result:** +- Final cluster conditions have `status: True` for both condition `{"type": "Ready"}` and `{"type": "Available"}` + +## Test Title: Adapter framework can work correctly with adapter configuration dependency relationships + +### Description + +This test validates that the adapter framework correctly handles dependency relationships between multiple adapter configurations. It verifies that adapters execute in the correct order based on preconditions, with dependent adapters waiting for prerequisite adapters to complete successfully before proceeding. + +--- + +| **Field** | **Value** | +|-----------|-----------| +| **Pos/Neg** | Positive | +| **Priority** | Tier1 | +| **Status** | Draft | +| **Automation** | Not Automated | +| **Version** | MVP | +| **Created** | 2026-01-29 | +| **Updated** | 2026-01-29 | + + +--- + +### Preconditions + +1. Environment is prepared using [hyperfleet-infra](https://github.com/openshift-hyperfleet/hyperfleet-infra) with all required platform resources +2. HyperFleet API and HyperFleet Sentinel services are deployed and running successfully + +--- + +### Test Steps + +#### Step 1: Prepare and deploy adapters with dependency chain +**Action:** +- Prepare three AdapterConfigs representing dependency chain: (The detailed test data will be added soon) + - `adapter-1-namespace-creator.yaml` - Creates namespace + - `adapter-2-workload.yaml` - Deploys workload in namespace via setting preconditions + - `adapter-3-workload.yaml` - Deploys another workload in namespace via setting preconditions +- Deploy the test adapters and other components using hyperfleet-chart + +**Expected Result:** +- All adapters load configuration successfully +- Adapter pods are running successfully +- Adapter logs show successful initialization +- Dependency relationships are configured correctly + +#### Step 2: Send POST request to create a new cluster +**Action:** +- Execute cluster creation request: +```bash +curl -X POST ${API_URL}/api/hyperfleet/v1/clusters \ + -H "Content-Type: application/json" \ + -d +``` + +**Expected Result:** +- API returns successful response + +#### Step 3: Verify execution order and state +**Action:** +- Send GET request to retrieve the specific cluster: +```bash +curl -X GET ${API_URL}/api/hyperfleet/v1/clusters/{cluster_id}/statuses +``` + +**Expected Result:** +- The first adapter (namespace-creator) works in progress +- The second and third adapters wait for preconditions to be met +- Adapter information shows: + - Adapter 1 "message": "namespace is not active and ready" and 'status' is 'false' + - Adapter 2 "message": "Validation is pending" and 'status' is 'false' + - Adapter 3 "message": "Validation is pending" and 'status' is 'false' + +#### Step 4: Verify sequential execution and resource creation +**Action:** +- Verify Kubernetes resources: namespace is created +- Send GET request to retrieve the specific cluster: +```bash +curl -X GET ${API_URL}/api/hyperfleet/v1/clusters/{cluster_id}/statuses +``` +- Monitor adapter execution progress + +**Expected Result:** +- Namespace is created by adapter 1 +- After namespace creation, adapter 2 and adapter 3 execute +- Adapter 2 and adapter 3's related resources are created in the namespace +- Adapters execute in correct order based on preconditions: + 1. Adapter 1 (namespace-creator) completes first + 2. Adapter 2 (workload) executes after adapter 1 completes + 3. Adapter 3 (another workload) executes after adapter 1 completes +- First adapter reports NamespaceReady status to API + + +#### Step 5: Verify final cluster state +**Action:** +- Send GET request to retrieve cluster statuses: +```bash +curl -X GET ${API_URL}/api/hyperfleet/v1/clusters/{cluster_id} +``` + +**Expected Result:** +- Final cluster conditions have `status: True` for both condition `{"type": "Ready"}` and `{"type": "Available"}` + + +## Test Title: Adapter framework can detect and report invalid adapter configuration failures in deploying progress + +### Description + +This test validates that the adapter framework correctly detects and reports failures caused by invalid adapter configuration, including template rendering errors, malformed CloudEvents, invalid AdapterConfig structure, and CEL evaluation errors. It ensures proper error handling and status reporting for configuration-related issues. + +--- + +| **Field** | **Value** | +|-----------|-----------| +| **Pos/Neg** | Negative | +| **Priority** | Tier2 | +| **Status** | Draft | +| **Automation** | Not Automated | +| **Version** | MVP | +| **Created** | 2026-01-30 | +| **Updated** | 2026-01-30 | + + +--- + +### Preconditions +1. Environment is prepared using [hyperfleet-infra](https://github.com/openshift-hyperfleet/hyperfleet-infra) with all required platform resources +2. HyperFleet API and HyperFleet Sentinel services are deployed and running successfully + +--- + +### Test Steps + +#### Step 1: Test template rendering errors +**Action:** +- Configure AdapterConfig with invalid AdapterConfig (missing required fields, invalid field types) +- Deploy the test adapter + +**Expected Result:** +- Adapter detects template rendering error +- Log reports failure with clear error message + +#### Step 2: Test malformed CloudEvents +**Action:** +- Configure AdapterConfig with malformed CloudEvent (missing required fields, invalid JSON structure) +- Deploy the test adapter + +**Expected Result:** +- Adapter rejects malformed CloudEvent +- Log reports CloudEvent validation error + + +#### Step 3: Test CEL evaluation errors +**Action:** +- Configure AdapterConfig with invalid CEL expressions in payload construction +- Create cluster to trigger CEL evaluation + +**Expected Result:** +- Adapter detects CEL evaluation error +- Error message includes the failing expression + +--- + +## Test Title: Adapter framework can detect and report failures to cluster API endpoints + +### Description + +This test validates that the adapter framework correctly detects and reports failures when attempting to create invalid Kubernetes resources on the target cluster. It ensures that when an adapter's configuration contains invalid K8s resource objects, the framework properly handles the API server rejection, logs meaningful error messages, and reports the failure status back to the HyperFleet API with appropriate condition states and error details. + + +--- + +| **Field** | **Value** | +|-----------|-----------| +| **Pos/Neg** | Negative | +| **Priority** | Tier2 | +| **Status** | Draft | +| **Automation** | Not Automated | +| **Version** | MVP | +| **Created** | 2026-01-30 | +| **Updated** | 2026-01-30 | + + +--- + +### Preconditions +1. Environment is prepared using [hyperfleet-infra](https://github.com/openshift-hyperfleet/hyperfleet-infra) with all required platform resources +2. HyperFleet API and HyperFleet Sentinel services are deployed and running successfully + +--- + +### Test Steps + +#### Step 1: Test template rendering errors +**Action:** +- Configure AdapterConfig with invalid AdapterConfig (invalid K8s resource object) +- Deploy the test adapter + +**Expected Result:** +- Adapter detects template rendering error +- Log reports failure with clear error message + +#### Step 2: Send POST request to create a new cluster +**Action:** +- Execute cluster creation request: +```bash +curl -X POST ${API_URL}/api/hyperfleet/v1/clusters \ + -H "Content-Type: application/json" \ + -d +``` + +**Expected Result:** +- API returns successful response + +#### Step 3: Wait for timeout and Verify Timeout Handling +**Action:** +- Wait for some minutes +- Verify adapter status + +**Expected Result:** +```bash + curl -X GET ${API_URL}/api/hyperfleet/v1/clusters//statuses \ + | jq -r '.items[] | select(.adapter=="") | .conditions[] | select(.type=="Applied")' + + # Example: + # { + # "type": "Available", + # "status": "False", + # "reason": "`invalid k8s object` resource is invalid", + # "message": "Invalid Kubernetes object" + # } +``` + +--- + + +## Test Title: Adapter framework can detect and handle resource timeouts + +### Description + +This test validates that the adapter framework correctly detects and handles resource timeouts when adapter Jobs exceed configured timeout limits. + +--- + +| **Field** | **Value** | +|-----------|-----------| +| **Pos/Neg** | Negative | +| **Priority** | Tier2 | +| **Status** | Draft | +| **Automation** | Not Automated | +| **Version** | MVP | +| **Created** | 2026-01-30 | +| **Updated** | 2026-01-30 | + + +--- + +### Preconditions +1. Environment is prepared using [hyperfleet-infra](https://github.com/openshift-hyperfleet/hyperfleet-infra) with all required platform resources +2. HyperFleet API and HyperFleet Sentinel services are deployed and running successfully + +--- + +### Test Steps + +#### Step 1: Configure adapter with timeout setting +**Action:** +- Configure AdapterConfig with non-existed conditions that can't meet the precondition +```yaml + preconditions: + - name: "clusterStatus" + apiCall: + method: "GET" + url: "{{ .hyperfleetApiBaseUrl }}/api/hyperfleet/{{ .hyperfleetApiVersion }}/clusters/{{ .clusterId }}" + timeout: 10s + retryAttempts: 3 + retryBackoff: "exponential" + capture: + - name: "clusterName" + field: "name" + - name: "clusterPhase" + field: "status.phase" + - name: "generationId" + field: "generation" + conditions: + - field: "clusterPhase" + operator: "in" + values: ["NotReady", "Ready"] +``` +- Deploy the test adapter + +**Expected Result:** +- Adapter loads configuration successfully +- Adapter pods are running successfully +- Adapter logs show successful initialization + +#### Step 2: Send POST request to create a new cluster +**Action:** +- Execute cluster creation request: +```bash +curl -X POST ${API_URL}/api/hyperfleet/v1/clusters \ + -H "Content-Type: application/json" \ + -d +``` + +**Expected Result:** +- API returns successful response + +#### Step 3: Wait for timeout and Verify Timeout Handling +**Action:** +- Wait for some minutes +- Verify adapter status + +**Expected Result:** +```bash + curl -X GET ${API_URL}/api/hyperfleet/v1/clusters//statuses \ + | jq -r '.items[] | select(.adapter=="") | .conditions[] | select(.type=="Applied")' + + # Example: + # { + # "type": "Applied", + # "status": "False", + # "reason": "JobTimeout", + # "message": "Validation job did not complete within 30 seconds" + # } +``` +----- + +## Test Title: K8s object will be created correctly on targeted cluster with Maestro enabled + +### Description + +This test validates that the adapter framework creates ManifestWork resources correctly when Maestro is enabled, allowing resources to be propagated to targeted managed clusters. It verifies ManifestWork generation, structure, and successful delivery to target clusters. + +--- + +| **Field** | **Value** | +|-----------|-----------| +| **Pos/Neg** | Positive | +| **Priority** | Tier0 | +| **Status** | Draft | +| **Automation** | Not Automated | +| **Version** | MVP | +| **Created** | 2026-01-30 | +| **Updated** | 2026-01-30 | + + +--- + +### Preconditions + +1. Environment is prepared using [hyperfleet-infra](https://github.com/openshift-hyperfleet/hyperfleet-infra) with all required platform resources +2. HyperFleet API and HyperFleet Sentinel services are deployed and running successfully + +--- + +### Test Steps + +#### Step 1: Configure adapter with Maestro enabled +**Action:** +- Configure AdapterConfig with Maestro enable. See full example: [adapter-business-logic-template-MVP.yaml](https://github.com/openshift-hyperfleet/architecture/blob/main/hyperfleet/components/adapter/framework/configs/adapter-business-logic-template-MVP.yaml) +- Deploy the test adapter + +**Expected Result:** +- Adapter loads configuration successfully +- Adapter pods are running successfully +- Adapter logs show successful initialization + +#### Step 2: Send POST request to create a new cluster +**Action:** +- Execute cluster creation request: +```bash +curl -X POST ${API_URL}/api/hyperfleet/v1/clusters \ + -H "Content-Type: application/json" \ + -d +``` + +**Expected Result:** +- API returns successful response + +#### Step 3: Verify ManifestWork +**Action:** +- Verify ManifestWork + +**Expected Result:** +- ManifestWork is created successfully with templated values rendered + +#### Step 4: Verify Kubernetes Resources Management +**Action:** +- Verify configured resources created + +**Expected Result:** +- Resource is created successfully with templated values rendered + +#### Step 5: Verify adapter status + +**Action:** +- Retrieve adapter statuses information: +```bash +curl -X GET ${API_URL}/api/hyperfleet/v1/clusters/{cluster_id}/statuses +``` + +**Expected Result:** +- Adapter status payload contains the following: + +**Condition Types:** + - All required condition types are present: `Applied`, `Available`, `Health` + - Each condition has `status: "True"` when successful + - `reason`: Human-readable summary of the condition state + - `message`: Detailed human-readable description + - `created_time`: Timestamp when the condition was first created + - `last_transition_time`: Timestamp of the last status change + - `last_updated_time`: Timestamp of the most recent update + - `observed_generation`: Set to `1` for the initial cluster generation + +#### Step 6: Verify cluster final status + +**Action:** +- Retrieve cluster status information: +```bash +curl -X GET ${API_URL}/api/hyperfleet/v1/clusters/{cluster_id} +``` +**Expected Result:** +- Final cluster conditions have `status: True` for both condition `{"type": "Ready"}` and `{"type": "Available"}` +--- \ No newline at end of file