fix(ci): remove broken integ teardown safety net + map stack name to commit sha#567
Merged
scottschreckengaust merged 4 commits intoJul 9, 2026
Conversation
… workflow (aws-samples#566) The if: always() backstop ran raw aws cloudformation delete-stack / wait stack-delete-complete as the GitHub OIDC role, which has no cloudformation:DeleteStack or DescribeStacks permission. Both calls AccessDenied on every run; the un-guarded wait exits 255 and fails the job even when the integ test passed and integ-runner already destroyed the stack (e.g. run 28892156232: SUCCESS in 291s, then teardown-step failure). integ-runner remains the teardown path (cdk destroy on success and failure via the CDK bootstrap roles). Stranded-stack reclaim for cancelled/crashed runs is owned by aws-samples#400 (non-blocking cleanup) and aws-samples#72 (scheduled sweep).
Replace the single hardcoded backgroundagent-integ stack name with a run-unique int-<short-sha> derived from the resolved head SHA, threaded into integ-runner via INTEG_STACK_NAME. The test falls back to a stable backgroundagent-integ-local name so the local mise //cdk:integ path is unchanged. A stranded stack from a cancelled/crashed run no longer collides with or head-of-line blocks a later run under one fixed name; the int- prefix marks it ephemeral for the scheduled sweep (aws-samples#72). Concurrency-comment updated: run-unique names make single-concurrency a cost bound, not a correctness requirement. Partial pull-forward of aws-samples#400's ephemeral-name ask, scoped to naming.
Same-commit re-runs (re-run, or dispatch + workflow_run on one SHA) now get distinct stack names, matching aws-samples#400's proposed int-{short-sha}-{run} scheme.
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #567 +/- ##
=======================================
Coverage ? 88.99%
=======================================
Files ? 222
Lines ? 53019
Branches ? 5531
=======================================
Hits ? 47182
Misses ? 5837
Partials ? 0 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
scottschreckengaust
requested changes
Jul 9, 2026
scottschreckengaust
left a comment
Contributor
There was a problem hiding this comment.
Add a unique stackName
Co-authored-by: Scott Schreckengaust <scottschreckengaust@users.noreply.github.com>
ayushtr-aws
added a commit
to ayushtr-aws/sample-autonomous-cloud-coding-agents
that referenced
this pull request
Jul 9, 2026
Fold in upstream aws-samples#567 (integ.yml overhaul) which conflicted with the Phase-1 changes in the same job: - Drop the raw-CLI "Ensure stacks torn down" safety net: the GitHub OIDC role has no cloudformation:DeleteStack/DescribeStacks, so it only ever AccessDenied and failed otherwise-passing runs (aws-samples#566/aws-samples#567). integ-runner still tears down via the CDK bootstrap roles; stranded-stack reclaim is owned by the sweeper (aws-samples#72) and non-blocking cleanup (aws-samples#400). - Set both stack-name env vars on the run step so the two integ tests get distinct CloudFormation names: INTEG_STACK_NAME=int-<sha>-<run> for the Phase-0 smoke test (aws-samples#567's scheme) and COMMIT_HASH for the Phase-1 lifecycle test (int-<hash>). Keep the plain `mise //cdk:integ` invocation (our mise.toml runs integ-runner --verbose and takes config via env, not the -c github:* CDK context, which no integ test source reads). - Keep the Phase-1 gate env (INTEG_SANDBOX_REPO/INTEG_PAT_SECRET_ID) and the bgagent/* sandbox branch/PR cleanup step.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two coupled changes to
.github/workflows/integ.yml(+ the integ smoke test), both about the integ harness's teardown/naming plumbing.1. Remove the broken
Ensure stack torn downsafety net (#566)The
if: always()backstop ran rawaws cloudformation delete-stack/wait stack-delete-completeas the GitHub OIDC role, which has nocloudformation:DeleteStackorDescribeStackspermission. Both callsAccessDeniedon every run; the un-guardedwaitexits 255 and fails the job even when the integ test passed and integ-runner already destroyed the stack.Evidence — run 28892156232:
SUCCESS integ.task-api-smoke-TaskApiSmoke/DefaultTest 291.541s(integ-runner's own deploy → assert → destroy completed), then the safety-net step failed withAccessDeniedon both CloudFormation calls.integ-runner remains the teardown path — it forces
cdk destroyon success and failure via the CDK bootstrap roles (which hold the CloudFormation permissions). Stranded-stack reclaim for cancelled/crashed runs is owned by #400 (non-blocking cleanup) and #72 (scheduled sweep).2. Map the integ stack name to the commit SHA (
int-<short-sha>-<run-number>)Replaces the single hardcoded
backgroundagent-integstack name with a run-uniqueint-<short-sha>-<run-number>derived from the resolved head SHA, threaded into integ-runner via a newINTEG_STACK_NAMEenv var. The test falls back tobackgroundagent-integ-localso the localmise //cdk:integpath is unchanged.Why it pairs with #1: with the blocking safety net gone, a stranded stack from a cancelled/crashed run would otherwise sit under one fixed name and head-of-line block every future run (can't
CREATEa name that already exists inDELETE_FAILED, andcancel-in-progress: falsequeues behind it). A run-unique name removes that failure mode (the run number also makes same-commit re-runs distinct); theint-prefix marks the stack ephemeral for #72's sweep. The concurrency group is now a cost/parallelism bound, not a correctness requirement.Validation
mise //cdk:compile— cleanmise //cdk:eslint— cleanzizmor .github/workflows/integ.yml— no findingsArea
tooling— rootmise.toml, scripts, CI workflowscdk— infrastructure, handlers, constructs (integ smoke test only)Related
Fixes #566. Partially addresses #400 (naming). Related: #72, #236.
Changes
Ensure stack torn downstep frominteg.yml; update comments so the teardown story stays accurate (integ-runner in-run destroy; feat(cdk): integ harness — ephemeral stack name + offline cdk-diff gate + non-blocking stranded-stack cleanup #400/feat: automated ephemeral stack cleanup (scheduled) #72 for stranded stacks).int-<short-sha>-<run-number>from the resolved head SHA + run number and pass it viaINTEG_STACK_NAMEtomise //cdk:integ.integ.task-api-smoke.tsreadsINTEG_STACK_NAME, defaulting tobackgroundagent-integ-localfor the unchanged local path.Acknowledgment
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of the project license.