Skip to content

fix(scripts): add OIDCIssuerHost to generate-federation-iac and test for drift - #1256

Draft
IAmJSD wants to merge 1 commit into
LeanerCloud:mainfrom
IAmJSD:fix/issue-153
Draft

IAmJSD wants to merge 1 commit into
LeanerCloud:mainfrom
IAmJSD:fix/issue-153

Conversation

@IAmJSD

@IAmJSD IAmJSD commented Oct 23, 2026

Copy link
Copy Markdown
Collaborator

Root cause

scripts/generate-federation-iac.go renders the same templates as the server's GET /api/federation/iac, but from its own copy of the template data struct. iacData's doc comment said "Keep in sync with any template variable changes", and nothing enforced that. By the time of this PR, #1710 had added SourceAccountID, CUDlyAPIURL and ContactEmail. One field was still missing: OIDCIssuerHost, which aws-cfn-deploy.sh.tmpl references. So --format bundle still failed for every AWS target with a non-AWS source. Reproduced on main (a14dc8f):

$ go run scripts/generate-federation-iac.go --target aws --source gcp \
    --account-name prod --account-id 123456789012 \
    --oidc-subject-claim 123456789012345678901 --format bundle --output "$T/b.zip"
Error: render internal/iacfiles/templates/aws-cfn-deploy.sh.tmpl: template: iac:36:22: executing "iac" at <.OIDCIssuerHost>: can't evaluate field OIDCIssuerHost in type main.iacData

The existing end-to-end tests didn't catch this. They cover tfvars for every combination but bundle only for Azure targets, and TestGenerator_ShellTemplatesEscapeValues swaps in a probe for aws-cfn-deploy.sh.tmpl precisely because the real template couldn't be rendered.

Fix

  1. Field. iacData gains OIDCIssuerHost. populateData derives it from OIDCIssuerURL with the same strings.TrimPrefix(..., "https://") that buildGenericIaCData uses on the server. escapeData walks every string field by reflection, so the new field is shell-escaped in .sh output without any extra code.
  2. Structural drift guard (new scripts/generate_federation_iac_drift_test.go):
    • TestIaCData_MatchesServerTemplateData reads iacData and federationIaCData from source with go/parser and requires the same field set.
    • TestIaCData_CoversEveryTemplateField parses every internal/iacfiles/templates/*.tmpl with text/template, walks the parse tree, and fails if a template references a field iacData lacks. This catches a new template field before anyone runs the script. If a template ever uses with or range, the test fails with "extend templateFieldRefs" rather than guessing what dot means.
    • TestIaCData_FieldWalkerSeesKnownFields is a positive control, so a walker that finds nothing can't make the two tests above pass vacuously.
    • TestGenerator_AWSWIFBundleRendersDeployScript runs the real --format bundle for aws←gcp and aws←azure and checks the OIDCIssuerHost= value in cloudformation/deploy-cfn.sh. It only reads the rendered script and never runs it, since running it would call the AWS CLI.
  3. Doc comment. It no longer claims an invariant nothing enforced. It now says the script can't import the server's type and names the tests that keep the two in step.
  4. Stale test rationales (audit A14-028):
    • TestGenerator_CrossAccountDoesNotRequireSubjectClaim now passes --source-account-id, asserts exit 0, and checks that the target account ID is rendered. Before, it only checked stderr, so it passed whatever the aws→aws path returned.
    • Updated the comments on TestGenerator_AcceptsValidOIDCSubjectClaim and probeDeployScript, which said the tfvars and deploy-script paths could not render.

Why a source-level check instead of sharing one type

The issue body asks not to couple the script's struct to internal/api. Importing internal/api (or moving the type into internal/iacfiles) would make a //go:build ignore dev tool depend on a large server package. The drift tests read both files as source and import neither, so the script stays decoupled while the drift still fails CI. cristim's follow-up comment suggested this pair of checks (field-name comparison plus every-template coverage).

Closes #153

cristim's follow-ups on the issue (from cli#1691)

  1. Missing SourceAccountID / full field list. In scope and fixed. SourceAccountID, CUDlyAPIURL and ContactEmail and their flags were already on main via #1710. This PR adds the remaining OIDCIssuerHost, derived as the server does, plus the drift tests above.
  2. shellEscape does not escape }. Out of scope and still present on main. internal/api/handler_federation.go shellEscape (and the script's copy) still leave } alone. 8 templates still put {{.ContactEmail}} inside "${CUDLY_CONTACT_EMAIL:-...}": aws-cfn-deploy, aws-cross-account-cfn-deploy, aws-cross-account-cli, aws-wif-cli, azure-wif-cli, azure-wif-deploy, gcp-sa-impersonation-cli, gcp-wif-cli. Only the bicep deploy script was fixed, in fix(iac): keep '}' and JSON metacharacters in the Bicep/ARM contact email intact (#963 follow-ups) #992 (Follow-ups from #963 review: '}' and JSON metacharacters in the Bicep/ARM contact email #991). This is a server-side escaping change, not generator/handler drift, and it touches the server's CLI templates, so it needs its own issue. As the issue comment notes, it can't run commands, because $, `, " and \ are all escaped.
  3. Render paths that bypass escaping. Already fixed on main, nothing to do. The bicep azure-wif-bicep-deploy.sh.tmpl render now goes through shellEscapeData (sec(api): shell-escape the Bicep/ARM deploy script render (#958) #963), and the .auto.tfvars render goes through hclEscapeData.

Out of scope / noted

  • Running golangci-lint directly on scripts/generate-federation-iac.go reports 11 issues on lines this PR does not touch, e.g. recognise (misspell), gosec G306 on 0o644, revive error-strings. CI doesn't lint this file because it carries //go:build ignore, and none of the findings are on changed lines.

Verification

Run on this branch with AWS_CONFIG_FILE=/dev/null AWS_SHARED_CREDENTIALS_FILE=/dev/null AWS_EC2_METADATA_DISABLED=true and Go 1.26.9. This is unit and subprocess evidence: the tests run the compiled script against the repo's real templates. No cloud calls.

Fail before the fix (new tests, script unchanged):

$ (cd scripts && go test -count=1 -v -run 'TestIaCData_|TestGenerator_AWSWIFBundleRendersDeployScript' .)
--- FAIL: TestIaCData_MatchesServerTemplateData        (script lacks "OIDCIssuerHost")
--- FAIL: TestIaCData_CoversEveryTemplateField
        Should be empty, but was [OIDCIssuerHost (used by aws-cfn-deploy.sh.tmpl)]
--- PASS: TestIaCData_FieldWalkerSeesKnownFields
--- FAIL: TestGenerator_AWSWIFBundleRendersDeployScript/gcp
--- FAIL: TestGenerator_AWSWIFBundleRendersDeployScript/azure
        stderr: Error: render internal/iacfiles/templates/aws-cfn-deploy.sh.tmpl: ... can't evaluate field OIDCIssuerHost in type main.iacData
FAIL

Pass after:

$ (cd scripts && go test -count=1 -v -run 'TestIaCData_|TestGenerator_AWSWIFBundleRendersDeployScript|TestGenerator_CrossAccountDoesNotRequireSubjectClaim' .)
--- PASS: TestIaCData_MatchesServerTemplateData
--- PASS: TestIaCData_CoversEveryTemplateField
--- PASS: TestIaCData_FieldWalkerSeesKnownFields
--- PASS: TestGenerator_AWSWIFBundleRendersDeployScript (gcp, azure)
--- PASS: TestGenerator_CrossAccountDoesNotRequireSubjectClaim
ok  	github.com/LeanerCloud/cloud-commitments-platform/scripts
$ (cd scripts && go test -count=1 .)
ok  	github.com/LeanerCloud/cloud-commitments-platform/scripts	7.253s

Mutation probes. Each was applied temporarily and reverted, and each one fails the tests:

Mutation Caught by
drop the OIDCIssuerHost = TrimPrefix(...) line, keep the field TestGenerator_AWSWIFBundleRendersDeployScript (gcp, azure)
add {{.NewField}} to gcp-wif.tfvars.tmpl TestIaCData_CoversEveryTemplateField: NewField (used by gcp-wif.tfvars.tmpl)
add a field to the server's federationIaCData only TestIaCData_MatchesServerTemplateData
add a {{with .AccountName}} block to a template TestIaCData_CoversEveryTemplateField: "*parse.WithNode ... extend templateFieldRefs"

Other checks:

  • go build ./..., go vet ./..., go vet -tags integration ./..., go vet scripts/generate-federation-iac.go: all clean.
  • golangci-lint run --allow-parallel-runners ./scripts/... (v2.10.1): 0 issues.
  • gocyclo -over 10 -ignore "_test\.go" scripts/generate-federation-iac.go: no output.
  • go test ./internal/mocks/ -count=1: ok.

Combined-tree verification. This branch was tested together with six other queued branches on a local, never-pushed tree: queue/platform @ 77d116c6 = origin/main a14dc8f2 plus these heads, merged in this order:

Branch Tested head
fix/issue-47 62feb4f
fix/issue-76 3b8c241
fix/issue-1233 e1638b7
fix/issue-153 8729074
fix/issue-532 0971624
fix/issue-64 fefdc43
fix/issue-54 91827b9

Linux, Go 1.26.9, AWS_CONFIG_FILE=/dev/null AWS_SHARED_CREDENTIALS_FILE=/dev/null AWS_EC2_METADATA_DISABLED=true; Docker Postgres via testcontainers:

  • Passed: go build ./..., go vet ./..., go vet -tags integration ./... and the full go test -race -short -count=1 ./....
  • Static checks (changed packages): golangci-lint v2.10.1 found 0 issues, with and without --build-tags integration, and gocyclo -over 10 is silent on the changed non-test files. These ran at b61a9c30, which differs from 77d116c6 only by one .gitallowed line on fix/issue-1233.
  • Integration tests, every changed package, one package at a time (-tags integration -count=1):
    • These packages pass: cmd/cleanup-lambda, internal/analytics, internal/auth, internal/database/postgres/migrations and internal/purchase.
    • internal/api passes, except two tests whose Postgres container didn't start, or whose migrations hit a timeout, under machine load. Rerun alone on the same tree, both pass: TestDeleteManagedPlanByExecutionStatus and TestNotifiedRow_RescheduleDoesNotOverwriteConcurrentCancel.
    • internal/config passes except TestPostgresStoreDB_ResolveStaleRIExchange_ValidatesCloudCheck, which fails the same way on unmodified main (fixed by fix(config): compare resolve checked_at at the database's microsecond precision #1252).
    • Packages were run with a 60-minute timeout (rather than CI's 20) because the VM was shared with other test runs. Some reruns used TESTCONTAINERS_RYUK_DISABLED=true to avoid a testcontainers reaper race under load.
  • Secrets: git-secrets, using the repo's scripts/setup-git-secrets.sh patterns in a fresh repo with an isolated HOME, found nothing in the changed files, and a planted fake key was flagged. scripts/test-git-secrets-allowlist.sh: 137 PASS, 0 FAIL.
  • Nothing was applied to any cloud.

Labels

A maintainer, please mirror the issue's triage labels onto this PR: impact/internal, triaged, priority/p3, severity/low, urgency/eventually, effort/xs, type/bug.

🤖 Generated with Claude Code

…for drift

iacData in scripts/generate-federation-iac.go lacked OIDCIssuerHost, which
aws-cfn-deploy.sh.tmpl references, so --format bundle failed for every AWS
target with a non-AWS source. Add the field and derive it from
OIDCIssuerURL the same way the server does.

The doc comment asked people to keep iacData in sync with the server's
federationIaCData, and nothing enforced it. New tests read both structs
with go/parser and every template with text/template: they fail when the
two structs stop having the same fields, or when any template references
a field iacData lacks. Doing it from source keeps the script decoupled
from internal/api. The doc comment now points at these tests.

Also drop the stale "iacData is missing fields" rationale from three
tests. TestGenerator_CrossAccountDoesNotRequireSubjectClaim now asserts
a successful render instead of only checking stderr (audit A14-028).

Closes LeanerCloud#153

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@IAmJSD IAmJSD added impact/internal Team-internal only triaged Item has been triaged priority/p3 Polish / idea / may never ship severity/low Minor harm urgency/eventually No deadline effort/xs Trivial / one-liner type/bug Defect labels Oct 23, 2026
@coderabbitai

coderabbitai Bot commented Oct 23, 2026

Copy link
Copy Markdown
Contributor

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

@IAmJSD

IAmJSD commented Oct 23, 2026

Copy link
Copy Markdown
Collaborator Author

Heads-up on an interaction with a queued branch: fix/issue-324 (IAmJSD fork, PR not yet open) removes the CloudFormation OIDCIssuerHost parameter entirely and derives the host from OIDCIssuerURL (audit A13b-014). If that lands first, this PR's OIDCIssuerHost generator field and the drift-test expectations for it become unnecessary, and this PR will be updated to drop them. If this PR lands first, #324's branch will drop the field from the generator and adjust these drift tests. Either way the combined result is a generator with no OIDCIssuerHost.

@IAmJSD

IAmJSD commented Oct 23, 2026

Copy link
Copy Markdown
Collaborator Author

Follow-up to my note above: the fix/issue-324 branch is now #1312. Whichever of #1256 and #1312 merges second drops the OIDCIssuerHost generator field (and this PR's drift-test expectations for it).

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

effort/xs Trivial / one-liner impact/internal Team-internal only priority/p3 Polish / idea / may never ship severity/low Minor harm triaged Item has been triaged type/bug Defect urgency/eventually No deadline

Projects

None yet

Development

Successfully merging this pull request may close these issues.

chore(scripts): generate-federation-iac.go iacData has drifted out of sync with the templates it renders

1 participant