-
Notifications
You must be signed in to change notification settings - Fork 200
Add bump-tf skill #5990
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
pietern
wants to merge
1
commit into
main
Choose a base branch
from
bump-tf-skill
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+162
−0
Open
Add bump-tf skill #5990
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,98 @@ | ||
| --- | ||
| name: bump-tf | ||
| description: "Use when the user says 'bump the TF provider', 'bump terraform provider', 'update the terraform provider to <version>', 'bump TF to latest', or otherwise wants the pinned Databricks Terraform provider version raised and the new provider mappings pulled into the CLI." | ||
| user-invocable: true | ||
| allowed-tools: Read, Edit, Write, Bash, Glob, Grep, AskUserQuestion | ||
| --- | ||
|
|
||
| # Bump Terraform Provider | ||
|
|
||
| The pinned provider version lives in `bundle/internal/tf/codegen/schema/version.go`. | ||
| Everything else (`bundle/internal/tf/schema/*` including `root.go`, and `bundle/terraform_dabs_map/generated.go`) is generated from it, so the only file you hand-edit for the bump itself is `version.go`. | ||
| Do not edit the generated files, and do not touch the `databricks-tf-provider/...` version comment in `libs/testdiff/replacement.go` (the version is masked in acceptance output, so changing it is pure noise). | ||
|
|
||
| ## Steps | ||
|
|
||
| **1. Resolve the target version.** | ||
| Use the version the user gave (strip a leading `v`). | ||
| For "latest"/no version, resolve the newest GitHub release. | ||
| Either way, confirm the tag exists on GitHub, since you can't bump to an unreleased version: | ||
|
|
||
| ```bash | ||
| gh api repos/databricks/terraform-provider-databricks/releases --jq '.[0].tag_name' # latest | ||
| gh api repos/databricks/terraform-provider-databricks/releases/tags/v{version} --jq '.published_at' | ||
| ``` | ||
|
|
||
| **2. Bump the constant.** | ||
| Set `const ProviderVersion = "{version}"` in `bundle/internal/tf/codegen/schema/version.go`. | ||
|
|
||
| **3. Regenerate the schema and field map:** | ||
|
|
||
| ```bash | ||
| ./task generate-tf-schema # regenerates bundle/internal/tf/schema/* (incl. root.go) | ||
| gofmt -s -w bundle/internal/tf/schema | ||
| ./task generate-schema-map # regenerates bundle/terraform_dabs_map/generated.go | ||
| go build ./... | ||
| ``` | ||
|
|
||
| A `Warning: Skipping file generation for databricks_quality_monitor ...` line from codegen is expected. | ||
| If `generate-tf-schema` fails with `no available releases match the given constraints {version}`, the registry hasn't indexed the release yet. | ||
| See [registry-workaround.md](registry-workaround.md), then continue. | ||
|
|
||
| **4. Refresh goldens, then VERIFY.** | ||
|
|
||
| ```bash | ||
| go test ./acceptance -run '^TestAccept$' -update -timeout=60m | ||
| go test ./acceptance -run '^TestAccept$' -timeout=60m # MUST pass on its own | ||
| ``` | ||
|
|
||
| The verify pass is not optional. | ||
| Bundle tests run under an `EnvMatrix` of both engines (`terraform`, `direct`). | ||
| When a schema change makes one engine error while the other succeeds, the variants produce different output; `-update` runs both and each overwrites the other's `output.txt`, so it can silently settle on the passing variant and report `ok` while the golden is actually wrong. | ||
| Only the non-update run catches this. | ||
| (Ignore `rejecting_proxy.go: blocking proxy` log lines, which are normal. | ||
| A test that times out under full parallel load but passes when run alone is a flake, not a regression.) | ||
|
|
||
| **5. Resolve behavior changes** if the verify pass fails. | ||
| Diff the generated schema (`git diff bundle/internal/tf/schema/resource_<x>.go`) and adapt the **test**, not the generated code: | ||
|
|
||
| - *Field became required* → the terraform engine rejects a fixture the direct engine still accepts. | ||
| Add the now-required field to the fixture `databricks.yml` (copy the shape from a sibling test) so both engines match again. | ||
| - *Field removed* but still supported by the direct engine → terraform warns `unknown field: <field>` and drops it. | ||
| Restrict the test to the direct engine, with a comment: | ||
|
|
||
| ```toml | ||
| # The Terraform provider dropped `<field>` in v{version}; direct engine only. | ||
| EnvMatrix.DATABRICKS_BUNDLE_ENGINE = ["direct"] | ||
| ``` | ||
|
|
||
| Regenerate the affected test's `out*` files with `go test ./acceptance -run 'TestAccept/<path>' -update` (never hand-edit them), then re-run the full verify pass. | ||
|
|
||
| **6. Changelog fragment.** | ||
| Add a `dependency-updates` entry per the `pr-checklist` skill's "Changelog entry" section: | ||
|
|
||
| ``` | ||
| Bump Terraform provider from v{old_version} to v{version} (#{pr_number}). | ||
| ``` | ||
|
|
||
| Add it without `(#NNNN)` now; backfill the number after the PR exists, then run `./task links` to expand it into the full markdown link in place and commit the result. | ||
|
|
||
| **7. Commit, push, PR.** | ||
| Run `./task fmt` and `./task lint-q` (if either touches `acceptance/`, a fixture is wrong, so fix the source rather than editing output). | ||
| Commit and push; if the push 403s, the account needs write access to `databricks/cli` (`gh auth switch`). | ||
| Then follow the `pr-checklist` skill for the PR. | ||
| **Do not run `gh pr create` without the user's explicit permission.** | ||
| Commit body and PR description: | ||
|
|
||
| ``` | ||
| ## Changes | ||
|
|
||
| Bump the pinned Databricks Terraform provider from v{old_version} to v{version}. | ||
|
|
||
| Notable schema changes: | ||
| - <one terse bullet per user-visible resource/field change> | ||
|
|
||
| ## Tests | ||
|
|
||
| Acceptance goldens regenerated via `./task test-update`. | ||
| ``` | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,64 @@ | ||
| # Appendix: Terraform Registry lag workaround | ||
|
|
||
| `./task generate-tf-schema` runs the codegen tool, which runs `terraform init`. | ||
| That pulls the provider from the **Terraform Registry** (`registry.terraform.io`), **not** from GitHub. | ||
| A freshly published GitHub release is not indexed by the registry immediately. | ||
| Indexing lags the GitHub release by anywhere from ~30 min to a few hours. | ||
|
|
||
| You only need this workaround if `./task generate-tf-schema` fails with: | ||
|
|
||
| ``` | ||
| Could not retrieve the list of available versions for provider | ||
| databricks/databricks: no available releases match the given constraints {version} | ||
| ``` | ||
|
|
||
| If codegen succeeded, ignore this file. | ||
|
|
||
| ## Fix: point Terraform at a local filesystem mirror | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. have you seen this be needed? |
||
|
|
||
| Download the provider zip from the GitHub release (which always exists) into a filesystem-mirror layout, and set `TF_CLI_CONFIG_FILE` so `terraform init` resolves `databricks/databricks` from the mirror instead of the registry. | ||
| The env var propagates into the `go run .` that `generate-tf-schema` invokes. | ||
|
|
||
| Put the mirror in the repo's own `tmp/` directory, which is gitignored (the `/tmp/` entry in `.gitignore`). | ||
| Do NOT use the system `/tmp`: a sandboxed session may only be allowed to write inside the repo, and a mirror dir placed at a non-ignored path in the repo would pollute the acceptance test scan and get swept into `git add -A`. | ||
|
|
||
| ```bash | ||
| # Download the provider zip from GitHub into a filesystem-mirror layout (repo-root ./tmp is gitignored). | ||
| MIRROR="$(pwd)/tmp/tf_mirror" | ||
| DIR="$MIRROR/registry.terraform.io/databricks/databricks" | ||
| mkdir -p "$DIR" | ||
| gh release download v{version} --repo databricks/terraform-provider-databricks \ | ||
| --pattern 'terraform-provider-databricks_{version}_linux_amd64.zip' \ | ||
| --dir "$DIR" --clobber | ||
|
|
||
| # Tell Terraform to resolve databricks/databricks from the mirror, everything else direct. | ||
| cat > "$MIRROR/cli.tfrc" <<EOF | ||
| provider_installation { | ||
| filesystem_mirror { | ||
| path = "$MIRROR" | ||
| include = ["registry.terraform.io/databricks/databricks"] | ||
| } | ||
| direct { | ||
| exclude = ["registry.terraform.io/databricks/databricks"] | ||
| } | ||
| } | ||
| EOF | ||
|
|
||
| # Re-run codegen against the mirror. | ||
| TF_CLI_CONFIG_FILE="$MIRROR/cli.tfrc" ./task generate-tf-schema | ||
| ``` | ||
|
|
||
| Adjust the `linux_amd64` arch in the download pattern for a different platform, e.g. `darwin_arm64`. | ||
| You can also just re-run `./task generate-tf-schema` without the mirror once the registry catches up, if you prefer. | ||
|
|
||
| Codegen still fetches the real GitHub SHA256 checksums for `root.go`, so nothing is faked. | ||
| The mirror only shortcuts the registry version lookup. | ||
|
|
||
| This workaround is only for codegen (Step 3). | ||
| The acceptance tests (Step 4) fetch the provider straight from the GitHub release via `acceptance/install_terraform.py`, so they do not depend on the registry and need no workaround. | ||
|
|
||
| Clean up the mirror when the bump is done: | ||
|
|
||
| ```bash | ||
| rm -rf ./tmp/tf_mirror | ||
| ``` | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Given very deterministic instructions, can the whole thing be a Python script? The skill can still be there but it runs the script and then debugs if it does not work (and improves the script further).