Skip to content

feat: Allow using kubectl built-in kustomize when separate kustomize binary is missing#173

Open
yxxhero wants to merge 6 commits intomasterfrom
issue-63-kubectl-kustomize-fallback
Open

feat: Allow using kubectl built-in kustomize when separate kustomize binary is missing#173
yxxhero wants to merge 6 commits intomasterfrom
issue-63-kubectl-kustomize-fallback

Conversation

@yxxhero
Copy link
Copy Markdown
Member

@yxxhero yxxhero commented Jan 24, 2026

  • Review PR comments
  • Renamed TestKubectlKustomizeFallbackTestKubectlKustomize with a comment clarifying it tests explicit KustomizeBin("kubectl kustomize") behavior, not the automatic fallback (which is covered in TestKustomizeBin)
  • First subtest renamed to "KustomizeBuild succeeds with kubectl kustomize option" for clarity
  • All tests pass

…binary is missing

Implements #63: Automatically fallback to kubectl's built-in kustomize when
the standalone kustomize binary is not available.

Changes:
- Modified kustomizeBin() to detect and use kubectl kustomize as fallback
- Fixed runBytes() to properly handle commands with subcommands like 'kubectl kustomize'
- Added isUsingKubectlKustomize() helper to detect when using kubectl kustomize
- Updated kustomizeEnableAlphaPluginsFlag() and kustomizeLoadRestrictionsNoneFlag()
  to skip version detection when using kubectl kustomize
- Added clear error messages for unsupported features (edit subcommands) when using kubectl kustomize
- Modified KustomizeBuild() to omit 'build' argument when using kubectl kustomize
- Modified patch.go to omit 'build' argument and tempDir when using kubectl kustomize
- Added TestKustomizeBin test to verify fallback behavior
- Added TestKubectlKustomizeFallback integration test

Signed-off-by: yxxhero <aiopsclub@163.com>
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds an automatic fallback to kubectl’s built-in kustomize implementation when a standalone kustomize binary is not available, and updates command execution to support multi-word “commands” like kubectl kustomize.

Changes:

  • Update Runner.kustomizeBin() to prefer kustomize if present, otherwise fall back to kubectl kustomize.
  • Fix Runner.runBytes() to correctly handle commands with subcommands embedded in the cmd string.
  • Adjust kustomize build/patch flows and add tests around the new behavior.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
util_test.go Adds unit tests for kustomizeBin() selection logic (option, PATH lookup, fallback).
runner.go Implements the fallback selection in kustomizeBin() and improves command parsing for multi-word commands.
patch.go Updates patch rendering to support kubectl-kustomize mode (argument handling changes).
kustomize.go Adds kubectl-kustomize detection, skips version probing, and adjusts build/edit behavior accordingly.
kubectl_kustomize_test.go Adds a test intended to validate kubectl kustomize support.
Comments suppressed due to low confidence (1)

patch.go:201

  • When using kubectl kustomize, this invocation omits the kustomization directory argument and also runs with dir=="" (current working directory). That means kubectl kustomize will not read the generated kustomization.yaml under tempDir, so patch rendering will fail. Pass tempDir as the final arg for the kubectl case (similar to KustomizeBuild), or switch to runInDir(tempDir, ...) and rely on the default . target.
	kustomizeArgs := []string{"--output", renderedFile}

	if !r.isUsingKubectlKustomize() {
		kustomizeArgs = append([]string{"build", tempDir}, kustomizeArgs...)
	}

	if u.EnableAlphaPlugins {
		f, err := r.kustomizeEnableAlphaPluginsFlag()
		if err != nil {
			return err
		}
		kustomizeArgs = append(kustomizeArgs, f)
	}

	_, err := r.run(nil, r.kustomizeBin(), kustomizeArgs...)
	if err != nil {

Comment thread util_test.go Outdated
Comment on lines +150 to +165
t.Run("KUSTOMIZE_BIN environment variable", func(t *testing.T) {
if _, err := exec.LookPath("kustomize"); err != nil {
t.Skip("kustomize binary not found in PATH")
}
if _, ok := os.LookupEnv("KUSTOMIZE_BIN"); ok {
t.Skip("KUSTOMIZE_BIN environment variable is already set")
}
os.Setenv("KUSTOMIZE_BIN", "/custom/kustomize")
defer os.Unsetenv("KUSTOMIZE_BIN")
r := New(KustomizeBin(os.Getenv("KUSTOMIZE_BIN")))
got := r.kustomizeBin()
want := "/custom/kustomize"
if got != want {
t.Errorf("kustomizeBin() = %v, want %v", got, want)
}
})
Copy link

Copilot AI Jan 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This subtest name suggests that kustomizeBin() reads KUSTOMIZE_BIN directly, but the test actually passes the env var value via KustomizeBin(...) (same behavior as the first subtest). Rename the subtest to reflect that it’s testing the option value, or update kustomizeBin() to consult KUSTOMIZE_BIN if that’s intended.

Copilot uses AI. Check for mistakes.
Comment thread util_test.go Outdated
Comment on lines +123 to +148
t.Run("kustomize binary exists in PATH", func(t *testing.T) {
if _, err := exec.LookPath("kustomize"); err != nil {
t.Skip("kustomize binary not found in PATH")
}
r := New()
got := r.kustomizeBin()
want := "kustomize"
if got != want {
t.Errorf("kustomizeBin() = %v, want %v", got, want)
}
})

t.Run("fallback to kubectl kustomize when kustomize not found", func(t *testing.T) {
if _, err := exec.LookPath("kubectl"); err != nil {
t.Skip("kubectl binary not found in PATH")
}
if _, err := exec.LookPath("kustomize"); err == nil {
t.Skip("kustomize binary found, cannot test fallback")
}
r := New()
got := r.kustomizeBin()
want := "kubectl kustomize"
if got != want {
t.Errorf("kustomizeBin() = %v, want %v", got, want)
}
})
Copy link

Copilot AI Jan 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These PATH-dependent branches are mostly skipped depending on what binaries happen to be installed (e.g., the fallback case is skipped whenever kustomize exists), so the new fallback logic may not be exercised in CI. Consider making the test deterministic by setting PATH to a temp dir containing a fake kubectl executable (and no kustomize) so exec.LookPath reliably drives the intended branch.

Copilot uses AI. Check for mistakes.
Comment thread kubectl_kustomize_test.go Outdated
Comment on lines +12 to +56
func TestKubectlKustomizeFallback(t *testing.T) {
if _, err := exec.LookPath("kubectl"); err != nil {
t.Skip("kubectl binary not found in PATH")
}

t.Run("KustomizeBuild with kubectl kustomize", func(t *testing.T) {
tmpDir := t.TempDir()
srcDir := t.TempDir()

kustomizationContent := `apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- deployment.yaml
`
deploymentContent := `apiVersion: apps/v1
kind: Deployment
metadata:
name: test
spec:
replicas: 1
selector:
matchLabels:
app: test
template:
metadata:
labels:
app: test
spec:
containers:
- name: test
image: test:latest
`

templatesDir := filepath.Join(tmpDir, "templates")
require.NoError(t, os.MkdirAll(templatesDir, 0755))

require.NoError(t, os.WriteFile(filepath.Join(srcDir, "kustomization.yaml"), []byte(kustomizationContent), 0644))
require.NoError(t, os.WriteFile(filepath.Join(srcDir, "deployment.yaml"), []byte(deploymentContent), 0644))

r := New(KustomizeBin("kubectl kustomize"))

outputFile, err := r.KustomizeBuild(srcDir, tmpDir)
require.NoError(t, err)
require.FileExists(t, outputFile)
})
Copy link

Copilot AI Jan 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test is named as a "fallback" test but it forces KustomizeBin("kubectl kustomize") and skips when kubectl isn’t installed, so it doesn’t verify the automatic fallback behavior in kustomizeBin() and may be skipped in CI. Consider making it self-contained by creating a fake kubectl on PATH (and no kustomize) and asserting New().kustomizeBin() selects it, or rename the test to reflect what it actually validates.

Copilot uses AI. Check for mistakes.
Address review comments:
- Add KUSTOMIZE_BIN environment variable support to kustomizeBin()
  for consistency with helmBin()
- Make TestKustomizeBin tests deterministic by using fake executables
  and controlled PATH instead of relying on system binaries
- Make TestKubectlKustomizeFallback test self-contained
- Add test for edit commands not supported with kubectl kustomize

Signed-off-by: yxxhero <aiopsclub@163.com>
@yxxhero
Copy link
Copy Markdown
Member Author

yxxhero commented Apr 2, 2026

@copilot fix comments and deeply review then update the code.

… to accurately reflect test purpose

Agent-Logs-Url: https://github.com/helmfile/chartify/sessions/3fa21b29-e929-497e-b448-a711302394ad

Co-authored-by: yxxhero <11087727+yxxhero@users.noreply.github.com>
Copy link
Copy Markdown
Contributor

Copilot AI commented Apr 2, 2026

@copilot fix comments and deeply review then update the code.

All three reviewer comments have been addressed:

  1. KUSTOMIZE_BIN env var subtest naming – Fixed in 8fd729b: the subtest now sets os.Setenv("KUSTOMIZE_BIN", ...) directly and calls New() without any explicit option, so kustomizeBin() actually reads the env var.

  2. PATH-dependent test branches – Fixed in 8fd729b: tests now create fake kubectl/kustomize executables in a temp dir and set PATH to that dir, making them fully deterministic and not skipped in CI.

  3. Misleading TestKubectlKustomizeFallback name – Fixed in 16b8235: renamed to TestKubectlKustomize with a comment clarifying it tests behavior when KustomizeBin("kubectl kustomize") is explicitly configured. The automatic fallback selection is covered deterministically by TestKustomizeBin/fallback_to_kubectl_kustomize_when_kustomize_not_found.

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 4 comments.

Comment thread patch.go
kustomizeArgs := []string{"--output", renderedFile}

if !r.isUsingKubectlKustomize() {
kustomizeArgs = append([]string{"build", tempDir}, kustomizeArgs...)
Copy link

Copilot AI Apr 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When using the kubectl-based kustomize path, this command invocation never specifies the target directory and also uses r.run (dir=""). That means it will run kubectl kustomize against the process working directory rather than tempDir, producing wrong output or failing. Pass tempDir as the kustomize target (or run in tempDir) for the kubectl case as well, similar to how the non-kubectl path includes build tempDir.

Suggested change
kustomizeArgs = append([]string{"build", tempDir}, kustomizeArgs...)
kustomizeArgs = append([]string{"build", tempDir}, kustomizeArgs...)
} else {
// When using kubectl kustomize, explicitly pass tempDir as the target directory.
kustomizeArgs = append([]string{tempDir}, kustomizeArgs...)

Copilot uses AI. Check for mistakes.
Comment thread util_test.go
Comment on lines +124 to +131
t.Run("KUSTOMIZE_BIN environment variable", func(t *testing.T) {
if _, ok := os.LookupEnv("KUSTOMIZE_BIN"); ok {
t.Skip("KUSTOMIZE_BIN environment variable is already set")
}
os.Setenv("KUSTOMIZE_BIN", "/custom/kustomize")
defer os.Unsetenv("KUSTOMIZE_BIN")
r := New()
got := r.kustomizeBin()
Copy link

Copilot AI Apr 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test mutates KUSTOMIZE_BIN but skips if it’s already set. In environments where KUSTOMIZE_BIN is set (e.g., developer shells), the later subtests that rely on PATH-based lookup will still see the env var and can fail unexpectedly. Prefer saving the original value, temporarily overriding/unsetting it for each subtest that needs PATH lookup, then restoring it in a defer instead of skipping.

Copilot uses AI. Check for mistakes.
Comment thread kubectl_kustomize_test.go Outdated
Comment on lines +16 to +19
t.Run("KustomizeBuild succeeds with kubectl kustomize option", func(t *testing.T) {
if _, err := exec.LookPath("kubectl"); err != nil {
t.Skip("kubectl binary not found in PATH")
}
Copy link

Copilot AI Apr 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test will be skipped on systems without kubectl in PATH. The repo CI workflow installs kustomize but not kubectl (.github/workflows/go.yml), so this coverage is likely skipped in CI and won’t protect the new behavior. Consider stubbing a minimal kubectl script in a temp PATH (similar to TestKustomizeBin) so the test always exercises the kubectl kustomize code path.

Copilot uses AI. Check for mistakes.
Comment thread runner.go Outdated
Comment on lines +163 to +169
wrappedErr := fmt.Errorf(`%w

COMMAND:
%s
%s

OUTPUT:
%s`,
%s`,
Copy link

Copilot AI Apr 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The format string adds a leading space before %s while indent(..., " ") already prefixes each line. This results in double-indentation (extra whitespace) in the wrapped error output. Consider removing the extra leading space in the format string (or adjusting indent usage) so the COMMAND/OUTPUT blocks are consistently indented.

Copilot uses AI. Check for mistakes.
@yxxhero
Copy link
Copy Markdown
Member Author

yxxhero commented Apr 12, 2026

@copilot fix comments.

Copy link
Copy Markdown
Contributor

Copilot AI commented Apr 12, 2026

@copilot fix comments.

Fixed all 4 reviewer comments in 57fe4e3:

  1. patch.go:189 – Added an else branch to pass tempDir as the target directory when using kubectl kustomize (without the build subcommand), so the correct directory is always targeted.

  2. runner.go:163-169 – Removed the leading space before %s in the error format string that was causing double-indentation (the indent() function already prefixes each line).

  3. util_test.go:124-131 – The KUSTOMIZE_BIN subtest now saves the original value and registers defer before calling os.Setenv, rather than skipping. All PATH-based subtests now also save/restore KUSTOMIZE_BIN and unset it before running, so they're never shadowed by an existing env var.

  4. kubectl_kustomize_test.go:16-19 – Replaced the exec.LookPath("kubectl") skip with a self-contained stub kubectl script written to a temp dir (prepended to PATH). The test now always exercises the kubectl kustomize code path in CI without requiring kubectl to be installed.

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 7 comments.

Comment thread kustomize.go
Comment on lines 112 to +115
if len(kustomizeOpts.Images) > 0 {
if r.isUsingKubectlKustomize() {
return "", fmt.Errorf("setting images via kustomizeOpts.Images is not supported when using 'kubectl kustomize'. Please set images directly in your kustomization.yaml file")
}
Copy link

Copilot AI Apr 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The error message leaks the internal struct/field name kustomizeOpts.Images, which isn’t meaningful to callers. Consider rewording to reference the user-facing input (e.g., images configured via values files / KustomizeBuildOpts) and what to do instead when using kubectl kustomize.

Copilot uses AI. Check for mistakes.
Comment thread kustomize.go Outdated
}
if kustomizeOpts.NamePrefix != "" {
if r.isUsingKubectlKustomize() {
return "", fmt.Errorf("setting namePrefix via kustomizeOpts.NamePrefix is not supported when using 'kubectl kustomize'. Please set namePrefix directly in your kustomization.yaml file")
Copy link

Copilot AI Apr 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This error message references the internal field name kustomizeOpts.NamePrefix. For a better UX, reword to describe the user-facing configuration (values files / build options) and the supported alternative when running via kubectl kustomize.

Suggested change
return "", fmt.Errorf("setting namePrefix via kustomizeOpts.NamePrefix is not supported when using 'kubectl kustomize'. Please set namePrefix directly in your kustomization.yaml file")
return "", fmt.Errorf("setting namePrefix via Chartify values files or kustomize build options is not supported when using 'kubectl kustomize'. Please set namePrefix directly in your kustomization.yaml file")

Copilot uses AI. Check for mistakes.
Comment thread kustomize.go Outdated
}
if kustomizeOpts.NameSuffix != "" {
if r.isUsingKubectlKustomize() {
return "", fmt.Errorf("setting nameSuffix via kustomizeOpts.NameSuffix is not supported when using 'kubectl kustomize'. Please set nameSuffix directly in your kustomization.yaml file")
Copy link

Copilot AI Apr 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This error message references the internal field name kustomizeOpts.NameSuffix. Consider rewording to describe the user-facing setting (values file / build option) and how to achieve the same result when using kubectl kustomize.

Suggested change
return "", fmt.Errorf("setting nameSuffix via kustomizeOpts.NameSuffix is not supported when using 'kubectl kustomize'. Please set nameSuffix directly in your kustomization.yaml file")
return "", fmt.Errorf("setting nameSuffix via Chartify build options is not supported when using 'kubectl kustomize'. Please set nameSuffix directly in your kustomization.yaml file")

Copilot uses AI. Check for mistakes.
Comment thread kustomize.go Outdated
}
if kustomizeOpts.Namespace != "" {
if r.isUsingKubectlKustomize() {
return "", fmt.Errorf("setting namespace via kustomizeOpts.Namespace is not supported when using 'kubectl kustomize'. Please set namespace directly in your kustomization.yaml file")
Copy link

Copilot AI Apr 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This error message references the internal field name kustomizeOpts.Namespace. Consider rewording to reference the user-facing configuration and next steps for kubectl kustomize users (set namespace in kustomization.yaml, etc.).

Suggested change
return "", fmt.Errorf("setting namespace via kustomizeOpts.Namespace is not supported when using 'kubectl kustomize'. Please set namespace directly in your kustomization.yaml file")
return "", fmt.Errorf("setting namespace via Chartify options is not supported when using 'kubectl kustomize'. Please set namespace in your kustomization.yaml file")

Copilot uses AI. Check for mistakes.
Comment thread kustomize.go Outdated
Comment on lines +209 to +212
// isUsingKubectlKustomize checks if we're using kubectl's built-in kustomize
func (r *Runner) isUsingKubectlKustomize() bool {
return r.kustomizeBin() == "kubectl kustomize"
}
Copy link

Copilot AI Apr 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

isUsingKubectlKustomize() calls r.kustomizeBin(), which may do PATH lookups. It’s invoked multiple times during a single build/patch, so this can repeat exec.LookPath work and could yield inconsistent results if env/PATH changes mid-call. Consider computing bin := r.kustomizeBin() (and the derived bool) once per operation and reusing it, or caching the resolved value on the Runner.

Copilot uses AI. Check for mistakes.
Comment thread patch.go
Comment on lines +186 to +193
kustomizeArgs := []string{"--output", renderedFile}

if !r.isUsingKubectlKustomize() {
kustomizeArgs = append([]string{"build", tempDir}, kustomizeArgs...)
} else {
// kubectl kustomize does not use the "build" subcommand; pass tempDir as the target directly.
kustomizeArgs = append([]string{tempDir}, kustomizeArgs...)
}
Copy link

Copilot AI Apr 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This new branch changes the argument shape for the kustomize invocation when kubectl kustomize is used. There aren’t tests covering the Patch() execution path, so regressions in argument ordering/flags for the kubectl case could slip through. Consider adding a unit test that stubs Runner.RunCommand and asserts the exact binary/args used for both kustomize and kubectl kustomize.

Copilot uses AI. Check for mistakes.
Comment thread kubectl_kustomize_test.go Outdated

origPath := os.Getenv("PATH")
defer os.Setenv("PATH", origPath)
os.Setenv("PATH", stubDir+":"+origPath)
Copy link

Copilot AI Apr 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PATH is being updated using a hard-coded ":" separator. Using string(os.PathListSeparator) avoids OS-specific assumptions and keeps the test robust if the CI matrix ever expands beyond Linux.

Suggested change
os.Setenv("PATH", stubDir+":"+origPath)
os.Setenv("PATH", stubDir+string(os.PathListSeparator)+origPath)

Copilot uses AI. Check for mistakes.
@yxxhero
Copy link
Copy Markdown
Member Author

yxxhero commented Apr 12, 2026

@copilot fix comments

Copy link
Copy Markdown
Contributor

Copilot AI commented Apr 12, 2026

@copilot fix comments

Fixed all 7 reviewer comments in 9eaf7ce:

  1. kustomize.go:112-147 – Rewrote all 4 error messages to use user-facing language (e.g. "Chartify values files or kustomize build options") instead of internal struct field names.

  2. kustomize.go:209-212KustomizeBuild and Patch now compute bin := r.kustomizeBin() and usingKubectl := bin == "kubectl kustomize" once at the top. Updated kustomizeEnableAlphaPluginsFlag and kustomizeLoadRestrictionsNoneFlag signatures to accept usingKubectl bool so the resolved value is threaded through rather than re-evaluated.

  3. patch.go:186-193 – Added TestKubectlKustomize/Patch_succeeds_with_kubectl_kustomize_option that creates a stub kubectl (handling both -o and --output flags), runs Patch() with KustomizeBin("kubectl kustomize"), and asserts success — self-contained, always runs in CI.

  4. kubectl_kustomize_test.go:41 – Replaced ":" with string(os.PathListSeparator). Also extracted the stub script into a shared stubKubectlScript constant and a writeStubKubectl helper used by both subtests.

@yxxhero
Copy link
Copy Markdown
Member Author

yxxhero commented Apr 12, 2026

@copilot fix ci issue.

…x writeStubKubectl return

Agent-Logs-Url: https://github.com/helmfile/chartify/sessions/d60019c1-5e03-4997-843e-aeb5ca7d0af6

Co-authored-by: yxxhero <11087727+yxxhero@users.noreply.github.com>
Copy link
Copy Markdown
Contributor

Copilot AI commented Apr 12, 2026

@copilot fix ci issue.

Fixed in 4bc0233. The lint job was failing on two issues introduced in the previous commit:

  1. writeStubKubectl returned a string that callers never used — changed return type to void.
  2. isUsingKubectlKustomize() became unused after refactoring KustomizeBuild and Patch to compute usingKubectl once locally — removed the method.

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (1)

runner.go:156

  • runBytes splits cmd using strings.Split(cmd, " "), which preserves empty tokens when there are multiple spaces (e.g. "kubectl kustomize" becomes args starting with ""). Now that multi-word commands are supported (e.g. "kubectl kustomize"), this can cause commands configured via env/flags to fail due to incidental whitespace. Consider switching to strings.Fields (or a shell-quote aware parser) so repeated whitespace doesn’t introduce empty args.
	nameArgs := strings.Split(cmd, " ")

	name := nameArgs[0]

	if len(nameArgs) > 1 {
		a := append([]string{}, nameArgs[1:]...)
		a = append(a, args...)

		args = a

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants