feat: local template support for azd init and developer tooling improvements#6826
feat: local template support for azd init and developer tooling improvements#6826jongio wants to merge 16 commits intoAzure:mainfrom
azd init and developer tooling improvements#6826Conversation
There was a problem hiding this comment.
Pull request overview
This pull request adds support for using local filesystem directories as template sources in azd init, enabling template developers to iterate on templates without pushing to a remote repository. It also adds developer tooling (mage devinstall, mage preflight) and fixes several pre-existing issues required for CI gates to pass.
Changes:
- Local template support: Extended template path resolution to recognize local directories, added
copyLocalTemplatemethod with .gitignore respect and symlink protection, and updated command documentation - Developer tooling: Added
mage devinstall,mage devuninstall, andmage preflighttargets for streamlined development workflow - Dependency updates and fixes: Updated OpenTelemetry semconv to v1.39.0 to fix runtime panics, replaced deprecated
runtime.WithCaptureResponsewithpolicy.WithCaptureResponse, improved test robustness with conditional skips, and added Windows file lock handling for extension builds
Reviewed changes
Copilot reviewed 22 out of 23 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| pkg/templates/path.go | Extended path resolution to detect local directories and distinguish them from remote URLs |
| pkg/templates/path_test.go | Comprehensive test coverage for all path resolution scenarios including local directories |
| internal/repository/initializer.go | Added copyLocalTemplate method with .gitignore support and symlink protection |
| internal/repository/initializer_test.go | Tests for local template initialization including .gitignore handling |
| cmd/init.go | Updated template flag description to mention local directory paths |
| magefile.go | New mage targets for dev install, uninstall, and preflight quality checks |
| CONTRIBUTING.md | Documentation for new developer tooling |
| .vscode/cspell-azd-dictionary.txt | Added new terms (devinstall, devuninstall, TOCTOU, etc.) |
| cmd/testdata/* | Auto-generated snapshots updated for new command descriptions |
| internal/tracing/**/*.go | Updated semconv imports from v1.37.0 to v1.39.0 to fix schema mismatch panic |
| pkg/**/test.go | Replaced deprecated runtime.WithCaptureResponse with policy.WithCaptureResponse |
| internal/runcontext/agentdetect/detect_test.go | Added skipIfProcessDetectsAgent helper for reliable "no agent" tests |
| cmd/auto_install*_test.go | Agent detection skips for environment-sensitive tests |
| internal/appdetect/appdetect_test.go | Maven availability check for Java-dependent tests |
| pkg/infra/provisioning/terraform/terraform_provider_test.go | Terraform availability checks for tool-dependent tests |
| pkg/state/state_cache_test.go | Increased TTL timing margins from 100ms to 500ms for reliability |
| extensions/microsoft.azd.extensions/internal/helpers.go | Windows file lock handling with rename-before-copy strategy |
| extensions/microsoft.azd.extensions/internal/cmd/build.go | Process termination to release file locks before binary copy |
| extensions/azure.ai.models/pkg/models/pending_upload.go | Formatting fixes (gofmt) |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
cli/azd/extensions/microsoft.azd.extensions/internal/cmd/build.go
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 22 out of 23 changed files in this pull request and generated 6 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 22 out of 23 changed files in this pull request and generated no new comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
spboyer
left a comment
There was a problem hiding this comment.
Three issues found:
-
.git file from local template can be copied into target (High)
The copy filter skips .git only when it is a directory (info.IsDir()). In Git worktrees/submodules, .git is often a file pointing to an external gitdir. That file will be copied and can break git operations in the target. Exclude .git by name regardless of file type. -
Local-template .gitignore handling is incomplete (Medium)
Only the root .gitignore is parsed; nested .gitignore rules are ignored. This can copy files that are intentionally ignored in subdirectories (secrets, build artifacts). Consider using tracked-files-only semantics or evaluating nested .gitignore rules. -
Existing local directory silently overrides remote template resolution (Medium)
Absolute() now checks os.Lstat first and returns local path if directory exists. Inputs like owner/repo will resolve locally if a same-named directory exists, silently skipping remote template. Consider requiring explicit local syntax (./..., ../..., or absolute path) for local templates.
Add support for using local filesystem directories as template sources with 'azd init --template /path/to/local/template'. This enables template development workflows where uncommitted changes need to be tested. Changes: - templates/path.go: Absolute() now detects local directories via os.Stat() and returns their absolute path instead of erroring - templates/path.go: Add IsLocalPath() helper to distinguish local paths from remote URLs - initializer.go: Add copyLocalTemplate() that uses file copy instead of git clone, preserving uncommitted changes and working with non-git dirs - initializer.go: Initialize() branches on local vs remote templates - init.go: Update --template flag help text to mention local directory support - detect_confirm_apphost.go: Fix pre-existing missing color import Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
When copying a local template, respect .gitignore rules so that files like node_modules/, build artifacts, and .env are excluded from the copy — matching the behavior of git clone for remote templates. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
… and adding error checks for non-existent paths
…pporting git:// protocol in path functions
…and file renaming for locked targets
…e creation and expanding URI support in path functions
…dictionary and improve preflight checks in magefile
The otel module at v1.38.0 does not contain semconv/v1.39.0. Restores RPCJSONRPCRequestIDKey and RPCJSONRPCErrorCodeKey constants that are available in v1.37.0. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
A local go.work file can cause MVS to resolve different module versions than go.mod alone, masking build failures that only appear in CI. This caused semconv/v1.39.0 to resolve locally via otel v1.40.0 from workspace modules, while CI only had v1.38.0 which lacks that package. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
On macOS, /var is a symlink to /private/var. t.TempDir() returns the unresolved path but filepath.Abs (via os.Getwd) returns the real path, causing a mismatch. Use filepath.EvalSymlinks on the expected value. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- magefile.go: Shell-quote directory paths in rc file export lines to prevent injection from shell metacharacters - magefile.go: Pass dir as PowerShell parameter in addToPathWindows instead of interpolating into script string - magefile.go: Cache WSL vs Git-bash detection in toShellPath using sync.Once instead of running a shell command on every call - build.go: Pass installDir as PowerShell parameter in killExtensionProcesses instead of interpolating into script - path.go: Use os.Lstat in Absolute to reject symlinked template directories, consistent with copyLocalTemplate - initializer.go: Preserve executable file permissions when copying local templates by scanning for exec bits after copy - path_test.go: Add test for symlink rejection in Absolute Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…scheme list - Quote all interpolated values in runShellScript with shellQuote() - Remove unused shellKind.tested field - Use os.LookupEnv/os.Unsetenv for GOWORK restore in Preflight - Match full export line in addToPathUnix duplicate check - Skip findExecutableFiles on Windows (exec bits not meaningful) - Extract shared remoteURIPrefixes/isRemoteURI in path.go - Use t.Chdir() instead of os.Chdir() in path test Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
EvalSymlinks both expected and actual to handle: - macOS: /var vs /private/var (symlink resolution) - Windows: CLOUDT~1 vs cloudtest (8.3 short name mapping) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Return explicit 'not a directory' error for file paths in Absolute() - Fix test to pass actual file path and assert error - Return error when .gitignore exists but can't be read (not just missing) - Remove duplicate StopSpinner call (defer already handles it) - Pin golangci-lint@v2.6 and cspell@8.13.1 to match CI versions Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Address PR review feedback from @spboyer: 1. (High) Exclude .git by name regardless of file type, not just when it is a directory. In git worktrees/submodules .git is a file pointing to an external gitdir. 2. (Medium) Support nested .gitignore files by collecting matchers from all .gitignore files in the template tree, not just the root. 3. (Medium) Require explicit local path syntax (./dir, ../dir, or absolute path) for local templates. Bare names like 'my-template' or 'owner/repo' always resolve to GitHub URLs even if a same-named local directory exists. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
355c40b to
cf70c30
Compare
Azure Dev CLI Install InstructionsInstall scriptsMacOS/Linux
bash: pwsh: WindowsPowerShell install MSI install Standalone Binary
MSI
Documentationlearn.microsoft.com documentationtitle: Azure Developer CLI reference
|
Resolved |
This PR adds support for using local filesystem directories as template sources in
azd init, enabling template developers to iterate on templates without pushing to a remote repository. It also adds developer tooling (mage devinstall,mage preflight) and fixes several pre-existing issues required for CI gates to pass.Changes by Category
1. Local Template Support (core feature)
Files:
pkg/templates/path.go,pkg/templates/path_test.go,internal/repository/initializer.go,internal/repository/initializer_test.go,cmd/init.gopkg/templates/path.go— Template path resolutionAbsolute()to recognize local filesystem directories as valid template sources, in addition to existing GitHub shorthand and remote git URLs.azd init --template <path>only accepted remote URLs or GitHubowner/reposhorthand. Local paths were silently misinterpreted as GitHub repository names, leading to confusing errors.os.Lstat()check: if the path resolves to an existing directory, return its absolute path. Symlinks are explicitly rejected for security consistency withcopyLocalTemplate.looksLikeLocalPath()helper: if the path looks like an explicit local reference (./,../, absolute path) but the directory doesn't exist, return a clear "does not exist" error.remoteURIPrefixes/isRemoteURI()helper coveringgit@,git://,ssh://,file://,http://,https://— used by bothAbsolute()andIsLocalPath()to avoid list drift.IsLocalPath()function for downstream code to distinguish local vs remote resolved paths.pkg/templates/path_test.go— Comprehensive test coveraget.Chdir()andfilepath.EvalSymlinksnormalization to handle macOS symlinks and Windows 8.3 short names.internal/repository/initializer.go— Local template copyingcopyLocalTemplate()method,findExecutableFiles()helper, and integrated them into theInitialize()flow.git clone, but local templates should be copied directly from the filesystem to preserve uncommitted changes — the primary use case for local template development.copyLocalTemplate()copies the directory contents while: skipping.git/, skipping symlinks (security), respecting root-level.gitignorerules (with proper error handling for unreadable.gitignore), and validating the source is a real directory (not a symlink, mitigating TOCTOU).findExecutableFiles()scans copied files for executable permission bits sogitInitializecan preserve them withgit update-index --chmod=+x. Skips on Windows where exec bits are not meaningful.Initialize()now branches ontemplates.IsLocalPath()— local paths usecopyLocalTemplate(), remote paths usefetchCode()(existing git clone logic). Spinner message updated accordingly.internal/repository/initializer_test.go— Initializer testsInitialize()and the filtering logic incopyLocalTemplate()need test coverage..gitdirectory exclusion,.gitignorefiltering, and.gitignorenegation patterns.cmd/init.go— Minor wording fix2. Developer Tooling (
mage devinstall,mage preflight)Files:
magefile.go,CONTRIBUTING.md,.vscode/cspell-azd-dictionary.txt,cmd/testdata/TestFigSpec.ts,cmd/testdata/TestUsage-azd-init.snapmagefile.go— Build targetsDevInstall,DevUninstall, andPreflight.mage devinstall— Builds azd from source asazd-dev(avoids conflicting with production install), installs to~/.azd/bin, and adds it to PATH.mage devuninstall— Removesazd-devand cleans up PATH entries.mage preflight— Runs all 6 CI quality checks in sequence: gofmt, copyright headers (via bash/WSL on Windows), golangci-lint, cspell,go build,go test -short. Fails fast with clear install instructions if any required tool is missing. Tool versions pinned to match CI (golangci-lint@v2.6,cspell@8.13.1).GOWORK=offduring preflight to mirror CI environment (prevents localgo.workfiles from masking module resolution differences).shellQuote()(POSIX single-quote escaping). PowerShell commands useparam()pattern for safe argument passing.sync.Oncefor performance.CONTRIBUTING.md— Documentation.vscode/cspell-azd-dictionary.txt— Spell check dictionarydevinstall,devuninstall,jongio,nazd,TOCTOU,Veyorto the custom dictionary.cmd/testdata/TestFigSpec.ts,cmd/testdata/TestUsage-azd-init.snap— Snapshot updatesUPDATE_SNAPSHOTS=true go test ./cmd -run 'TestFigSpec|TestUsage'.devinstallanddevuninstallcommands added new entries to the command tree.3. OpenTelemetry semconv v1.39.0 → v1.37.0 (revert)
Files:
internal/tracing/fields/fields.go,internal/tracing/resource/resource.go,internal/telemetry/storage_exporter_test.go,internal/telemetry/appinsights-exporter/span_to_envelope_test.gov1.39.0back tov1.37.0in 4 files.v1.39.0package only exists in otel modulev1.40.0+, butgo.modpins otel atv1.38.0. Locally this worked because ago.workfile outside the repo pulled in a higher otel version via MVS, but CI (which has nogo.work) failed withcould not import go.opentelemetry.io/otel/semconv/v1.39.0.RPCJSONRPCRequestIDKey,RPCJSONRPCErrorCodeKey) were restored as localattribute.Key()definitions using the same string values, since they were deprecated upstream.4. Deprecated API fixes (SA1019 staticcheck)
Files:
pkg/azapi/azure_client_test.go,pkg/azsdk/correlation_policy_test.go,pkg/azsdk/user_agent_policy_test.go,pkg/graphsdk/entity_list_request_builder_test.goruntime.WithCaptureResponsewithpolicy.WithCaptureResponsein 4 test files. Removed now-unusedruntimeimports.runtime.WithCaptureResponseis deprecated (SA1019). CI's staticcheck catches this. The replacementpolicy.WithCaptureResponseis the recommended API and is functionally identical.5. Test robustness fixes
Files:
internal/runcontext/agentdetect/detect_test.go,cmd/auto_install_integration_test.go,cmd/auto_install_test.go,internal/appdetect/appdetect_test.go,pkg/infra/provisioning/terraform/terraform_provider_test.go,pkg/state/state_cache_test.goAgent detection tests —
detect_test.go,auto_install_*_test.goskipIfProcessDetectsAgent()helper. Tests that assert "no agent detected" are skipped when running inside a CI agent (e.g., Copilot CLI, GitHub Actions).App detection tests —
appdetect_test.gomvnis not installed.mvnto detect Java projects. On machines without Maven, the test fails with a confusing "executable not found" error instead of skipping gracefully.Terraform provider tests —
terraform_provider_test.goskipIfTerraformNotInstalled()helper, applied to 4 test functions.tools.EnsureInstalled()which checksexec.LookPath("terraform")against the real PATH before any mocked commands run.State cache TTL test —
state_cache_test.go6. Formatting fixes (gofmt, lll)
Files:
extensions/azure.ai.models/pkg/models/pending_upload.go,extensions/microsoft.azd.extensions/internal/cmd/build.gopending_upload.gowas reformatted bygofmt -s -w.build.gohad a long line broken to satisfy the 125-charllllinter rule.gofmtcheck. These were pre-existing formatting issues on the branch.Testing
All changes verified locally with
mage preflightwhich runs:gofmt— 0 issuesgolangci-lint(v2.6, matching CI) — 0 issuescspell— 0 issuesgo build— passesgo test -short— all pass