Skip to content

Add importer-provider extension capability for infrastructure generation#7452

Draft
vhvb1989 wants to merge 18 commits intomainfrom
ext-importer-support
Draft

Add importer-provider extension capability for infrastructure generation#7452
vhvb1989 wants to merge 18 commits intomainfrom
ext-importer-support

Conversation

@vhvb1989
Copy link
Copy Markdown
Member

@vhvb1989 vhvb1989 commented Apr 2, 2026

Summary

Adds a new importer-provider capability to the azd extension framework, enabling extensions to generate infrastructure (Bicep/Terraform) from project-specific definition formats.

This POC enables creating extensions for scenarios like #7425, where projects want to define infrastructure in languages like C# or TypeScript instead of writing Bicep directly. The demo extension illustrates this pattern using markdown files as an analogy — where the .md files with resource definitions play the role that .cs or .ts files would in a real implementation. The extension reads these definitions and produces the Bicep that azd provisions.

How it works

# azure.yaml
name: my-app
infra:
  importer:
    name: demo-importer    # extension-provided importer
    # options:              # extension-owned settings (optional)
    #   path: custom-folder
services:
  app:
    host: staticwebapp
    language: js
    project: ./src/app
    dist: dist
  • azd provision: If no infra/ folder exists, the importer generates temporary Bicep at runtime
  • azd infra gen: The importer generates Bicep files into infra/ (ejection)
  • After ejection: azd uses the ejected files directly, skipping the importer (supports customization)

Key design decisions

  • Importers live under infra.importer, not in the services list — services remain clean for deployable units only
  • options is an extension-owned map — each extension defines its own settings (path, format, etc.) with its own defaults
  • Infra override: Once infra/ exists with generated files, azd uses those and skips the importer
  • Backward compatible: Aspire auto-detection via CanImport on services is preserved for existing projects

What is included

Core framework

  • Importer interface with CanImport, Services, ProjectInfrastructure, GenerateAllInfrastructure
  • ImportManager refactored from hardcoded DotNetImporter to pluggable []Importer list
  • ImporterRegistry singleton for extension-registered importers
  • ImporterConfig with extension-owned options map in provisioning.Options

gRPC layer

  • importer.proto with bidirectional Stream RPC
  • Extension SDK: ImporterProvider interface, ImporterManager, ImporterEnvelope
  • Server-side ImporterGrpcService with capability verification
  • ExternalImporter adapter implementing Importer over gRPC

Demo extension

  • DemoImporterProvider reads .md files with azd-infra-gen/v1 front-matter header
  • Parses resource definitions (resource group, static web app with azd-service-name tag)
  • Generates main.bicep, main.parameters.json, resources.bicep
  • Default folder: demo-importer/ (overridable via options.path)

Sample project (test/functional/testdata/samples/extension-importer/)

  • Combines importer-generated infra with a deployable Static Web App service
  • Works end-to-end with azd up to deploy to Azure

Documentation

  • docs/extensions/extension-custom-importers.md — authoring guide

What is NOT included

  • azd init integration: Extension importers are not invoked during azd init. The init command uses built-in appdetect with hardcoded language detectors. Extensions can work around this by adding their own init command (e.g., azd my-importer init), similar to azd ai agent init. A future project-detector capability could integrate extensions into azd init auto-detection, following the same strategy where extensions define what to detect and what to write.

Closes #7425

vhvb1989 and others added 16 commits April 1, 2026 07:49
…e importer list

Introduce the Importer interface that captures the contract for project importers
(CanImport, Services, ProjectInfrastructure, GenerateAllInfrastructure). This enables
extensions to provide custom importers via the extension framework.

Key changes:
- Define Importer interface in pkg/project/importer.go
- Make DotNetImporter implement Importer with Name() returning "Aspire"
- Refactor ImportManager from hardcoded dotNetImporter field to []Importer list
- ImportManager iterates all registered importers (first match wins)
- DotNetImporter.CanImport now accepts *ServiceConfig and checks language
  before expensive dotnet CLI detection
- Move Aspire-specific constraints (single service, ContainerApp target)
  into ImportManager's importer iteration with importer name in messages
- Update IoC registration to build importer list with DotNetImporter
- Update all test files to use new NewImportManager([]Importer{...}) signature

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Implement the complete gRPC infrastructure for importer extensions:

Proto & Code Generation:
- New importer.proto with ImporterService (bidirectional Stream RPC)
- Messages: CanImport, Services, ProjectInfrastructure, GenerateAllInfrastructure
- GeneratedFile type for serializing fs.FS over gRPC

Extension SDK (pkg/azdext/):
- ImporterProvider interface for extension-side implementation
- ImporterManager for client-side gRPC stream management
- ImporterEnvelope for message envelope operations
- ExtensionHost.WithImporter() registration method
- Full lifecycle integration in ExtensionHost.Run()

gRPC Server (internal/grpcserver/):
- ImporterGrpcService implementing server-side stream handling
- Capability verification and broker-based message dispatch
- IoC registration of ExternalImporter on provider registration

Core Integration:
- ExternalImporter adapter implementing project.Importer over gRPC
- Handles ServiceConfig/ProjectConfig proto conversion
- Temp directory management for ProjectInfrastructure
- memfs reconstruction for GenerateAllInfrastructure

Extension Framework:
- ImporterProviderCapability and ImporterProviderType constants
- Added to listenCapabilities for extension startup
- Updated extension.schema.json with new capability and provider type

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
IoC Registration:
- ImportManager now accepts ServiceLocator for future extensibility
- Added lazy ImportManager registration for gRPC service access
- ImporterGrpcService uses AddImporter() to register external importers
  at runtime instead of IoC named registration
- Updated all test files with new NewImportManager(importers, locator) signature

Demo Extension:
- DemoImporterProvider detects projects via demo.manifest.json marker file
- Generates minimal Bicep infrastructure (resource group)
- Registered in listen.go with host.WithImporter("demo-importer", factory)
- Added importer-provider capability and provider entry to extension.yaml

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Test Infrastructure:
- New extension-importer sample project with azure.yaml and demo.manifest.json
- Integration test (Test_CLI_Extension_Importer) that builds/installs demo extension,
  copies sample project, and verifies extension starts with importer capability

Registry:
- Updated registry.json with importer-provider capability and demo-importer
  provider entry for microsoft.azd.demo extension

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The ExternalImporter was sending RelativePath (e.g. './src') to the
extension, but the extension process has a different working directory
and couldn't find project files. Now sends the fully resolved absolute
path via svc.Path() so extensions can access files regardless of CWD.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The ImporterGrpcService (singleton) was adding importers to a scoped
ImportManager instance via lazy resolution, but the azd command action
would get a different scoped ImportManager that didn't have the
extension importers.

Fix: introduce ImporterRegistry as a singleton shared between the gRPC
service (which adds importers on extension registration) and all
ImportManager instances (which query the registry via allImporters()).
This ensures extension-registered importers are visible to any command
that uses the ImportManager.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Logs the number of available importers, their names/types, and the
result of each CanImport() call. Run with --debug to see this output,
which helps diagnose why extension importers may not be invoked.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The infra generate command was missing the extensions middleware,
so extension-provided importers were never started. Added
UseMiddleware for both hooks and extensions, matching the pattern
used by infra create and infra delete.

Also added debug logging to GenerateAllInfrastructure to trace
importer availability and CanImport results.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Replace demo.manifest.json approach with a markdown-based resource
definition format using azd-infra-gen/v1 front-matter header.

Sample project (extension-importer):
- infra-gen/resources.md defines a resource group and storage account
  with tags using a readable markdown format with YAML-like properties
- azure.yaml points to ./infra-gen as the service project path

Demo importer (DemoImporterProvider):
- CanImport: scans directory for .md files with azd-infra-gen/v1 header
- Parses resource definitions from markdown (H1 = resource, - key: value)
- Generates main.bicep (resource group + module reference)
- Generates resources.bicep (storage account with sku, kind, tags)
- Proper Bicep string interpolation for env var references
- Both ProjectInfrastructure and GenerateAllInfrastructure produce
  the full file set

Note: azd init integration for extension-based project detection is
not yet implemented. Currently extensions participate only at provision
and infra gen time via the ImportManager. Init-time detection would
require extending the appdetect framework to query importers.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Architectural redesign: importers are no longer defined as services.
Instead, azure.yaml has a new infra.importer field with name and path:

  infra:
    importer:
      name: demo-importer    # matches extension-registered importer
      path: ./infra-gen       # path to importer project files

This keeps the services list clean for only deployable services that
can be built and packaged. The importer is a separate concern that
generates infrastructure, not a deployable unit.

Key changes:
- Added ImporterConfig struct to provisioning.Options (name + path)
- ImportManager.ProjectInfrastructure checks infra.importer before
  auto-detection (backward compat with Aspire preserved)
- ImportManager.GenerateAllInfrastructure likewise checks infra.importer
- Importer interface: ProjectInfrastructure and GenerateAllInfrastructure
  now take importerPath string instead of *ServiceConfig
- DotNetImporter wraps the new interface with path->ServiceConfig adapters
- ExternalImporter sends path via ServiceConfig.RelativePath over gRPC
- Updated sample to use infra.importer with infra-gen/resources.md

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The importer config now uses an open-ended options map instead of a
fixed path field, giving extensions full control over their settings:

  infra:
    importer:
      name: demo-importer
      options:           # extension-owned, schema defined by extension
        path: custom-dir # optional override, extension defines default

Key changes:
- ImporterConfig.Path replaced with Options map[string]any
- Added GetOption(key, default) helper for string option lookup
- Importer interface methods now receive (projectPath, ImporterConfig)
  so extensions can resolve paths and settings themselves
- Proto updated: infrastructure requests send projectPath + options map
- ImporterProvider interface uses (projectPath, options map[string]string)

Demo extension updates:
- Default path is now 'demo-importer' (convention-based, no config needed)
- Reads 'path' from options to allow override
- Sample project renamed infra-gen/ to demo-importer/
- azure.yaml simplified: just name, no path needed

This illustrates the pattern: extensions own their defaults and options.
Users only need to specify the importer name. Options are optional overrides.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Demonstrates combining an extension importer with a deployable service:

Sample project structure:
  azure.yaml          - defines infra.importer + 'app' service
  demo-importer/      - resource definitions (.md files)
  src/app/            - static web app (vanilla JS/HTML)
    dist/             - pre-built static files for deployment
    package.json      - no-op build script

azure.yaml:
  infra:
    importer:
      name: demo-importer     # generates all infrastructure
  services:
    app:
      host: staticwebapp       # deployable service
      language: js
      project: ./src/app
      dist: dist

Generated infrastructure (resources.bicep) includes:
- Storage account with tags
- Static Web App with 'azd-service-name: app' tag linking it to the
  service in azure.yaml, enabling azd to deploy to the correct resource

This shows the clean separation: the importer owns infrastructure
generation, services own build/deploy. They connect via azd-service-name tags.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
ProjectInfrastructure was using SendAndWait which doesn't handle
progress messages. When the extension sent a progress update before
the response, SendAndWait received the progress message and returned
nil for GetProjectInfrastructureResponse(), causing the 'missing
response' error.

Fix: use SendAndWaitWithProgress which correctly filters progress
messages and waits for the actual response.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The demo importer now generates main.parameters.json with the standard
azd parameter mappings (environmentName, location, principalId) for
both ProjectInfrastructure (runtime) and GenerateAllInfrastructure
(azd infra gen).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Keep only the resource group and static web app in the demo sample
to focus on the essential pattern.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
New doc: cli/azd/docs/extensions/extension-custom-importers.md

Covers the importer-provider capability, how it works, the demo
importer as an analogy for real-world use cases like #7425, writing
your own importer, combining with services, infra override/ejection,
and current limitations (azd init integration).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
vhvb1989 and others added 2 commits April 2, 2026 05:04
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Feature: CDK Language Support — Author Azure Infrastructure in Any Programming Language

1 participant