feat: Add CloudFormation Language Extensions support (Fn::ForEach) #8637

bnusunny · 2026-02-09T00:16:17Z

Description

This PR adds support for CloudFormation Language Extensions in SAM CLI, addressing GitHub issue #5647.

Features

Fn::ForEach - Iterate over collections to generate resources
Fn::Length - Get the length of an array
Fn::ToJsonString - Convert objects to JSON strings
Fn::FindInMap with DefaultValue - Map lookups with fallback values
Conditional DeletionPolicy/UpdateReplacePolicy - Use intrinsic functions like Fn::If in resource policies

Key Design Decisions

In-Memory Expansion Only - Templates are expanded in memory for SAM CLI operations, but the original unexpanded template is preserved for CloudFormation deployment
Dynamic Artifact Properties via Mappings - Fn::ForEach blocks with dynamic artifact properties (e.g., CodeUri: ./src/${Name}) are supported via a Mappings transformation
Locally Resolvable Collections Only - Fn::ForEach collections must be resolvable locally; cloud-dependent values (Fn::GetAtt, Fn::ImportValue) are not supported with clear error messages

Supported Commands

sam build - Builds all expanded functions, preserves original template
sam package - Preserves Fn::ForEach structure with S3 URIs
sam deploy - Uploads original template for CloudFormation to process
sam validate - Validates language extension syntax
sam local invoke - Invokes expanded functions by name
sam local start-api - Serves ForEach-generated API endpoints
sam local start-lambda - Serves all expanded functions

Example

Transform:
  - AWS::LanguageExtensions
  - AWS::Serverless-2016-10-31

Resources:
  Fn::ForEach::Functions:
    - Name
    - [Alpha, Beta, Gamma]
    - ${Name}Function:
        Type: AWS::Serverless::Function
        Properties:
          Handler: ${Name}.handler
          CodeUri: ./src
          Runtime: python3.9

Resolves #5647

Testing

Comprehensive unit tests for the language extensions engine
Property-based tests using Hypothesis
Integration tests for all supported commands
Test templates covering static/dynamic CodeUri, nested stacks, parameter collections

Checklist

Unit tests added
Integration tests added
Documentation in code comments
Error messages include actionable workarounds

bnusunny · 2026-02-09T01:07:11Z

Integration test: https://github.com/aws/aws-sam-cli/actions/runs/21808626015

Add core library for processing CFN Language Extensions including: - Fn::ForEach processor for iterating over collections - Fn::Length, Fn::ToJsonString intrinsic function resolvers - Fn::FindInMap with DefaultValue support - Conditional DeletionPolicy/UpdateReplacePolicy support - SAM CLI integration module with partial resolution mode Includes comprehensive unit tests and property-based tests. Addresses GitHub issue #5647

- Add in-memory template expansion with original template preservation - Add Stack.original_template_dict for CloudFormation deployment - Add PACKAGEABLE_RESOURCE_ARTIFACT_PROPERTIES and DynamicArtifactProperty - Add _process_language_extensions() to SamTranslatorWrapper - Add ForEach guards to validator and warning system - Support locally resolvable collections with clear error messages Templates are expanded in memory for local operations while preserving the original Fn::ForEach structure for CloudFormation deployment.

- Expand Fn::ForEach in memory to build all generated functions - Preserve original template structure in .aws-sam/build/template.yaml - Generate Mappings for dynamic artifact properties (CodeUri: ./${Name}) - Replace dynamic CodeUri with Fn::FindInMap referencing generated Mappings - Support --resource-id with expanded function names

- Preserve Fn::ForEach structure in packaged template - Generate Mappings with S3 URIs for dynamic artifact properties - Use content-based S3 hashing for each expanded artifact - Emit warning for parameter-based collections with dynamic CodeUri - Validate collection values for CloudFormation Mapping key compatibility

- Upload original unexpanded template to CloudFormation - Add clear error message for missing Mapping keys at deploy time - Suggest re-running sam package when collection values change

- Test sam build with static and dynamic CodeUri - Test sam package preserves Fn::ForEach with Mappings transformation - Test sam validate with valid/invalid language extension templates - Test sam local invoke with expanded function names - Test sam local start-api with ForEach-generated API endpoints - Test nested stacks with language extensions

The integration tests incorrectly expected "${Name}" and "${FunctionName}" as the second argument to Fn::FindInMap, but the implementation correctly uses {"Ref": "Name"} and {"Ref": "FunctionName"} because bare ${Var} strings are not resolved by Fn::ForEach inside Fn::FindInMap arguments.

- Add _validate_foreach_nesting_depth() and _calculate_max_foreach_depth() methods to ForEachProcessor to validate nesting depth before expansion - Define MAX_FOREACH_NESTING_DEPTH = 5 constant per CloudFormation limits - Integrate validation into process_template() to fail early with clear error - Add unit tests for depth calculation and validation (14 new tests) - Add integration tests for sam validate and sam build with nested loops - Create test templates for valid (5 levels) and invalid (6 levels) nesting - Fix mypy type annotation for empty dict in test file Validates Requirements: 18.1-18.7

- Add integration test template for sam package with dynamic ImageUri - Add integration test template for sam build with dynamic ImageUri - Add TestPackageLanguageExtensionsImageUri class for package tests - Add TestBuildCommand_LanguageExtensions_DynamicImageUri class for build tests - Add test_deploy_with_language_extensions_dynamic_imageuri for deploy tests - Fix mypy type error in _find_artifact_uri_for_resource (line 798) - Update requirements.md with acceptance criteria 20.8, 20.9, 20.10 - Update tasks.md with Task 28 for ImageUri integration tests Validates Requirements: 6.6, 12.1, 19.2, 20.5, 20.6, 20.7, 20.8, 20.9, 20.10

…tensions - Introduce expand_language_extensions() as canonical Phase 1 entry point in sam_integration.py returning LanguageExtensionResult dataclass - Add template-level cache keyed on (path, mtime, params_hash) to eliminate redundant expansions across components - Remove duplicated expansion logic from SamTranslatorWrapper, PackageContext, SamTemplateValidator, and SamLocalStackProvider - SamTranslatorWrapper.run_plugins() is now Phase 2 only (SAM transform) - Move helper functions (check_using_language_extension, contains_loop_variable, detect_dynamic_artifact_properties, resolve_collection, etc.) to sam_integration module with backward-compatible delegates on wrapper - Add clear_expansion_cache() called by RefreshableSamFunctionProvider for warm container file change invalidation - Add comprehensive unit and property tests for phase separation and caching Implements requirements 20-21, tests for requirements 23-24.

Docker repository names must be lowercase. Changed ForEach collection values from ["Alpha", "Beta"] to ["alpha", "beta"] in dynamic ImageUri templates and updated all corresponding test assertions.

…llers - SamLocalStackProvider passes language_extension_result to SamBaseProvider.get_template() - SamTemplateValidator accepts template_path and passes it to expand_language_extensions() for caching - validate.py and resource_mapping_producer.py pass template_path to SamTemplateValidator - SamBaseProvider.get_template/get_resolved_template_dict accept and forward language_extension_result - Remove unused delegation methods from SamTranslatorWrapper

…per_language_extensions

…es with intrinsic functions When nested templates pass down parameters via intrinsic functions (e.g. {"Ref": "SomeParam"}), parameter values can be dicts which are not hashable. Use json.dumps with sort_keys for a stable, hashable representation instead of hash(tuple(sorted(...))).

…ch compatibility Reuse the latest-x86_64 image already pulled in setUpClass instead of pulling the latest manifest list. Finch pushes manifest lists as-is to ECR, which Lambda rejects since it only supports single image manifests.

Switch from the Python/numpy handler to PreBuiltPython (returns 'Hello World' with no dependencies) to avoid numpy source-tree import errors in CI. The tests validate ForEach expansion and invoke, not dependency building.

…eserve deep copy When expand_language_extensions() returns had_language_extensions=False, the expanded_template is the same object reference as the input template. Passing this result to SamLocalStackProvider caused SamTranslatorWrapper to skip its deep copy, leading to in-place mutation of the template dict. This corrupted nested stack parameter resolution, causing layer ordering issues in test_download_two_layers for nested template variants. Only pass lang_ext_result when language extensions were actually used, so SamTranslatorWrapper falls back to its normal deep-copy behavior.

…ckaging When packaging templates with Fn::ForEach, the language extensions processor partially resolves Fn::Sub (e.g., resolving pseudo-parameters like AWS::Region but leaving resource references as literal placeholders). Copying the Outputs/Conditions from the exported template overwrites the original Fn::Sub intrinsics with plain strings, causing CloudFormation to treat references to ForEach-generated resources as unresolved literals. Also switch API integration test specs from aws_proxy (which requires Fn::Sub for Lambda ARNs) to mock integrations, since DefinitionUri-hosted OpenAPI specs cannot contain CloudFormation intrinsic functions.

_update_sam_mappings_relative_paths was treating Docker image references (e.g., emulation-python3.9-alpha:latest) as local file paths and prepending relative path prefixes (../../) when moving the template to the build output directory. Added an is_file() guard for ImageUri properties, consistent with the existing guard in _update_foreach_relative_paths. Docker image refs that don't resolve to actual local files are now left untouched.

Make detection, build output, and package output recursive so that dynamic artifact properties (e.g., CodeUri: ./services/${Service}) inside nested Fn::ForEach blocks are correctly handled with Mappings transformation. Changes: - Add outer_loops field to DynamicArtifactProperty for tracking enclosing ForEach loops - Make detect_foreach_dynamic_properties() recursive with outer_loops parameter to detect dynamic properties at any nesting depth - Make _update_foreach_artifact_paths() (build) recursive with outer_context parameter; extract helper methods to satisfy ruff PLR0912 branch limit - Make _update_foreach_with_s3_uris() (package) recursive for nested ForEach blocks - Update _generate_artifact_mappings() to support compound keys when property references both outer and inner loop variables - Update _replace_dynamic_artifact_with_findmap() to traverse through outer_loops chain to locate nested ForEach body - Add unit tests, property tests, and integration tests - Update spec (Requirement 25, Tasks 38-43)

bnusunny requested a review from a team as a code owner February 9, 2026 00:16

github-actions bot added area/package sam package command area/deploy sam deploy command area/build sam build command pr/internal labels Feb 9, 2026

bnusunny force-pushed the feat-language-extension branch from 0be94d0 to 5d6cbf3 Compare February 9, 2026 00:33

bnusunny force-pushed the feat-language-extension branch 6 times, most recently from 735ea10 to e68efa0 Compare February 13, 2026 02:47

bnusunny added 17 commits February 13, 2026 02:55

feat: Add Language Extensions support to sam deploy

54f9ceb

- Upload original unexpanded template to CloudFormation - Add clear error message for missing Mapping keys at deploy time - Suggest re-running sam package when collection values change

style: Fix formatting with make format

f114ee7

fix: Use Union syntax for Python 3.8+ compatibility

e870e0b

fix: use lowercase collection values for Docker-compatible image tags

bdfb87d

Docker repository names must be lowercase. Changed ForEach collection values from ["Alpha", "Beta"] to ["alpha", "beta"] in dynamic ImageUri templates and updated all corresponding test assertions.

test: add coverage tests for sam_integration.py (70% → 99%)

a308f21

style: apply make format to validate, test_sam_integration, test_wrap…

dca8046

…per_language_extensions

fix: add type annotations for language_extension_result parameters

89b1735

bnusunny added 9 commits February 13, 2026 02:55

Remove blank line in sam_base_provider.py imports

41bf9a9

chore: fix the formating

49a00b8

bnusunny force-pushed the feat-language-extension branch from e68efa0 to 4ed8396 Compare February 13, 2026 02:55

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Add CloudFormation Language Extensions support (Fn::ForEach) #8637

feat: Add CloudFormation Language Extensions support (Fn::ForEach) #8637

Uh oh!

bnusunny commented Feb 9, 2026

Uh oh!

bnusunny commented Feb 9, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

feat: Add CloudFormation Language Extensions support (Fn::ForEach) #8637

Are you sure you want to change the base?

feat: Add CloudFormation Language Extensions support (Fn::ForEach) #8637

Uh oh!

Conversation

bnusunny commented Feb 9, 2026

Description

Features

Key Design Decisions

Supported Commands

Example

Testing

Checklist

Uh oh!

bnusunny commented Feb 9, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant