feat: Add CloudFormation Language Extensions support (Fn::ForEach)

[bnusunny]

Description

This PR adds support for CloudFormation Language Extensions in SAM CLI, addressing GitHub issue #5647.

Features

• Fn::ForEach - Iterate over collections to generate resources
• Fn::Length - Get the length of an array
• Fn::ToJsonString - Convert objects to JSON strings
• Fn::FindInMap with DefaultValue - Map lookups with fallback values
• Conditional DeletionPolicy/UpdateReplacePolicy - Use intrinsic functions like Fn::If in resource policies

Key Design Decisions

1. In-Memory Expansion Only - Templates are expanded in memory for SAM CLI operations, but the original unexpanded template is preserved for CloudFormation deployment
2. Dynamic Artifact Properties via Mappings - Fn::ForEach blocks with dynamic artifact properties (e.g., CodeUri: ./src/${Name}) are supported via a Mappings transformation
3. Locally Resolvable Collections Only - Fn::ForEach collections must be resolvable locally; cloud-dependent values (Fn::GetAtt, Fn::ImportValue) are not supported with clear error messages

Supported Commands

• sam build - Builds all expanded functions, preserves original template
• sam package - Preserves Fn::ForEach structure with S3 URIs
• sam deploy - Uploads original template for CloudFormation to process
• sam validate - Validates language extension syntax
• sam local invoke - Invokes expanded functions by name
• sam local start-api - Serves ForEach-generated API endpoints
• sam local start-lambda - Serves all expanded functions

Example

Transform:
  - AWS::LanguageExtensions
  - AWS::Serverless-2016-10-31

Resources:
  Fn::ForEach::Functions:
    - Name
    - [Alpha, Beta, Gamma]
    - ${Name}Function:
        Type: AWS::Serverless::Function
        Properties:
          Handler: ${Name}.handler
          CodeUri: ./src
          Runtime: python3.9

Resolves #5647

Testing

• Comprehensive unit tests for the language extensions engine
• Integration tests for all supported commands
• Test templates covering static/dynamic CodeUri, nested stacks, parameter collections

Checklist

• Unit tests added
• Integration tests added
• Documentation in code comments
• Error messages include actionable workarounds

[bnusunny]

Integration test: https://github.com/aws/aws-sam-cli/actions/runs/21808626015

[reedham-aws]

This is my review of the first commit that adds all the language extension support in its own library. I didn't review the tests, but I really don't think that's feasible to review due to the sheer size.

[reedham-aws]

I'm going to leave one comment here but as a general issue I don't think we should be having task identifiers in the code like this. They're not particularly useful, especially because those docs aren't even committed.

[reedham-aws]

I guess I'm fine having these doctest strings, but we don't actually use them for anything so they're a little clunky. We don't do this anywhere else in the code.

[reedham-aws]

Style nit: I think having this as @staticmethod conflicts with the name starting with _.

[reedham-aws]

It seems to me like _process_section and _process_properties are exactly the same. The section_name parameter isn't even used here, so I feel like we could merge the methods and start using it to differentiate.

[reedham-aws]

We use the same hardcoded string several times in this method, probably better to have it in one place and use a variable.

[reedham-aws]

Do we not show anything for something in the list greater than index 1?

[reedham-aws]

try/except not necessary if you're going to raise the exception with no processing.

[reedham-aws]

I don't think visited needs to be cleared here, would we not end up rechecking the same nodes?

[reedham-aws]

Don't need all this documentation to wrap json.dumps

[reedham-aws]

Probably same for yaml down below too.

[reedham-aws]

This method is duplicate to one in processors/foreach.py

[reedham-aws]

My review of the second commit; once again, I did not review any tests.

[reedham-aws]

Most of the code in this method is a direct copy of _update_relative_paths. I get that we need to do some preprocessing, but let's simplify so we're not copying code like this.

[reedham-aws]

This block is copied from get_template_artifacts_format. Lets break it into a helper.

[reedham-aws]

I'm sure it does, but without looking further when does this happen? I want to make sure we're not forgetting this.

[reedham-aws]

I think this is a good method to use, but it seems like it was kind of unevenly applied. For example, in samcli/lib/package/artifact_exporter.py, it's not really used at all when it could be. There might be other places where it can be used that I haven't seen yet.

I also think the name is not descriptive enough to show that it's skipping foreach blocks.

[reedham-aws]

nit: I was confused by loop_variable here. In other places in the library (eg. the _expand_foreach method in foreach.py) this is referred to as the identifier. I think they both make sense, and I don't think it's worth changing, just wanted to call that out.

[reedham-aws]

Should be at top level

[reedham-aws]

We have this as a global in samcli/commands/_utils/template.py. I think it makes more sense to be defined as a global in this file.

[reedham-aws]

I won't comment on more, but here is a time we could use iter_resources.

[reedham-aws]

Instead of changing this method to wrap something from the library, lets just use that method instead.

[reedham-aws]

Should be top level

[reedham-aws]

I don't know what the exact amount of integration tests that would be good for this feature would be, but I feel like this is too many. I think our testing strategy should be more about testing the library itself; if that works, then builds should work too. A basic case wouldn't hurt, but I think that we risk really bloating our test suites here. Very much open to the opinions of others.

[reedham-aws]

I've also noticed there's not a lot of logging in this PR. I think that would be good to have in some places, but it would be difficult to add all at once.