[1/6] Migrate AWS variable sources to SDK v3

[GrahamCampbell]

Migrate all AWS-backed variable sources to SDK v3 in one PR.

Target files:

• lib/configuration/variables/sources/instance-dependent/get-ssm.js
• lib/configuration/variables/sources/instance-dependent/get-cf.js
• lib/configuration/variables/sources/instance-dependent/get-s3.js
• lib/configuration/variables/sources/instance-dependent/get-aws.js
• New variable-source cached command sender, likely lib/configuration/variables/sources/instance-dependent/create-cached-aws-variable-source-command-sender.js
• New S3 body helper, likely lib/aws/s3-body-to-string.js
• Related tests under test/unit/lib/configuration/variables/sources/instance-dependent/
• New cached command sender tests under test/unit/lib/configuration/variables/sources/instance-dependent/
• New S3 body helper tests under test/unit/lib/aws/s3-body-to-string.test.js

Add dependencies:

• @aws-sdk/client-ssm: ^3.975.0
• @aws-sdk/client-sts: ^3.975.0
• @aws-sdk/client-cloudformation: ^3.975.0

Keep existing dependency:

• @aws-sdk/client-s3: ^3.975.0

PR 1 Helper Design

Add a variable-source-local cached SDK v3 command sender. Suggested API:

const createCachedAwsVariableSourceCommandSender = require("./create-cached-aws-variable-source-command-sender");

const sender = createCachedAwsVariableSourceCommandSender({
  getProvider: () => serverlessInstance.getProvider("aws"),
  Client: SSMClient,
});

await sender.send(GetParameterCommand, input, {
  region,
});

Required behavior:

• Do not call getProvider() until a command is sent.
• Include variable-source in the helper filename, exported function name, and test filename.
• Resolve effectiveRegion as region === undefined ? provider.getRegion() : region.
• Cache clients per sender instance by effective region.
• Build clients with await getProvider().getAwsSdkV3Config({ region: effectiveRegion }).
• Pass the returned config object directly into new Client(config).
• Send new Command(input) through client.send(command).
• Always cache transformed command-result promises because this helper is only for variable sources that already used cached requests.
• Cache key must include command name, effective region, and deep-sorted command input.
• Support transformResult({ result, commandName, input, region, effectiveRegion }), defaulting to identity, so callers can convert non-reusable SDK results before caching.
• For S3 GetObjectCommand, cache the converted string value, not a raw response with a consumable Body stream.
• Delete failed cached promises so transient failures are retried later.
• Apply a per-sender concurrency limit of 2 around the SDK send and result transformation.
• Keep helper private; do not export it as a plugin-facing API.

Non-goals for this helper:

• Do not accept service/method strings.
• Do not maintain command maps.
• Do not normalize every SDK v3 error globally in this first PR.
• Do not try to replace provider.request().
• Do not import this helper from deploy/remove/logs/info/runtime internals.

PR 1 SSM Variable Migration

Use SSMClient and GetParameterCommand.

Preserve:

• ${ssm:/path} syntax.
• ${ssm(region):/path} syntax.
• raw option.
• noDecrypt option.
• String result handling.
• StringList splitting unless raw is set.
• SecureString JSON parsing behavior.
• Missing parameter returning null.
• Cache behavior previously provided by { useCache: true }.

V3 missing-parameter detection:

• Treat error.name === 'ParameterNotFound' as missing.
• Treat error.Code === 'ParameterNotFound' as missing if present.
• Temporarily keep the old normalized code check if it simplifies fixture compatibility.

Credential-provider error behavior:

• Expired session or invalid token errors must surface as variable resolution errors.
• They must not be treated as missing parameters.
• They must not fall back to ambient credentials.

PR 1 CloudFormation Variable Migration

Use CloudFormationClient and DescribeStacksCommand.

Preserve:

• ${cf:stack.output} syntax.
• ${cf(region):stack.output} syntax.
• Missing output returning null.
• Missing stack returning null.
• Cache behavior previously provided by { useCache: true }.

V3 missing-stack detection:

• Treat error.name === 'ValidationError' plus a message containing does not exist as missing.
• Keep compatibility with existing normalized error-code fixtures if cheap.

PR 1 S3 Variable Migration

Use S3Client and GetObjectCommand.

Preserve:

• ${s3:bucket/key} syntax.
• Missing key returning null.
• Missing bucket surfacing as a variable resolution error.
• Cache behavior previously provided by { useCache: true }.
• String conversion of object body.
• Repeated references to the same S3 object must return the converted string each time, including when the SDK response body is a one-shot stream.
• Do not cache a raw GetObjectCommand output with a consumable Body as the final reusable value.
• Do not apply aws-s3-accelerate to S3 variable GetObject reads.

Add a reusable S3 body helper for GetObjectCommand output bodies. It should live outside the variable source so it can be reused later by rollback and artifact-read migrations.

Recommended helper file:

• lib/aws/s3-body-to-string.js

Recommended helper behavior:

• Default encoding is utf8.
• Return '' for null or undefined bodies after the caller has already determined that the object exists.
• Return strings unchanged.
• Prefer body.transformToString(encoding) when present, because this is the AWS SDK v3 SdkStreamMixin path documented for S3 GetObjectCommand output.
• Convert Buffer, Uint8Array, and ArrayBuffer with Buffer.
• Convert Blob via arrayBuffer() when Blob is available.
• Convert Web ReadableStream via getReader() when available.
• Convert Node readable streams with async iteration.
• Throw a ServerlessError with a stable code, such as UNSUPPORTED_S3_GET_OBJECT_BODY, for unsupported body shapes.
• Do not fall back to String(body) for unsupported objects, because real SDK streams would become "[object Object]".
• Do not attempt gzip, deflate, or brotli decompression. ContentEncoding is response metadata and should be handled later by a response-level helper if needed.

Supported body inputs:

• string
• Buffer
• Uint8Array
• ArrayBuffer
• Node readable stream
• Web ReadableStream
• Blob
• SDK body with transformToString()

V3 missing-key detection:

• Treat error.name === 'NoSuchKey' as missing.
• Treat error.Code === 'NoSuchKey' as missing if present.
• Temporarily keep old normalized code checks if useful.

PR 1 AWS Variable Migration

Use STSClient and GetCallerIdentityCommand for ${aws:accountId}.

Preserve:

• ${aws:region} behavior without AWS calls.
• ${aws:accountId} value extraction from Account.
• Cache behavior previously provided by { useCache: true }.

PR 1 Test Plan

Helper tests:

• Builds clients with provider.getAwsSdkV3Config({ region: effectiveRegion }).
• Passes config into client constructor unchanged.
• Sends command instances with expected input.
• Does not resolve the AWS provider until a command is sent.
• Caches identical transformed command-result promises by default.
• Cache key includes effective region.
• Cache key includes command input.
• Cache key is stable for semantically identical command input objects with different property order.
• Rejected cached promise is evicted.
• Rejected transformResult promise is evicted.
• transformResult receives { result, commandName, input, region, effectiveRegion }.
• Omitted region and explicit provider region share one cache entry.
• The helper limits SDK send and result transformation concurrency to 2 per sender.

S3 body helper tests:

• Converts string.
• Converts Buffer.
• Converts Uint8Array.
• Converts ArrayBuffer.
• Converts Node readable stream.
• Converts Web ReadableStream.
• Converts Blob when available.
• Uses transformToString() when present.
• Propagates errors from transformToString().
• Propagates Node stream read errors.
• Throws UNSUPPORTED_S3_GET_OBJECT_BODY for unsupported objects without stringifying contents.
• Does not attempt decompression based on body bytes alone.
• Handles empty/undefined body in the same way the S3 variable source expects.

SSM tests:

• Existing string parameter resolves.
• Region-scoped parameter forwards region to SDK v3 config.
• StringList resolves to array by default.
• StringList with raw resolves to raw comma-delimited string.
• SecureString JSON resolves to object by default.
• SecureString non-JSON resolves to string.
• SecureString with raw resolves to raw string.
• noDecrypt sets WithDecryption: false.
• Missing parameter resolves to null.
• Credential-provider errors surface as variable resolution errors.
• Repeated same parameter is cached.
• Same parameter with different WithDecryption is not treated as same cache entry.
• Same parameter with different region is not treated as same cache entry.

CloudFormation tests:

• Existing output resolves.
• Region-scoped output forwards region to SDK v3 config.
• Missing output resolves to null.
• Missing stack resolves to null.
• Non-stack ValidationError surfaces as a variable resolution error.
• Repeated same stack lookup is cached.

S3 tests:

• Existing object resolves to string.
• Existing object with stream body resolves to string.
• Missing key resolves to null.
• Missing bucket surfaces as variable resolution error.
• Invalid address and non-string address behavior remains unchanged.
• Repeated same object lookup is cached.
• Repeated same object lookup with a one-shot stream body returns the converted string on every reference.
• Cached S3 variable resolution does not attempt to consume the same Body stream twice.
• S3 variable reads do not implicitly enable S3 acceleration.

AWS source tests:

• ${aws:accountId} uses STSClient and GetCallerIdentityCommand.
• Account is returned unchanged.
• ${aws:region} remains independent of SDK clients.
• STS errors surface as variable resolution errors.
• Repeated account lookup is cached.

Documentation tests are not needed, but docs must be reviewed for overclaiming.

PR 1 Documentation

Do not update user-facing variable docs in PR 1.

Do not claim framework-wide AWS SSO/IAM Identity Center support.

If release notes are written separately, they must say only that internal AWS-backed variable sources moved to SDK v3 credential resolution, and they must explicitly avoid saying the framework supports SSO.

Deploy, remove, logs, metrics, packaging, and runtime AWS operations continue to migrate separately.

PR 1 Validation Commands

Run targeted helper and variable tests:

npx mocha --config "test/mocha/unit.cjs" \
  "test/unit/lib/configuration/variables/sources/instance-dependent/create-cached-aws-variable-source-command-sender.test.js" \
  "test/unit/lib/aws/s3-body-to-string.test.js" \
  "test/unit/lib/configuration/variables/sources/instance-dependent/get-*.test.js"

Run AWS/provider safety slice:

npx mocha --config "test/mocha/unit.cjs" \
  "test/unit/lib/aws/*.test.js" \
  "test/unit/lib/plugins/aws/provider.test.js"

Run formatting and lint:

npm run prettify && npm run lint

Optional full suite before merge:

npm test -- -b

Manual test config:

custom:
  accountId: ${aws:accountId}
  cfValue: ${cf:some-stack.SomeOutput}
  s3Value: ${s3:some-bucket/some-key}
  ssmValue: ${ssm:/some/parameter}

PR 1 definition of done:

• All four AWS-backed variable sources use SDK v3.
• Existing variable behavior is unchanged.
• Variable source request caching is preserved.
• Credential provider functions are passed to SDK v3 clients unchanged.
• Missing-resource behavior is compatible.
• Credential-provider errors surface and are not mistaken for missing resources.
• S3 variable caching stores the converted body value and does not reuse consumed streams.
• S3 variable reads do not enable transfer acceleration.
• Docs and release notes avoid framework-wide SSO/IAM Identity Center support claims.

[Copilot]

Pull request overview

This PR migrates AWS-backed variable sources used during configuration/variable resolution from the legacy AWS SDK v2 provider.request(...) flow to direct AWS SDK v3 clients, while introducing a shared cached-command sender to preserve caching and limit concurrency.

Changes:

• Switch aws, cf, s3, and ssm variable sources to use AWS SDK v3 clients/commands via a shared cached sender utility.
• Add a generic, concurrency-limited cached command sender for AWS variable sources.
• Add an s3BodyToString helper (with tests) to safely convert various S3 GetObject body shapes into strings.

Reviewed changes

Copilot reviewed 13 out of 13 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
lib/configuration/variables/sources/instance-dependent/get-ssm.js	Use @aws-sdk/client-ssm + cached sender; update not-found detection logic for v3-style errors.
lib/configuration/variables/sources/instance-dependent/get-s3.js	Use @aws-sdk/client-s3 + cached sender; convert GetObject bodies via s3BodyToString.
lib/configuration/variables/sources/instance-dependent/get-cf.js	Use @aws-sdk/client-cloudformation + cached sender; treat missing-stack as null while surfacing other ValidationErrors.
lib/configuration/variables/sources/instance-dependent/get-aws.js	Use @aws-sdk/client-sts + cached sender for ${aws:accountId}.
lib/configuration/variables/sources/instance-dependent/create-cached-aws-variable-source-command-sender.js	New shared cached sender (per-region client cache, per-command+input cache, concurrency limit).
lib/aws/s3-body-to-string.js	New helper to convert SDK v3 S3 bodies (string/buffer/streams/blob/etc.) to string safely.
test/unit/lib/configuration/variables/sources/instance-dependent/get-ssm.test.js	Rewrite tests to stub AWS SDK v3 SSM client/command; add caching/credential-provider assertions.
test/unit/lib/configuration/variables/sources/instance-dependent/get-s3.test.js	Rewrite tests to stub AWS SDK v3 S3 client/command; add body-shape + caching assertions.
test/unit/lib/configuration/variables/sources/instance-dependent/get-cf.test.js	Rewrite tests to stub AWS SDK v3 CF client/command; add caching + ValidationError behavior assertions.
test/unit/lib/configuration/variables/sources/instance-dependent/get-aws.test.js	Rewrite tests to stub AWS SDK v3 STS client/command; add caching/credential-provider assertions.
test/unit/lib/configuration/variables/sources/instance-dependent/create-cached-aws-variable-source-command-sender.test.js	New unit tests covering caching, eviction on failure, and concurrency limiting behavior.
test/unit/lib/aws/s3-body-to-string.test.js	New unit tests covering supported/unsupported S3 body shapes and error propagation.
package.json	Add AWS SDK v3 clients for CloudFormation, SSM, and STS.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.