Skip to content

Remove deprecated S3 encryption clients, replaced credentials providers and unused code - #4002

Open
averyk22 wants to merge 3 commits into
release/1.12from
remove-deprecated-apis
Open

averyk22 wants to merge 3 commits into
release/1.12from
remove-deprecated-apis

Conversation

@averyk22

@averyk22 averyk22 commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

Issue #, if available:

Description of changes:
S3 encryption: Remove S3EncryptionClient, S3EncryptionClientV2 and CryptoConfigurationV2, and make S3EncryptionClientBase's constructors protected so the base can't be built directly with legacy encryption settings. The V1/V2-write, V3-read integration tests now write legacy objects through a test-only subclass of the base.

Credentials providers: Remove ProcessCredentialsProvider and the original ProfileConfigFileAWSCredentialsProvider; ProfileConfigFileAWSCredentialsProvider stays as a deprecated alias of ProfileCredentialsProvider. Before removing them, fix two gaps in ProfileCredentialsProvider that the old providers covered (3d22bb5): it dropped the account ID, and its default constructor ignored AWS_DEFAULT_PROFILE. AWSCredentialsProvider.h also drops the GeneralHTTPCredentialsProvider.h include kept for 1.11.

Unused code: Remove Aws::Utils::GetTheLights, AWSMultiMap.h and CommonCryptorSPI.h. Aws::MultiMap stays in AWSMap.h because URI.h uses it.

Not run locally: the s3-encryption integration tests, which need AWS resources.

Check all that applies:

  • Did a review by yourself.
  • Added proper tests to cover this PR. (If tests are not applicable, explain.)
  • Checked if this PR is a breaking (APIs have been changed) change.
  • Checked if this PR will not introduce cross-platform inconsistent behavior.
  • Checked if this PR would require a ReadMe/Wiki update.

Check which platforms you have built SDK on to verify the correctness of this PR.

  • Linux
  • Windows
  • Android
  • MacOS
  • IOS
  • Other Platforms

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@averyk22 averyk22 changed the title Remove the deprecated S3 encryption clients and unused core code Remove deprecated S3 encryption clients, replaced credentials providers and unused code Nov 16, 2026
credentials.SetExpiration(DateTime{expiration});
const auto sessionTokenCursor = crtCredentials.GetSessionToken();
credentials.SetSessionToken({reinterpret_cast<char*>(sessionTokenCursor.ptr), sessionTokenCursor.len});
const auto accountIdCursor = crtCredentials.GetAccountId();

@averyk22 averyk22 Nov 16, 2026 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Users can set aws_account_id in their profile, or have their credential_process command return
AccountId, so DynamoDB uses its account-based endpoints. ProfileCredentialsProvider, which the
default chain uses and which replaces the providers removed in this PR, read the account ID
through the CRT but never copied it into AWSCredentials, so the SDK silently ignored it: in the
default preferred mode DynamoDB fell back to the regional endpoint, and required mode failed.
The removed providers still returned it, so this fix keeps the account ID working once they're
gone.

* CRT-based credentials provider that sources credentials from config files with full SEP compliance.
* Supports assume role, credential_source, role chaining, and all SEP scenarios.
*
* The config and credentials files are read once, when the provider is created. Credentials that expire, such as

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Refresh: documented, not changed: refreshRateMs is ignored and Reload() is empty. The CRT reads the files once, and static keys never expire, so rotated keys need a new provider. Expiring credentials still refresh. Re-reading the files would be a behavior change, so this PR only documents it. Also note: DEFAULT_REFRESH_RATE_MS in the .cpp is really the CRT fetch timeout, not a refresh rate.

RunTestWithCredentialsProvider(std::move(credsProvider), "v");
auto credsProvider = Aws::MakeShared<Aws::Auth::ProfileCredentialsProvider>(TEST_LOG_TAG);
ASSERT_STREQ("test-process-key", credsProvider->GetAWSCredentials().GetAWSAccessKeyId().c_str());
// TODO: ProfileCredentialsProvider reports CREDENTIALS_PROFILE ("n") even when credentials come from credential_process; restore "v" once it reports the source.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Known gap: ProfileCredentialsProvider reports CREDENTIALS_PROFILE even for credential_process. Follow-up to restore v.

Remove S3EncryptionClient, S3EncryptionClientV2 and CryptoConfigurationV2,
and make S3EncryptionClientBase's constructors protected so the base can't
be built directly with legacy encryption settings. The V1/V2-write,
V3-read integration tests now write legacy objects through a test-only
subclass of the base.

Remove Aws::Utils::GetTheLights, AWSMultiMap.h and CommonCryptorSPI.h,
which nothing used. Aws::MultiMap stays in AWSMap.h because URI.h uses it.

The credential provider removals are held back: ProfileCredentialsProvider
does not yet return the account ID from the credentials file or
credential_process, so it cannot replace them.
@averyk22
averyk22 force-pushed the remove-deprecated-apis branch 2 times, most recently from 4fdc324 to 58a08b0 Compare November 20, 2026 18:35
* Initializes with refreshRateMs as the frequency at which the file is reparsed in milliseconds. Defaults to 5 minutes.
* Initializes with the profile from AWS_DEFAULT_PROFILE, then AWS_PROFILE, then "default".
*/
ProfileCredentialsProvider(long refreshRateMs = REFRESH_THRESHOLD);

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

refreshRateMs is currently ignored by ProfileCredentialsProvider. This was the case before this PR, which keeps the parameter on purpose.

Since #3721 (Feb 27, 2026, first released in 1.11.760), ProfileCredentialsProvider has wrapped the CRT profile provider. All three constructors mark refreshRateMs as unused and Reload() is empty, so the config and credentials files are read once when the provider is created.

Before #3721, the provider stored refreshRateMs (default REFRESH_THRESHOLD, 5 minutes) and re-read the files once that interval passed. Because ProfileCredentialsProvider is in the default credentials chain, a long-running app on 1.11.760+ whose static keys get rotated in ~/.aws/credentials keeps the old keys until it creates a new client or provider. Credentials that expire, such as from assume-role or credential_process, still refresh.

This PR keeps the refreshRateMs parameter so existing callers keep compiling. The code comments don't mention it, so they don't commit to either behavior while the open question is decided: restore the periodic re-read (for example, by recreating the CRT provider on the interval) or remove the parameter. Because the change already shipped in 1.11 releases, it should probably be tracked separately for main as well.

@averyk22
averyk22 force-pushed the remove-deprecated-apis branch from 58a08b0 to 441e101 Compare November 20, 2026 18:36
AWSCredentials credentials{};
const auto accountIdCursor = crtCredentials.GetAccessKeyId();
credentials.SetAWSAccessKeyId({reinterpret_cast<char*>(accountIdCursor.ptr), accountIdCursor.len});
const auto accessKeyIdCursor = crtCredentials.GetAccessKeyId();

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The old providers covered the profile-file cases but once deleted we need the CRT provider to cover it as well

CrtCredentialsProvider::ExtractCredentialsFromCrt never copied the CRT's
account ID, so every CRT-backed provider, including ProfileCredentialsProvider
in the default chain, dropped aws_account_id from the profile and AccountId
from credential_process output. DynamoDB then fell back from account-based
endpoints to the regional endpoint, and accountIdEndpointMode=required failed.

Document that ProfileCredentialsProvider reads the files once, and that its
default constructor picks the profile from AWS_PROFILE only.
Remove ProcessCredentialsProvider and the original
ProfileConfigFileAWSCredentialsProvider. The default chain stopped using them
once the CRT-backed ProfileCredentialsProvider took over their work, and with
the account ID and AWS_DEFAULT_PROFILE fixes it now covers what they did,
apart from re-reading the files on refreshRateMs.
ProfileConfigFileAWSCredentialsProvider stays as a deprecated alias of
ProfileCredentialsProvider. GetCredentialsFromProcess stays, since
STSProfileCredentialsProvider uses it.

Drop the GeneralHTTPCredentialsProvider.h include that AWSCredentialsProvider.h
kept for 1.11 compatibility.

Move the tests to ProfileCredentialsProvider: remove leading whitespace that
the shared config format reads as a continuation line, drop the reload check
and the ProcessCredentialsProvider tests, and point the credential_process
tests at an empty credentials file so a local [default] can't take over.
@averyk22
averyk22 force-pushed the remove-deprecated-apis branch from 441e101 to fb9e5cc Compare November 20, 2026 19:11

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants