Skip to content

Move endpoint resolution from interceptors to pipeline stage#6820

Open
S-Saranya1 wants to merge 6 commits intofeature/master/core-interceptors-migrationfrom
somepal/endpoint-resolution-refactoring
Open

Move endpoint resolution from interceptors to pipeline stage#6820
S-Saranya1 wants to merge 6 commits intofeature/master/core-interceptors-migrationfrom
somepal/endpoint-resolution-refactoring

Conversation

@S-Saranya1
Copy link
Copy Markdown

@S-Saranya1 S-Saranya1 commented Mar 26, 2026

Motivation and Context

The SDK uses ExecutionInterceptor for core endpoint resolution functionality. But this interface was
designed for customer extensibility, not internal SDK. This is Part 2 of the core interceptors migration, following the auth scheme resolution refactoring in #6755.

Modifications

1. Codegen — generate resolveEndpoint() in client classes

  • SyncClientClass, AsyncClientClass: Added resolveEndpoint() method generation containing service-specific endpoint resolution logic (param building, rules engine call, host prefix, auth scheme signer properties, business metrics).
  • JsonProtocolSpec, QueryProtocolSpec, XmlProtocolSpec: Pass callback via ClientExecutionParams.withEndpointResolver().
  • BaseClientBuilderClass: Removed endpoint interceptors from built-in interceptors list.

2. Codegen — generate per-service EndpointResolverUtils

  • EndpointResolverUtilsSpec: New codegen spec generating a per-service utility class (e.g., DynamoDbEndpointResolverUtils) with static helpers extracted from the old interceptor - ruleParams(), context params, host prefix, auth scheme property copying, metrics. Per-service because it references service-specific types.
  • EndpointRulesSpecUtils: Added helper methods for class name resolution.
  • EndpointProviderTasks: Generate EndpointResolverUtils instead of old interceptors.

3. Added EndpointResolutionStage

New pipeline stage in sdk-core that runs after AuthSchemeResolutionStage. Calls the service-specific callback, stores RESOLVED_ENDPOINT, copies endpoint headers, applies resolved URL, and reports ENDPOINT_RESOLVE_DURATION metric. Registered in both sync and async pipelines.

4. Added EndpointResolver interface and ClientExecutionParams

EndpointResolver functional interface in sdk-core, endpointResolver field on ClientExecutionParams, ENDPOINT_RESOLVER execution attribute, and wiring in AwsExecutionContextBuilder.

5. De-duplicated AwsEndpointProviderUtils

Created single shared class in aws-core replacing identical per-service generated copies. Deleted the AwsEndpointProviderUtils.java.resource template.

6. Updated S3 presigner and S3Utilities

Removed old endpoint interceptors from both. Added direct endpoint resolution methods that call S3EndpointResolverUtils.ruleParams() and apply the resolved URL.

7. Deleted dead code

Removed EndpointResolverInterceptorSpec, RequestEndpointInterceptorSpec, AuthSchemeInterceptorSpec, their tests, AwsEndpointProviderUtils.java.resource template, and dead generateAuthSchemeInterceptor() method.

Testing

  • Added EndpointResolutionStageTest - unit tests for the new pipeline stage
  • Verified existing functional tests for endpoint resolution behavior passing.
  • Modified codegen-fixture tests.

Screenshots (if appropriate)

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)

Checklist

  • I have read the CONTRIBUTING document
  • Local run of mvn install succeeds
  • My code follows the code style of this project
  • My change requires a change to the Javadoc documentation
  • I have updated the Javadoc documentation accordingly
  • I have added tests to cover my changes
  • All new and existing tests passed
  • I have added a changelog entry. Adding a new entry must be accomplished by running the scripts/new-change script and following the instructions. Commit the new file created by the script in .changes/next-release with your changes.
  • My change is to implement 1.11 parity feature and I have updated LaunchChangelog

License

  • I confirm that this pull request can be released under the Apache 2 license

@S-Saranya1 S-Saranya1 requested a review from a team as a code owner March 26, 2026 15:28
@S-Saranya1 S-Saranya1 force-pushed the somepal/endpoint-resolution-refactoring branch from 87d0960 to 305ba3a Compare March 27, 2026 16:38
@S-Saranya1 S-Saranya1 force-pushed the somepal/endpoint-resolution-refactoring branch from 6000d9f to 7b09efe Compare March 27, 2026 17:15
}

// Apply resolved endpoint URL
ClientEndpointProvider clientEndpointProvider =
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see that in the design doc, there's a backward compatibility issue and is solved by adding a check for customer modified url here. Why is that missed here?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added the logic.

return executionAttributes.getAttribute(AwsExecutionAttribute.FIPS_ENDPOINT_ENABLED);
}

/**
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just curious why do we deleted the comments here and below?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, I think the comments provide some value, it's not super obvious what's happening

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added the comments back.

"TestValue0"))))
.hasMessageContaining("stop");

assertThat(interceptor.context.httpRequest().matchingHeaders("TestHeader")).containsExactly("TestValue", "TestValue0");
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a subtle behavioral change and we can call it out as a known behavior change in the feature branch merging PR description. if downstream system is doing exact string matching on header value ordering (which is wrong) this could break them.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will call this out in feature branch merging PR

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto here, we need codegen tests

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are several codegen fixture tests to update, will create a separate PR for those to keep this one focused. For now, updated a few test files and added to this PR so the codegen changes are easy to review, also added the new codegen fixture test EndpointResolverUtilsSpecTest.

* permissions and limitations under the License.
*/

package software.amazon.awssdk.awscore.internal.endpoints;
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this being used across modules? If so, it shouldn't be internal

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah right, but the callers are also @SdkInternalApi right, so kept it that way, but WDYT should we leave it here or move it to a non-internal package.

@@ -38,10 +52,6 @@ public static Boolean fipsEnabledBuiltIn(ExecutionAttributes executionAttributes
return executionAttributes.getAttribute(AwsExecutionAttribute.FIPS_ENDPOINT_ENABLED);
}

/**
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, I think the comments provide some value, it's not super obvious what's happening

}

EndpointResolver resolver = attrs.getAttribute(SdkInternalExecutionAttribute.ENDPOINT_RESOLVER);
if (resolver == null) {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a valid case where there's no resolver?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm no valid case, removed that check.

Comment on lines +505 to +506
ClassName awsEndpointAttribute = ClassName.get("software.amazon.awssdk.awscore.endpoints", "AwsEndpointAttribute");
ClassName endpointAuthScheme = ClassName.get("software.amazon.awssdk.awscore.endpoints.authscheme", "EndpointAuthScheme");
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this necessary? codegen has a dependency on aws-core so you should be able to reference the class directly

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated!

/**
* Generates a per-service endpoint resolver utility class (e.g. {@code DynamoDbEndpointResolverUtils})
* containing all static helper methods for endpoint resolution: building endpoint params, host prefix
* resolution, auth scheme property copying, and business metrics.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to code generate this at all? What service specific logic does it have vs what is generic? If its generic, I'd lean towards moving it out of our codegeneration and into a core package. That makes codegen simpler, reduces duplication between services (reducing artifact size) and makes it easier to unit test.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm yeah, this needs to be code-generated. Most of the logic is building rule params, extracting context params, and applying host prefixes which is service-specific. These methods reference per-service types like S3EndpointParams, GetObjectRequest, etc. that live in each service module, so a shared class in core can't reference them.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah right, I got myself confused by EndpointResolverUtil and AwsEndpointProviderUtils face palm.

How difficult would it be to add the service prefix to this name, as we do for things like S3EndpointParams and S3EndpointProvider?

* no way to know, just from the HTTP request object, where the override path ends (if it's even there) and where
* the request path starts. Additionally, the rule itself may also append other parts to the endpoint override path.
* <p>
* To solve this issue, we pass in the endpoint set on the path, which allows us to the strip the path from the
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Assume this was pre-existing, but I think wording is incorrect:

we pass in the endpoint set on the path

I think this should be "we pass in the endpoint set on the client" (ie, path -> client)

/**
* Generates a per-service endpoint resolver utility class (e.g. {@code DynamoDbEndpointResolverUtils})
* containing all static helper methods for endpoint resolution: building endpoint params, host prefix
* resolution, auth scheme property copying, and business metrics.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah right, I got myself confused by EndpointResolverUtil and AwsEndpointProviderUtils face palm.

How difficult would it be to add the service prefix to this name, as we do for things like S3EndpointParams and S3EndpointProvider?

* Detects if a customer interceptor modified the HTTP request URL in modifyHttpRequest().
* Compares the current request's host and scheme against the snapshot taken before interceptors ran.
*/
private static boolean customerModifiedUrl(SdkHttpFullRequest.Builder request, ExecutionAttributes attrs) {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

naming nit, maybe interceptorModifiedEndpoint instead of customer?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah right, I got myself confused by EndpointResolverUtil and AwsEndpointProviderUtils face palm.

How difficult would it be to add the service prefix to this name, as we do for things like S3EndpointParams and S3EndpointProvider?

Yeah while generating it generates as DynamoDbEndpointResolverUtils,S3EndpointResolverUtils

}

/**
* Detects if a customer interceptor modified the HTTP request URL in modifyHttpRequest().
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just to be clear - so, we don't care if an interceptor modified the path right? That will be accounted for in the merge. Its only if they have changed the schema/host/port, ect that we would care right?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct, we only check host, scheme. For path, we're keeping it as is since setUri() handles the merging. We only skip endpoint application if the host, scheme changed.

if (preModifyUri == null) {
return false;
}
String requestHost = request.host();
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to also check port? (probalby an edge case, but not sure)? Any other components that might matter?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah good call, added port check. Don't think there are other components that matter.

ClientEndpointProvider clientEndpointProvider =
attrs.getAttribute(SdkInternalExecutionAttribute.CLIENT_ENDPOINT_PROVIDER);
if (customerModifiedUrl(request, attrs)) {
return request;
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can't remember exactly what our logic was previously here.... but is there a case (like S3 Path style) where we still need to apply parts of the resolved endpoint (but keep the interceptor modified host/schema ect)? I'm not 100% sure about this, just trying to think through some possible edge cases here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants