Skip to content

[Feature]: Deduplicate SerDes deserialization within an invocation #678

Description

@zhongkechen

What would you like?

Deduplicate repeated SerDes deserialization within a single Lambda invocation.

Context-dependent SerDes implementations may perform filesystem or network I/O. Repeated access to the same checkpointed result or exception currently invokes the configured SerDes again, even when the serialized input and durable payload identity are unchanged.

Possible Implementation

  • Add an invocation-scoped deserialization cache around SDK-managed SerDes calls.
  • Key entries by SerDes instance identity, durable execution ARN, operation/entity identity, payload kind when available, and a digest of the serialized input.
  • Cache successful values, including None, but never cache failures.
  • Bound completed entries with an LRU or weak-reference policy so large deserialized values are not retained indefinitely.
  • If SDK-managed child/concurrency threads can request the same value concurrently, share one in-flight result rather than issuing duplicate external reads.
  • Clear all entries at the end of the Lambda invocation; do not use a process-global cache across replays.
  • Add tests for cache hits, distinct contexts/data, None, failures, eviction, and concurrent callers.

Is this a breaking change?

No.

Does this require an RFC?

No.

Additional Context

This is primarily valuable for filesystem and other external-storage SerDes implementations. The Java SDK work in aws/aws-durable-execution-sdk-java#648 uses an invocation-scoped bounded cache and in-flight deduplication.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestparityProvides parity with other language implementations of the SDKpkg:sdkPackage: aws-durable-execution-sdk-python

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions