Repository navigation
Add modeled pagination design - #970
jonathan343 wants to merge 2 commits into
Conversation
stobrien89
left a comment
There was a problem hiding this comment.
Looks good. Similar to PHP's design with <operation>Paginated. Could the doc say what codegen does with the @paginated trait itself?
- Does it merge service-level defaults with the operation's trait?
- Does it fail generation when
inputToken,outputToken,pageSizeoritemsdon't resolve, oritemsisn't a list or map? We load JSON ASTs, so Smithy's validator may not have run. - DynamoDB's tokens are maps (
ExclusiveStartKey). How are "empty" and "repeated" defined for map tokens?
|
|
||
| ## Proposal | ||
|
|
||
| ### Response pages |
There was a problem hiding this comment.
How will maxItems be handled?
There was a problem hiding this comment.
Modeled page-size inputs stay as-is and apply per-request. Total-item limits are something boto offers. I should think about if we should add that here Gonna leave this our for now
| ### Iterator lifetime | ||
|
|
||
| This proposal uses single-use iterators. Pages and `.items()` consume the same | ||
| traversal; a second loop does not restart it. Customers create a new paginator |
There was a problem hiding this comment.
what happens when a second loop gets nothing?
There was a problem hiding this comment.
Once exhausted, it yields nothing and makes no requests. You'd create a new paginator for another pass.
|
|
||
| ### Iterator lifetime | ||
|
|
||
| This proposal uses single-use iterators. Pages and `.items()` consume the same |
There was a problem hiding this comment.
Yes, generators can be used in the implementation. We will need a wrapper for the .items() feature.
| This proposal does not add total-item limits, combined results, or encoded SDK | ||
| resume tokens. AWS-specific termination flags and other pagination rules not | ||
| represented by the Smithy trait require separate customization. | ||
|
|
There was a problem hiding this comment.
you'll want to define the termination condition. There are two customers may want.
- the "next token" field is empty.
- the "next token" field returned the same value as the previous page.
Without the 2nd option, it's difficult to terminate a tail operation that assumes you are trying to live-tail.
I recommend stop_on_same_token: bool:
client.paginate_live_tail(log_group="orders", stop_on_same_token: True)if it can be merged into the kwargs.
There was a problem hiding this comment.
I added stop_on_same_token, defaulting to True, to a new paginator configuration section in the doc. It ends normally after yielding the page. Setting it to False allows continued polling.
Summary
Adds a design proposal for typed
paginate_<operation>methods that iterate over complete response pages, with an optional.items()helper when the model identifies a collection.The proposal covers synchronous and asynchronous usage, lazy requests, iterator lifetime, and continuation behavior. It also explains the choice of page-first, single-use iteration and the alternatives considered.
This PR contains only the design document, so the interface can be reviewed independently of the implementation.
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.