Skip to content

Commit b5a44da

Browse files
Jonas Saabelclaude
andcommitted
feat: add comprehensive pagination helpers with Flow support
Implement Flow-based pagination utilities for all paginated API endpoints: - Add generic PaginatedResponse interface for type-safe pagination - Implement three pagination methods: asFlow(), collectAll(), asPagesFlow() - Add Flow helpers to all 6 APIs (DataSources, Blocks, Comments, Users, Search, Pages) - Create comprehensive pagination documentation guide - Add unit tests (9 tests) and integration tests (5 tests) Benefits: - Memory-efficient processing of large result sets - Reactive processing patterns with Kotlin Flow - Reduces boilerplate for manual pagination loops - Type-safe cursor handling Breaking changes: None - additive only 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
1 parent c58736e commit b5a44da

File tree

22 files changed

+2779
-29
lines changed

22 files changed

+2779
-29
lines changed

docs/README.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,7 @@
1616
- **[Comments](comments.md)** - Add and retrieve comments
1717

1818
### Features
19+
- **[Pagination](pagination.md)** - Handle paginated results efficiently
1920
- **[Rich Text DSL](rich-text-dsl.md)** - Format text with mentions, links, and more
2021
- **[File Uploads](file-uploads.md)** - Upload and manage files and images
2122
- **[Error Handling](error-handling.md)** - Handle API errors gracefully

docs/blocks.md

Lines changed: 37 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -368,14 +368,50 @@ notion.blocks.appendChildren(pageId) {
368368

369369
### Pagination Handling
370370

371-
`retrieveChildren()` automatically handles pagination:
371+
Retrieving block children supports multiple pagination approaches:
372+
373+
#### 1. Automatic Collection (Simple)
374+
375+
`retrieveChildren()` automatically fetches all children:
372376

373377
```kotlin
374378
// Retrieves ALL children, handling pagination internally
375379
val allBlocks = notion.blocks.retrieveChildren(pageId)
376380
println("Total blocks: ${allBlocks.size}")
377381
```
378382

383+
**Use when**: You need all children and the count is reasonable (< 1000 blocks).
384+
385+
#### 2. Flow-Based Streaming
386+
387+
For pages with many blocks, use Flow for efficient processing:
388+
389+
```kotlin
390+
// Memory-efficient - processes blocks as they're fetched
391+
notion.blocks.retrieveChildrenAsFlow(pageId).collect { block ->
392+
println("Processing ${block.type}: ${block.id}")
393+
// Process each block individually
394+
}
395+
```
396+
397+
**Use when**: Working with pages that have many blocks (1000+) or memory efficiency matters.
398+
399+
#### 3. Page-Level Flow
400+
401+
Access pagination metadata while processing:
402+
403+
```kotlin
404+
// Get complete responses with pagination info
405+
notion.blocks.retrieveChildrenPagedFlow(pageId).collect { response ->
406+
println("Fetched ${response.results.size} blocks (has more: ${response.hasMore})")
407+
response.results.forEach { block ->
408+
// Process blocks in this batch
409+
}
410+
}
411+
```
412+
413+
See **[Pagination](pagination.md)** for comprehensive guide and best practices.
414+
379415
### Error Handling
380416

381417
Handle common block API errors:

docs/data-sources.md

Lines changed: 44 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -194,7 +194,11 @@ val existingDb = notion.databases.retrieve("database-id")
194194
val dataSourceId = existingDb.dataSources?.firstOrNull()?.id
195195
```
196196

197-
### Automatic Pagination
197+
### Working with Pagination
198+
199+
Data source queries can return large result sets. The library provides three ways to handle pagination:
200+
201+
#### 1. Automatic Collection (Simple)
198202

199203
The `query()` method automatically fetches all matching pages:
200204

@@ -209,7 +213,45 @@ val allPages = notion.dataSources.query("data-source-id") {
209213
println("Total matching pages: ${allPages.size}")
210214
```
211215

212-
The library handles pagination transparently, fetching up to 100,000 records (1,000 pages × 100 records/page safety limit).
216+
**Use when**: Result sets are reasonably sized (< 1000 items) and you need all results.
217+
218+
#### 2. Flow-Based Streaming (Recommended for Large Sets)
219+
220+
For large result sets, use Flow to process items as they arrive:
221+
222+
```kotlin
223+
// Memory-efficient - processes pages as they're fetched
224+
notion.dataSources.queryAsFlow("data-source-id") {
225+
filter {
226+
select("Status").equals("To Do")
227+
}
228+
}.collect { page ->
229+
println("Processing: ${page.id}")
230+
// Process each page individually
231+
}
232+
```
233+
234+
**Use when**: Working with 1000+ items or when memory efficiency matters.
235+
236+
#### 3. Page-Level Flow (Batch Processing)
237+
238+
Access pagination metadata while processing:
239+
240+
```kotlin
241+
// Get complete responses with pagination info
242+
notion.dataSources.queryPagedFlow("data-source-id") {
243+
filter { /* ... */ }
244+
}.collect { response ->
245+
println("Fetched ${response.results.size} pages (has more: ${response.hasMore})")
246+
response.results.forEach { page ->
247+
// Process pages in this batch
248+
}
249+
}
250+
```
251+
252+
**Use when**: You need pagination metadata or want to process pages in batches.
253+
254+
See **[Pagination](pagination.md)** for comprehensive guide and best practices.
213255

214256
### Complex Filters
215257

0 commit comments

Comments
 (0)