feat(outbox-store): Use executeBatch() after processing entries instead of single executeUpdate() for each entry#71
Open
rawo wants to merge 5 commits into
Open
feat(outbox-store): Use executeBatch() after processing entries instead of single executeUpdate() for each entry#71rawo wants to merge 5 commits into
rawo wants to merge 5 commits into
Conversation
…ad of single executeUpdate() for each entry
Contributor
There was a problem hiding this comment.
Pull request overview
This PR implements KOJAK-75 by adding a batch update API to OutboxStore and switching the processing pipeline to persist delivery results using a single JDBC executeBatch() call per drained outbox batch, reducing DB roundtrips from N to 1 for the “update-after-processing” phase.
Changes:
- Added
OutboxStore.updateAfterProcessingBatch(entries)with a default per-entry fallback implementation. - Implemented optimized JDBC batch updates in
PostgresOutboxStoreandMysqlOutboxStore, updating only the mutable delivery-state columns. - Updated
OutboxProcessor.processNext()to call the new batch update method once per processed batch; added unit + integration tests plus a JMH benchmark and benchmark artifacts.
Reviewed changes
Copilot reviewed 10 out of 10 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| okapi-postgres/src/main/kotlin/com/softwaremill/okapi/postgres/PostgresOutboxStore.kt | Adds Postgres batch UPDATE via addBatch()/executeBatch() and extracts SQL strings to companion object. |
| okapi-mysql/src/main/kotlin/com/softwaremill/okapi/mysql/MysqlOutboxStore.kt | Adds MySQL batch UPDATE via addBatch()/executeBatch() and extracts SQL strings to companion object. |
| okapi-integration-tests/src/test/kotlin/com/softwaremill/okapi/test/store/OutboxStoreContractTests.kt | Adds contract test verifying mixed-status batch persistence works end-to-end. |
| okapi-core/src/test/kotlin/com/softwaremill/okapi/core/OutboxStoreTest.kt | Adds unit tests for default updateAfterProcessingBatch fallback behavior. |
| okapi-core/src/test/kotlin/com/softwaremill/okapi/core/OutboxProcessorTest.kt | Adds unit test asserting the processor calls the batch method exactly once and never per-entry. |
| okapi-core/src/main/kotlin/com/softwaremill/okapi/core/OutboxStore.kt | Introduces the new default batch update method on the store interface. |
| okapi-core/src/main/kotlin/com/softwaremill/okapi/core/OutboxProcessor.kt | Switches persistence step from per-entry updates to a single batch update call. |
| okapi-benchmarks/src/jmh/kotlin/com/softwaremill/okapi/benchmarks/OutboxStoreUpdateBatchBenchmark.kt | Adds JMH benchmark isolating the update phase and comparing individual vs batch updates. |
| benchmarks/results-postopt-KOJAK-75.md | Documents benchmark methodology and results for the batch update optimization. |
| benchmarks/outbox-store-update-batch.json | Stores raw JMH output for the reported benchmark run. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements KOJAK-75: replaces N individual
updateAfterProcessing()calls per outbox batch with a single batched JDBC write, cutting per-batch DB roundtrips from N to 1.OutboxStore.updateAfterProcessingBatch(entries)— new method with a default implementation that loopsupdateAfterProcessing()one at a time, so existing implementations (test stubs, future stores) don't break.PostgresOutboxStore/MysqlOutboxStoreoverride it with a singlePreparedStatementreused across all entries viaaddBatch()/executeBatch(), touching only the 5 mutable columns (status,updated_at,retries,last_attempt,last_error) rather than the full 11-column upsertpersist()uses. Postgres binds the id withsetObject(uuid); MySQL binds it withsetString(uuid.toString())(CHAR(36)column, no native UUID type).OutboxProcessor.processNext()now callsstore.updateAfterProcessingBatch(processed)once per drained batch instead of loopingupdateAfterProcessing()per entry.Benchmark
New
OutboxStoreUpdateBatchBenchmark(JMH) isolates the update phase for 1000 entries:individualUpdatesbatchUpdateFull writeup and raw JSON in
benchmarks/results-postopt-KOJAK-75.md.Test plan
OutboxStoreTest(defaultupdateAfterProcessingBatchloop behavior, incl. empty list),OutboxProcessorTest(assertsprocessNext()calls the batch method exactly once, never per-entry)OutboxStoreContractTests— mixed final statuses (DELIVERED + PENDING retry + FAILED) persisted via oneupdateAfterProcessingBatch()call, run against both Postgres and MySQL./gradlew ktlintCheckclean./gradlew buildgreen (core, postgres, mysql, micrometer, integration-tests, spring-boot)Refs: KOJAK-75 (blocked-by KOJAK-73, already merged — deliverBatch fire-flush-await)