docs: spec — heartbeat-based claim leasing for lib-data-stream-redis#73
Open
pditommaso wants to merge 2 commits into
Open
docs: spec — heartbeat-based claim leasing for lib-data-stream-redis#73pditommaso wants to merge 2 commits into
pditommaso wants to merge 2 commits into
Conversation
Design for decoupling claim-timeout's two overloaded roles (dead-consumer failover vs. re-poll cadence) by leasing a message to its consumer while accept() runs, plus a bounded max-processing-time safety valve. Motivated by seqeralabs/sched#600. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Paolo Di Tommaso <paolo.ditommaso@gmail.com>
Add §4.2 with four scenario timelines (the bug today, heartbeat keeping a blocking handler owned, a fast RUNNING poll with cadence preserved, and pod-death failover) plus a summary table. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Paolo Di Tommaso <paolo.ditommaso@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Design spec (no code) for
lib-data-stream-redisaddressing a class of duplicate / churning message processing caused byclaim-timeoutbeing silently overloaded with two conflicting roles.In
RedisMessageStream,XAUTOCLAIM's only liveness signal is idle time, so it cannot distinguish a dead consumer from a slow-but-alive one.claim-timeouttherefore doubles as both:XACK, so their stream entry only gets revisited once idle crossesclaim-timeout).These conflict: a handler whose
accept()runs longer thanclaim-timeoutgets its message reclaimed and re-executed while the original is still running — duplicate, possibly cross-pod.Approach
Lease a message to its consumer only while
accept()is on the stack: a background daemon re-claims it to itself (XCLAIM … 0 <id> JUSTID, resets idle without transferring data) everyclaim-timeout/3. The instantaccept()returns (trueorfalse), the lease stops:false(RUNNING, waiting) → not heartbeated → existingclaim-timeoutre-poll cadence unchanged;accept()→ heartbeat dies with the JVM → real failover preserved.Plus a bounded max-processing-time safety valve (default 15m): if a single
accept()runs longer, the heartbeat stops refreshing that entry so it becomes reclaimable again — bounding a hung-but-alive handler (does not interrupt the handler thread).Scope
RedisMessageStream+ three additivedefaultmethods onRedisStreamConfig→ zero required downstream change; consumers just bump to1.6.0.lib-cmd-queue-redisexecute/checkStatusthreading model — this is purely message-ownership leasing in the stream layer.LocalMessageStreamuntouched (no PEL, no claim concept).The spec includes thread-safety analysis, a failure-mode table, a Testcontainers test plan, rejected alternatives (raise-the-knob, per-consumer liveness registry, ack-on-receipt + re-enqueue), and a risk register.
Spec file:
docs/superpowers/specs/2026-06-30-lib-data-stream-redis-heartbeat-claim.mdMotivated by seqeralabs/sched#600.
🤖 Generated with Claude Code