feat(gas): add batch listen

[MasterPtato]

No description provided.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Preview	Comments	Updated (UTC)
rivetkit-serverless	Ready	Preview	Comment	Nov 27, 2025 0:59am

3 Skipped Deployments

Project	Deployment	Preview	Comments	Updated (UTC)
rivet-cloud	Ignored			Nov 27, 2025 0:59am
rivet-inspector	Ignored	Preview		Nov 27, 2025 0:59am
rivet-site	Ignored	Preview		Nov 27, 2025 0:59am

[MasterPtato]

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

• fix(runtime): evict runners on SIGTERM, add periodic shutdown progress messages #3550
• fix(pb): implement events and commands for actor wfs #3543
• fix: make all uses of protocol handle v4 <-> v3 #3528
• fix(pb): simplify runner wf #3483
• fix(gas): fix loop forgotten bug due to concurrency #3482
• fix(pb): rewrite runner wf to handle batch signals #3480
• fix(gas): fix batch listen, fix history for graceful signal send in workflows #3479
• feat(gas): add batch listen #3478 👈 (View in Graphite)
• chore(gas): add test for history with holes #3477
• chore: change streamingmode to exact for ranges with limits #3476
• fix(gas): optimize loops #3475
• fix: parallelize publish signal awaits #3523
• chore: fix diff between ee, fix plural api query params #3524
• main

---

How to use the Graphite Merge Queue

Add the label merge-queue to this PR to add it to the merge queue.

You must have a Graphite account in order to use the merge queue. Sign up using this link.

An organization admin has enabled the Graphite Merge Queue in this repository.

Please do not merge from GitHub as this will restart CI on PRs being processed by the merge queue.

This stack of pull requests is managed by Graphite. Learn more about stacking.

[claude]

Pull Request Review: Batch Listen Feature

This PR introduces batch signal listening functionality to the workflow engine, allowing workflows to receive multiple signals in a single operation. This is a significant enhancement that can improve throughput and reduce overhead when dealing with multiple signals.

Summary

Changes: 1,215 additions, 732 deletions across 16 files
Core Feature: Transforms single-signal listening to batch-based signal processing

---

Positive Aspects

Architecture & Design

• Well-structured approach: The batch listening is implemented at multiple layers (trait, context, database) with proper abstraction
• Backwards compatibility: The single listen() method is preserved and now delegates to listen_n() with limit=1, maintaining existing API contracts
• Consistent pattern: Both listen_with_timeout and listen_until variants support batch operations

Code Quality

• Type safety: The Listen trait now returns Vec<Self>, enforcing that all signal types support batch operations
• History tracking: New SignalsEvent properly tracks multiple signals with coordinated signal_ids, names, and bodies vectors
• Error handling: Maintains existing error patterns while adapting to batch scenarios

---

Critical Issues

1. Breaking Change: Macro-generated Listen Implementation

Location: engine/packages/gasoline-macros/src/lib.rs:375-381

The #[signal] macro now generates a Listen implementation that returns Vec<Self>:

async fn listen(ctx: &mut gas::prelude::ListenCtx, limit: usize) -> gas::prelude::WorkflowResult<Vec<Self>> {
    ctx
        .listen_any(&[<Self as gas::signal::Signal>::NAME], limit)
        .await?
        .into_iter()
        .map(|signal| Self::parse(&signal.signal_name, &signal.body))
        .collect()
}

Issue: For a single signal type, this will return a Vec containing at most limit signals of the same type. However, if only 1 signal is available, it returns a 1-element vec. The consumer must now handle this differently.

Impact: All workflows using ctx.listen::<SomeSignal>() will need updates, or you need migration logic.

Recommendation: Document this breaking change clearly and consider:

• Providing a migration guide for existing workflows
• Adding deprecation warnings if keeping old single-signal methods temporarily

2. Inconsistent Limit Behavior

Location: engine/packages/gasoline/src/ctx/listen.rs:40-49

pub async fn listen_any(
    &mut self,
    signal_names: &[&'static str],
    limit: usize,
) -> WorkflowResult<Vec<SignalData>> {
    // ...
    if signals.is_empty() {
        return Err(WorkflowError::NoSignalFound(Box::from(signal_names)));
    }

Issue: The function fetches up to limit signals across ALL signal names, but the limit is applied per signal name in the database layer (line 1953 in db/kv/mod.rs). This creates ambiguity:

• Does limit=5 mean 5 total signals, or 5 per signal type?
• The current implementation fetches up to limit per signal type, then flattens them

Example Problem:

// Listening for 2 signal types with limit=5
// Could return up to 10 signals (5 per type)!
ctx.listen_any(&["signal_a", "signal_b"], 5).await?

Recommendation:

• Clarify the limit semantics in documentation
• Consider renaming to limit_per_type or enforcing a global limit by taking only the first N after flattening

3. Missing Signal ID Tracking

Location: engine/packages/gasoline/src/history/event.rs:210-214

pub struct SignalsEvent {
    pub names: Vec<String>,
    pub bodies: Vec<Box<serde_json::value::RawValue>>,
}

Issue: The public SignalsEvent struct in the event history doesn't include signal_ids, but the debug version does (line 1535 in db/kv/debug.rs). This makes it impossible to:

• Correlate signals to their original IDs in user code
• Debug which specific signals were processed
• Implement proper idempotency checks

Recommendation: Add pub signal_ids: Vec<Id> to the public SignalsEvent struct.

4. Data Consistency Risk in History

Location: engine/packages/gasoline/src/db/kv/debug.rs:1542-1577

The TryFrom<WorkflowHistoryEventBuilder> for SignalsEvent checks that vectors are non-empty but doesn't validate they have the same length:

Ok(SignalsEvent {
    signal_ids: if value.indexed_signal_ids.is_empty() {
        return Err(WorkflowError::MissingEventData("signal_id"));
    } else {
        value.indexed_signal_ids
    },
    names: if value.indexed_names.is_empty() {
        return Err(WorkflowError::MissingEventData("name"));
    } else {
        value.indexed_names
    },
    bodies: if value.indexed_input_chunks.is_empty() {
        // ...
    }
})

Issue: If the vectors have different lengths (due to corruption or partial writes), this could cause panics or incorrect signal matching.

Recommendation: Add validation:

ensure!(
    value.indexed_signal_ids.len() == value.indexed_names.len() 
        && value.indexed_names.len() == value.indexed_input_chunks.len(),
    "signal event data vectors must have equal length"
);

---

Moderate Issues

5. Error Context Loss

Location: engine/packages/gasoline/src/ctx/workflow.rs:707

pub async fn listen<T: Listen>(&mut self) -> Result<T> {
    let signals = self.listen_n::<T>(1).in_current_span().await?;
    signals
        .into_iter()
        .next()
        .context("must return at least 1 signal")
}

Issue: The error message "must return at least 1 signal" is generic and doesn't indicate which signal type was expected. This makes debugging harder.

Recommendation:

.with_context(|| format!("must return at least 1 signal for type {}", std::any::type_name::<T>()))

6. Metrics Removed Without Explanation

Location: engine/packages/gasoline/src/ctx/listen.rs (removed lines 28-42)

The SIGNAL_PULL_DURATION metric was removed, but it provided valuable insight into database performance.

Recommendation: Either restore the metric with batch-aware labels, or document why it was removed in the commit message.

7. Unused Loop Location Parameter

Location: engine/packages/gasoline/src/db/kv/mod.rs:1922

_loop_location: Option<&Location>,

Issue: The parameter is prefixed with _, indicating it's unused. If it's not needed, remove it from the trait definition.

Recommendation: Clean up unused parameters or document why they're reserved for future use.

---

Minor Issues & Suggestions

8. Display Implementation Inefficiency

Location: engine/packages/gasoline/src/history/event.rs:77-83

EventData::Signals(signals) => {
    let mut unique_names = signals.names.clone();
    unique_names.sort();
    unique_names.dedup();
    write!(f, "signals {:?}", unique_names.join(", "))
}

Suggestion: This allocates and clones for every display call. Consider using a HashSet or caching if displayed frequently.

9. Naming: last_attempt vs last_try

Location: engine/packages/gasoline/src/ctx/listen.rs:17

The parameter was renamed from last_try to last_attempt. While more professional, ensure all documentation references are updated.

10. Test Coverage

Observation: No new test files were added for the batch listening functionality.

Recommendation: Add tests for:

• Batch signal reception with multiple signals
• Limit behavior with various signal counts
• History replay with SignalsEvent
• Edge cases (empty results, limit=0, etc.)

---

Security Considerations

11. Resource Exhaustion Risk

Location: Database signal fetching logic

Issue: The limit parameter is user-controlled (via listen_n()). If a workflow requests limit=1000000, it could:

• Consume excessive memory
• Cause database performance issues
• Enable DoS attacks

Recommendation:

• Add a maximum limit constant (e.g., MAX_BATCH_SIZE = 100)
• Document the reasonable limit range
• Consider adding metrics for large batch requests

---

Performance Considerations

Positive

• Batch operations reduce database round-trips
• Parallel signal stream fetching (line 1941-1972 in db/kv/mod.rs) is well-designed

Potential Concerns

• Multiple signal types with large limits could cause memory spikes
• No pagination for very large signal backlogs

---

Documentation & Style

Per CLAUDE.md:

• ✅ Uses structured logging correctly (e.g., tracing::info!(?x, "foo"))
• ✅ Lowercase log messages
• ✅ Proper error handling with anyhow::Result
• ✅ Follows timestamp naming (create_ts)
• ❌ Could use more inline documentation for complex functions like pull_next_signals

---

Recommendations

High Priority

1. Clarify limit semantics - Document whether limit is per-signal-type or total
2. Validate vector lengths in SignalsEvent deserialization
3. Add signal_ids to public API for better debugging and idempotency
4. Add integration tests for batch signal handling
5. Document breaking changes for workflows using the old API

Medium Priority

6. Restore or replace removed metrics
7. Add maximum limit validation to prevent resource exhaustion
8. Improve error messages with signal type information
9. Clean up unused parameters

Low Priority

10. Optimize Display implementation
11. Add inline documentation for complex database operations

---

Conclusion

This is a well-architected feature that adds valuable batch processing capabilities to the workflow engine. The implementation is mostly solid with good separation of concerns and proper abstraction layers.

However, there are critical issues around limit semantics and data consistency that should be addressed before merging. The breaking API changes also need clear documentation and migration guidance.

Overall Assessment: Approve with requested changes

Recommendation: Address critical issues #1-4, then merge. Other issues can be follow-up PRs.

---

🤖 Generated with Claude Code

[claude]

PR Review: feat(gas): add batch listen

Summary

This PR introduces batch signal listening to the Gasoline workflow engine, allowing workflows to listen for and process multiple signals at once instead of one at a time. This is a significant architectural improvement that can reduce latency and improve throughput for workflows that process multiple signals.

---

Code Quality & Best Practices

Strengths:

• ✅ Good use of structured logging with tracing (e.g., tracing::debug!(?x, "foo") pattern)
• ✅ Proper error handling with custom error types
• ✅ Backward compatibility: Old EventType::Signal is properly converted to the new EventType::Signals format
• ✅ Consistent naming conventions (snake_case, past tense for timestamps)
• ✅ Good separation of concerns between database layer and workflow context

Issues & Suggestions:

1. Missing signal_id in SignalsEvent (engine/packages/gasoline/src/history/event.rs:210-214)

   pub struct SignalsEvent {
       pub names: Vec<String>,
       pub bodies: Vec<Box<serde_json::value::RawValue>>,
   }

   The SignalsEvent struct is missing signal_ids: Vec<Id> field that appears to be used in the debug implementation and history keys. This could cause issues when replaying events or debugging.

2. Potential ordering issue (engine/packages/gasoline/src/db/kv/mod.rs:2059-2063)

   // Sort by ts
   signals.sort_by_key(|key| key.create_ts);
   
   // Apply limit
   Ok(signals.into_iter().take(limit).collect())

   The limit is applied AFTER sorting, but signals are fetched with a limit from each signal name subspace. If you have 100 signals of type A and 1 of type B, and request limit=10, you might get all 10 from A and miss the one from B even if B is older. Consider sorting BEFORE fetching or using a priority queue approach.

3. Deprecated metric removed (engine/packages/gasoline/src/metrics.rs)
   The SIGNAL_PULL_DURATION metric was removed but might still be valuable for performance monitoring. Consider keeping it and recording the total duration for pulling all signals in a batch.

4. API Breaking Change
   The Listen trait now requires returning Vec<Self> instead of Self, and CustomListener trait was removed. This is a breaking change that should be:

   • Documented in a changelog/migration guide
   • Validated that all existing implementations are updated

---

Potential Bugs

1. Empty signals vec guarantee (engine/packages/gasoline/src/ctx/workflow.rs:703-710)

   pub async fn listen<T: Listen>(&mut self) -> Result<T> {
       let signals = self.listen_n::<T>(1).in_current_span().await?;
   
       signals
           .into_iter()
           .next()
           .context("must return at least 1 signal")
   }

   The code assumes listen_n will never return an empty vec, but this isn't enforced by the type system. If listen_n returns empty, this will panic with "must return at least 1 signal". Consider adding an assertion in listen_n or handling this more gracefully.

2. Race condition in listen_n_until (engine/packages/gasoline/src/ctx/workflow.rs:1154-1182)
   After a sleep is interrupted, the code checks for signals in history. However, if the history check finds HistoryResult::New instead of signals, it continues to listen for new signals. There's a potential edge case where signals could be missed between the sleep interruption and the history check.

3. Buffer size assumption (engine/packages/gasoline/src/db/kv/mod.rs:2055)

   .buffer_unordered(1024)

   Hard-coded buffer of 1024 could be problematic if limit is much smaller or larger. Consider using buffer_unordered(limit.max(16).min(1024)) for better performance.

---

Performance Considerations

Improvements:

• ✅ Batching reduces database round trips
• ✅ Using buffer_unordered for concurrent signal processing
• ✅ Sorting and limiting happen after fetching, avoiding multiple DB queries

Concerns:

1. Flattening signal streams (engine/packages/gasoline/src/db/kv/mod.rs:1941-1961)

   futures_util::stream::iter(owned_filter.clone())
       .map(|signal_name| { /* ... */ })
       .flatten()

   Using .flatten() on multiple range queries could lead to uneven fetching if one signal type has many more entries than others. The current approach fetches limit from EACH signal type, then sorts and takes limit total. This could fetch more data than needed.

2. Multiple indexed key writes (engine/packages/gasoline/src/db/kv/keys/history.rs)
   The new implementation writes separate keys for:

   • Each signal ID (IndexedSignalIdKey)
   • Each signal name (IndexedNameKey)
   • Each signal input chunk (IndexedInputChunkKey)

   This could significantly increase write amplification. For N signals with M chunks each, you're writing ~N*(2+M) keys instead of N keys. Consider benchmarking the performance impact.

---

Security Concerns

1. No validation on limit parameter
   The limit parameter in listen_n is not validated. A malicious or buggy caller could pass usize::MAX which could cause:

   • Excessive memory allocation
   • Database query timeouts
   • DoS conditions

   Recommend adding: let limit = limit.min(MAX_SIGNAL_BATCH_SIZE); with a reasonable constant.

2. Signal body size not validated
   When combining chunks into signal bodies, there's no validation that the total size is reasonable. Large signal bodies could cause OOM issues.

---

Test Coverage

Missing:

• ❌ No test files modified in this PR
• ❌ No tests for listen_n functionality
• ❌ No tests for batch signal processing
• ❌ No tests for the new SignalsEvent type
• ❌ No tests for backward compatibility (old Signal → new Signals conversion)
• ❌ No tests for limit parameter edge cases (0, 1, large numbers)
• ❌ No tests for signal ordering when batching multiple signal types

Recommended Tests:

// Test batch listen with multiple signals
#[tokio::test]
async fn test_listen_n_multiple_signals() {
    // Send 5 signals, listen for 3, verify we get the oldest 3
}

// Test listen_n with limit larger than available signals
#[tokio::test]  
async fn test_listen_n_limit_exceeds_available() {
    // Send 2 signals, listen for 5, verify we get 2
}

// Test backward compatibility
#[tokio::test]
async fn test_old_signal_event_conversion() {
    // Load workflow with old Signal events, verify they convert to Signals
}

// Test signal ordering across types
#[tokio::test]
async fn test_listen_n_mixed_signal_types() {
    // Send signals of type A and B interleaved by time
    // Verify they're received in chronological order
}

---

Architecture & Design

Positive:

• The batch approach is a good architectural decision for high-throughput scenarios
• Clean separation between listen() (single) and listen_n() (batch) maintains backward compatibility at the API level
• The history cursor changes properly support the new batch model

Questions:

1. Why was CustomListener removed? Were there no use cases for it, or does batch listening subsume its functionality?
2. Should there be a way to listen for "all available signals up to N" vs "exactly N signals or error"? The current implementation errors if fewer signals are available.

---

Summary & Recommendations

This is a well-structured PR that adds valuable batching capabilities to the workflow engine. However, it needs:

Must Fix:

1. Add signal_ids field to SignalsEvent struct
2. Add validation for limit parameter to prevent DoS
3. Fix the ordering issue in signal fetching (fetch limit total, not limit per type)
4. Add comprehensive test coverage

Should Fix:

1. Consider keeping SIGNAL_PULL_DURATION metric
2. Document the breaking changes to Listen trait
3. Add buffer size tuning based on limit
4. Add signal body size validation

Nice to Have:

1. Add migration guide for users of old CustomListener trait
2. Benchmark write amplification impact
3. Consider exposing batch size as a configuration parameter

The core implementation is solid, but the missing tests and potential edge cases around limits and ordering need to be addressed before merging.

---

Generated by Claude Code

[claude]

Code Review

Summary

This PR adds batch signal listening capabilities to the Gasoline workflow engine, allowing workflows to receive multiple signals in a single operation instead of one at a time. This is a significant architectural improvement that should improve efficiency when dealing with high-volume signal processing.

Architecture Changes

✅ Well-designed changes:

1. Signal History Consolidation: The new SignalsEvent replaces individual signal events in history, batching multiple signals into a single event. This is more efficient and cleaner than the previous approach.

2. Indexed Storage: The implementation uses indexed keys (IndexedSignalIdKey, IndexedNameKey, IndexedInputKey) to store multiple signals within a single history event, which is a clean database schema design.

3. Backward Compatibility: The PR properly deprecates the old Signal event type (EventType::Signal = 1) while introducing Signals (EventType::Signals = 10), maintaining compatibility.

Code Quality

✅ Strengths:

1. Consistent API Design: The new listen_n() and listen_n_with_timeout() methods follow the existing naming conventions and patterns.

2. Proper Error Handling: Maintains proper error handling throughout, including history divergence checks in compare_signals().

3. Metrics & Observability: Maintains logging and metrics (signal receive lag tracking, etc.).

4. Macro Updates: The #[signal] macro and join_signal\! macro properly updated to return Vec<Self> instead of single values.

Potential Issues

⚠️ Critical Issues:

1. EventType Mismatch in insert::signals_event (packages/gasoline/src/db/kv/keys/history.rs:1645)

pub fn signals_event(/* ... */) -> Result<()> {
    common(
        subspace,
        tx,
        workflow_id,
        location,
        EventType::Signal,  // ❌ Should be EventType::Signals
        version,
        create_ts,
    )
}

This appears to use EventType::Signal (the deprecated type) instead of EventType::Signals. This could cause history corruption or replay issues.

Recommended fix:

EventType::Signals,  // Use the new batch signal type

2. Missing Validation on Signal Count

The SignalsEvent struct (packages/gasoline/src/history/event.rs:211-214) stores names and bodies as separate vectors but does not enforce that they have the same length:

pub struct SignalsEvent {
    pub names: Vec<String>,
    pub bodies: Vec<Box<serde_json::value::RawValue>>,
}

In the database deserialization code (packages/gasoline/src/db/kv/debug.rs:1545), there is a zip operation that could silently drop data if lengths do not match. Consider adding validation:

pub struct SignalsEvent {
    pub names: Vec<String>,
    pub bodies: Vec<Box<serde_json::value::RawValue>>,
}

impl SignalsEvent {
    pub fn new(names: Vec<String>, bodies: Vec<Box<serde_json::value::RawValue>>) -> Result<Self> {
        ensure\!(names.len() == bodies.len(), "signals names and bodies must have same length");
        ensure\!(\!names.is_empty(), "signals event must contain at least one signal");
        Ok(Self { names, bodies })
    }
}

⚠️ Medium Priority Issues:

3. Removed CustomListener Without Clear Migration Path

The PR removes the CustomListener trait and custom_listener() method entirely:

• Removed from packages/gasoline/src/ctx/versioned_workflow.rs (lines 128-136)
• Not mentioned in PR description

If any users were using CustomListener, this is a breaking change. Consider:

1. Adding a deprecation note/migration guide
2. Ensuring no internal code was using this functionality
3. Documenting this breaking change

4. Potential Performance Issue: Sort After Parallel Fetch

In pull_next_signals (packages/gasoline/src/db/kv/mod.rs:2059-2063):

.buffer_unordered(1024)
.try_collect::<Vec<_>>()
.await?;

// Sort by ts
signals.sort_by_key(|key| key.create_ts);

// Apply limit
Ok(signals.into_iter().take(limit).collect())

The code fetches up to limit signals from each signal name stream in parallel, then sorts all results by timestamp, then takes limit items. This means:

• If listening for N signal types with limit=10, you might fetch 10*N signals then discard all but 10
• The buffer_unordered(1024) suggests very high parallelism but the final limit might waste most of that work

Recommendation: Consider limiting the total fetch earlier or documenting the tradeoff.

Performance Considerations

✅ Good:

• Batching signals reduces database round-trips
• Parallel signal body fetching with buffer_unordered(1024)
• Maintains existing polling/subscription mechanisms

⚠️ Concern:

• The listen_n semantics could lead to workflows requesting large batches unnecessarily if not used carefully
• No apparent safeguard against very large batch sizes (could the limit parameter be validated against a max?)

Security

✅ No obvious security concerns

• Maintains existing workflow isolation
• No new injection vectors
• Proper deserialization error handling

Test Coverage

❌ Critical Gap: No Tests for Batch Listening

The PR adds significant new functionality but I could not find any tests for:

• listen_n() functionality
• listen_n_with_timeout()
• listen_n_until()
• Multiple signals in a single SignalsEvent
• History replay with batched signals

Recommendation: Add tests covering:

1. Basic listen_n(limit=5) with multiple signals arriving
2. Behavior when fewer than limit signals arrive
3. Correct ordering of batched signals by timestamp
4. History replay with SignalsEvent containing multiple signals
5. Timeout behavior with listen_n_with_timeout

Example test case:

#[workflow_test]
async fn test_batch_signal_listen() {
    // Publish 5 signals
    // ctx.listen_n::<MySignal>(10) should return all 5
    // Verify order matches timestamps
}

Additional Observations

1. Good: The listen() method is now implemented as listen_n(1) which is a clean refactor that ensures consistency.

2. Good: Updated print/debug utilities to handle the new SignalsEvent type.

3. Question: The PR description is empty. Consider adding:

   • Why batch listening is needed
   • Performance improvement metrics (if available)
   • Breaking changes documentation
   • Migration guide for CustomListener users

Recommendations

Must Fix:

1. Fix EventType::Signal → EventType::Signals in insert::signals_event
2. Add validation to SignalsEvent to ensure names.len() == bodies.len()
3. Add comprehensive test coverage for batch listening

Should Consider:

1. Add PR description explaining the changes and migration path
2. Document the breaking change (removal of CustomListener)
3. Review the fetch-all-then-limit performance pattern in pull_next_signals
4. Add max batch size validation to prevent abuse

Nice to Have:

1. Add examples of using listen_n in documentation
2. Performance benchmarks showing improvement over sequential signal listening

Verdict

This is a well-architected feature that makes logical sense for the workflow engine. However, it has a critical bug (wrong EventType) and lacks test coverage. Once these issues are addressed, this will be a solid improvement.

Status: Needs Changes 🔴

The EventType bug could cause production issues with history replay, and the lack of tests makes it risky to merge. Please address the critical issues before merging.

[claude]

PR Review: Add Batch Listen

Overview

This PR introduces batch signal listening capabilities to the Gasoline workflow engine, allowing workflows to receive multiple signals in a single operation.

Strengths

1. Well-Structured API Changes

• Listen trait now returns Vec - clean abstraction
• Backward compatibility via listen() calling listen_n(1)
• Clear batch semantics with listen_n() and listen_n_with_timeout()

2. Comprehensive Database Implementation

• Well-designed indexed signal storage keys (IndexedSignalIdKey, IndexedNameKey, IndexedInputKey)
• Proper chunked keys for large signal bodies
• SignalsEvent type properly supports multiple signals

3. Good Error Handling

• Empty vecs trigger NoSignalFound appropriately (ctx/listen.rs:68)
• History divergence checks validate signal events (history/cursor.rs:561-590)

4. Logging and Observability

• Structured logging with signal counts, IDs, names
• Per-signal metrics for receive lag
• Enhanced debug printing (engine/src/util/wf/mod.rs:470-497)

Concerns

1. Breaking API Change ⚠️
Listen trait signature changed. Any custom implementations will break.

Recommendations:

• Document as breaking change in PR description
• Search for manual Listen implementations
• Add migration notes

2. Potential Index Corruption 🔴
db/kv/debug.rs:897-940 rebuilds events from indexed keys with ordering checks. Partial transaction failures could cause corruption on replay.

Recommendations:

• Add integration tests for partial write failures
• Consider repair/recovery logic
• Document invariants

3. Empty Vec Semantics ⚠️
Inconsistent behavior:

• listen_n_with_timeout() → empty vec valid (timeout)
• listen_n() → empty vec errors

Recommendation: Document when empty vecs are expected vs errors

4. Test Coverage Gaps 🟡
Missing tests for:

• Batch listening with n > 1
• Mixed signal types in batches
• Replay with batch signals
• Edge cases (more requested than available)

5. Performance 💭
Sequential processing of signals for metrics/logging could add latency for large batches.

6. Magic Number 🟡
recv_lag > 3.0 check needs constant and explanation (ctx/listen.rs:82)

7. Removed EventData::Empty
Was Empty unused or does this affect backfilling?

Security & Safety

✅ No security concerns
✅ Proper transactions and conflict ranges
✅ Signal ack timestamps correct

Conclusion

Well-implemented feature! Main concerns:

1. Breaking change documentation needed
2. Index corruption risk - needs testing
3. Batch-specific test coverage needed

Recommendation: Add breaking change docs and batch tests before merging.

Overall: Approve with comments 🎉

[claude]

PR Review: feat(gas): add batch listen

This PR introduces batch signal listening to the Gasoline workflow engine, allowing workflows to receive multiple signals in a single operation. This is a significant performance optimization.

Architecture Changes

Core API Changes:

• Listen trait now returns Vec and accepts a limit parameter
• WorkflowCtx::listen() now internally calls listen_n(1)
• New methods: listen_n(), listen_n_with_timeout(), and listen_n_until()
• Removed CustomListener trait

Database Layer:

• New pull_next_signals() method with parallel signal fetching
• Signals sorted by timestamp and limited to requested count
• New EventType::Signals replaces individual signal events

History Management:

• New SignalsEvent with indexed data structures
• New history keys for indexed signals
• New cursor method: compare_signals()

Code Quality - Strengths

• Excellent use of parallel processing with buffer_unordered(1024)
• Proper transaction boundaries with conflict ranges
• Maintains backward compatibility
• Comprehensive tracing and metrics
• Proper error propagation

Areas for Improvement

1. Transaction Size (db/kv/mod.rs:1988-2057): Large batches could create very large transactions. Consider transaction size limits.

2. Sorting After Limit (db/kv/mod.rs:2059-2063): Fetches limit signals from each stream, then sorts and limits again. Could fetch more than needed. Add clarifying comment or adjust logic.

3. Error Context (ctx/workflow.rs:707-708): Use expect() instead of context() since empty result indicates a bug.

Potential Bugs

1. Index Ordering (db/kv/debug.rs:901-927): Assumes indices arrive in order. Use HashMap for robustness.

2. Empty Signals Vec (ctx/workflow.rs:1165-1177): When replaying with SleepState::Interrupted, code assumes non-empty vec. Add validation.

Performance

Positive:

• Parallel signal fetching is excellent
• Single transaction reduces round trips

Concerns:

• No bounds on signal body sizes (memory pressure)
• Metrics recorded in loop for each signal (could be slow for large batches)

Security

No significant issues. Proper conflict ranges prevent races, signal ack prevents double-delivery.

Test Coverage

Missing: No test files in PR. Critical tests needed:

1. listen_n() with limit > available signals
2. Signal ordering with mixed timestamps
3. History replay with batched signals
4. listen_n_until() timeout scenarios
5. Transaction conflicts during batch pull

Best Practices

Follows CLAUDE.md conventions well (structured logging, lowercase messages, anyhow::Result, proper timestamps).

Minor: Some functions exceed reasonable length (180 lines). Consider extracting helpers.

Final Recommendation

APPROVE with minor recommendations

Well-architected change with meaningful performance improvements. Code quality is high and follows conventions.

Before merging:

1. Add integration tests
2. Address empty signals vector edge case
3. Add clarifying comments for per-stream limit logic
4. Consider transaction size limits

Nice work! The parallel signal fetching is particularly well done.

[pkg-pr-new]

More templates

• example-ai-agent
• chat-room
• example-cloudflare-workers
• example-cloudflare-workers-hono
• example-cloudflare-workers-inline-client
• example-counter
• example-counter-next-js
• example-counter-serverless
• example-cursors
• example-cursors-raw-websocket
• example-deno
• example-sqlite
• example-elysia
• example-express
• example-freestyle
• example-hono
• example-hono-bun
• example-hono-react
• example-kitchen-sink
• example-next-js
• quickstart-actions
• quickstart-cross-actor-actions
• quickstart-multi-region
• quickstart-raw-websockets
• quickstart-realtime
• quickstart-scheduling
• quickstart-state
• example-raw-fetch-handler
• example-raw-websocket-handler
• example-raw-websocket-handler-proxy
• example-react
• starter
• example-stream
• example-trpc

@rivetkit/cloudflare-workers

pnpm add https://pkg.pr.new/rivet-dev/rivet/@rivetkit/cloudflare-workers@3478

@rivetkit/db

pnpm add https://pkg.pr.new/rivet-dev/rivet/@rivetkit/db@3478

@rivetkit/framework-base

pnpm add https://pkg.pr.new/rivet-dev/rivet/@rivetkit/framework-base@3478

@rivetkit/next-js

pnpm add https://pkg.pr.new/rivet-dev/rivet/@rivetkit/next-js@3478

@rivetkit/react

pnpm add https://pkg.pr.new/rivet-dev/rivet/@rivetkit/react@3478

rivetkit

pnpm add https://pkg.pr.new/rivet-dev/rivet/rivetkit@3478

@rivetkit/sql-loader

pnpm add https://pkg.pr.new/rivet-dev/rivet/@rivetkit/sql-loader@3478

@rivetkit/engine-runner

pnpm add https://pkg.pr.new/rivet-dev/rivet/@rivetkit/engine-runner@3478

@rivetkit/engine-runner-protocol

pnpm add https://pkg.pr.new/rivet-dev/rivet/@rivetkit/engine-runner-protocol@3478

commit: 3f3140e

[claude]

Code Review: feat(gas): add batch listen

This PR adds batch signal listening capabilities to Gasoline (the workflow engine), allowing workflows to receive multiple signals in a single operation. Overall, this is a well-structured feature addition.

Summary of Changes

• New batch listen APIs: listen_n(), listen_n_with_timeout(), listen_n_until() for receiving multiple signals
• Renamed event type: Signal → Signals (plural) to reflect batch semantics
• New SignalsEvent struct: Stores arrays of signal IDs, names, and bodies
• Updated Listen trait: Now returns Vec<Self> with a limit parameter
• Removed CustomListener trait: Functionality consolidated into the standard Listen trait
• Database changes: pull_next_signal → pull_next_signals with limit parameter

Positives

1. Clean API design: The batch APIs mirror the single-signal APIs nicely (listen → listen_n, etc.)
2. Good backwards compatibility: Single-signal methods now delegate to batch methods with limit=1
3. Proper history handling: New indexed keys (IndexedSignalIdKey, IndexedNameKey, IndexedInputKey) support storing multiple signals per event
4. Comprehensive debug support: Added SignalsEvent handling in print_history and debug code

Issues & Suggestions

1. Potential Data Inconsistency in SignalsEvent

File: engine/packages/gasoline/src/history/event.rs:210-214

pub struct SignalsEvent {
    pub names: Vec<String>,
    pub bodies: Vec<Box<serde_json::value::RawValue>>,
}

The SignalsEvent struct stores names and bodies as separate vectors that must remain in sync. However, the debug struct (db/debug.rs:175-180) also includes signal_ids:

pub struct SignalsEvent {
    pub signal_ids: Vec<Id>,
    pub names: Vec<String>,
    pub bodies: Vec<serde_json::Value>,
}

Suggestion: Consider using a single struct with aligned fields or a vector of tuples/structs to prevent potential misalignment issues.

2. Missing Signal IDs in History Event

File: engine/packages/gasoline/src/history/event.rs:210-214

The runtime SignalsEvent doesn't include signal_ids, but the debug version and database storage do. If signal IDs are needed for replay or debugging, they should be included in the runtime struct as well.

3. Sort After Limit Application Issue

File: engine/packages/gasoline/src/db/kv/mod.rs:2056-2060

// Sort by ts
signals.sort_by_key(|key| key.create_ts);

// Apply limit
Ok(signals.into_iter().take(limit).collect())

The signals are fetched with limit already applied to each pending signal stream (line 1950), then sorted, then limited again. This could lead to incorrect behavior:

• Each signal name stream is limited to limit signals
• If you have 3 signal types and limit=2, you could get up to 6 signals
• After sorting by timestamp and taking 2, you might miss older signals from other streams

Suggestion: The limit should be applied after aggregating and sorting all signals, not per-stream. Consider:

1. Remove the per-stream limit or set it higher
2. Keep only the final sort + limit

4. Unnecessary Variable in listen_any

File: engine/packages/gasoline/src/ctx/listen.rs:90

let recv_lag = (now as f64 - signal.create_ts as f64) / 1000.0;

Using 1000.0 instead of 1000. is fine, but the calculation is repeated for each signal. Consider extracting now outside the loop (which is already done, good!).

5. Removed Metric

File: engine/packages/gasoline/src/metrics.rs - Removed SIGNAL_PULL_DURATION

The SIGNAL_PULL_DURATION metric was removed. This could impact monitoring capabilities. Was this intentional?

Minor Suggestions

1. Documentation: Consider adding doc comments to the new listen_n* methods explaining the batch semantics and what happens with partial results

2. Error message clarity in ctx/workflow.rs:705:

   .context("must return at least 1 signal")

   This could be more descriptive about when this might fail.

3. Consider empty vec vs error: listen_n returns Result<Vec<T>> but errors if empty in listen(). Consider whether returning an empty vec on timeout is the desired behavior for listen_n_with_timeout vs listen_n.

Questions

1. Is there a maximum batch size that should be enforced? Large batches could impact memory usage and transaction sizes.

2. Should there be a listen_all_pending() method that drains all pending signals without a limit?

Test Coverage

I didn't see new tests for the batch listen functionality in this PR. Consider adding tests for:

• Batch listening with multiple signals of the same type
• Batch listening with mixed signal types
• Timeout behavior with partial results
• History replay with batch signals
• Edge cases: limit=0, limit=1 (should match single-signal behavior)

---

🤖 Generated with Claude Code

[claude]

PR Review: feat(gas): add batch listen

Summary

This PR adds batch signal listening capabilities to the gasoline workflow engine. The changes transition from single signal listening (pull_next_signal returning Option<SignalData>) to batch signal listening (pull_next_signals returning Vec<SignalData>). This is a well-structured change that enables more efficient signal processing.

---

✅ Positive Aspects

1. Clean API Design: The new listen_n<T>() and listen_n_with_timeout<T>() methods provide a clean extension to the existing API while maintaining backward compatibility through the single-signal listen<T>() method.

2. Proper Event Type Handling: The addition of EventType::Signals = 10 and EventData::Signals(SignalsEvent) maintains backward compatibility with the existing EventType::Signal = 1 (marked as deprecated).

3. Good Metrics Handling: The lag metrics are properly updated per-signal in the batch (ctx/listen.rs:60-77), maintaining observability.

4. Proper History Cursor Updates: The new compare_signals() method in cursor.rs:561-590 follows the established pattern of other comparison methods.

5. Removal of CustomListener: Consolidating custom_listener into the standard Listen trait simplifies the API.

---

⚠️ Potential Issues & Suggestions

1. Race Condition in pull_next_signals (db/kv/mod.rs:1984-2060)

let mut signals =
    futures_util::stream::iter(signals.into_iter().enumerate())
        .map(|(index, key)| { /* ... */ })
        .buffer_unordered(1024)  // <-- Unordered execution
        .try_collect::<Vec<_>>()
        .await?;

// Sort by ts
signals.sort_by_key(|key| key.create_ts);

Using buffer_unordered means signals are processed in non-deterministic order, and then sorted by create_ts. While this is then corrected by sorting, the signals_event_signal history entries (written inside the async block at line 2033-2042) will have indices that don't match the final sorted order. This could cause issues during replay.

Suggestion: Consider using buffered() instead of buffer_unordered() to maintain order, or sort the signals before writing history events.

2. Limit Applied After Collection (db/kv/mod.rs:2060)

Ok(signals.into_iter().take(limit).collect())

The limit is applied to each signal type's query independently (line 1950: limit: Some(limit)), but after collecting all signal types and sorting, the limit is applied again. This means if you have 3 signal types and limit=10, you could fetch up to 30 signals total but only return 10. This could waste resources.

Suggestion: Consider documenting this behavior or optimizing the query strategy.

3. Missing signal_ids in SignalsEvent (history/event.rs:211-214)

pub struct SignalsEvent {
    pub names: Vec<String>,
    pub bodies: Vec<Box<serde_json::value::RawValue>>,
}

The SignalsEvent struct doesn't include signal_ids, but the debug version in db/debug.rs:175-180 does. This inconsistency could make debugging/replay harder.

Suggestion: Add signal_ids to the main SignalsEvent struct for consistency.

4. Panic Potential with .context("must return at least 1 signal") (ctx/workflow.rs:704)

signals
    .into_iter()
    .next()
    .context("must return at least 1 signal")

If listen_n ever returns an empty vec due to a bug, this will cause an error. While the current implementation does return Err(WorkflowError::NoSignalFound) when signals are empty, this is defensive but worth noting.

5. Missing Metric for Signal Pull Duration (ctx/listen.rs)

The original code had a SIGNAL_PULL_DURATION metric that was removed. This could reduce observability.

Suggestion: Consider keeping the pull duration metric or adding a new one that covers the batch operation.

---

🔍 Code Quality Notes

1. Good naming: Renaming last_try to last_attempt improves clarity.

2. Consistent error messages: The error messages like "expected signals at {}, found nothing" are clear and helpful.

3. Proper cleanup: The removal of EventData::Empty in favor of EventData::Signals is cleaner.

---

📋 Test Coverage

I don't see new tests specifically for the batch listen functionality in this PR. Consider adding tests for:

• Listening for N signals returns exactly N (or fewer if not available)
• Signal ordering is preserved by create_ts
• History replay correctly handles multi-signal events
• Edge cases: limit=0, limit=1, limit > available signals

---

🔒 Security Considerations

No security concerns identified. The changes are internal to the workflow engine and don't expose new attack surfaces.

---

Overall Assessment

This is a solid feature addition that enables batch signal processing. The main concerns are:

1. The potential history index ordering issue with buffer_unordered
2. Missing tests for the new functionality

The architecture follows established patterns in the codebase and the API design is clean and backward compatible.

[claude]

PR Review: Add Batch Listen (feat(gas): add batch listen)

Summary

This PR introduces batch signal listening to the Gasoline workflow engine, allowing workflows to receive multiple signals at once instead of processing them one at a time. This is a significant enhancement that can improve throughput for workflows handling multiple concurrent signals.

Code Quality & Best Practices ✅

Strengths:

• Clean API design: The changes maintain backward compatibility by wrapping listen() in terms of listen_n(), which is good design
• Consistent naming: listen_n, listen_n_with_timeout, listen_n_until follow a clear naming pattern
• Good separation of concerns: The batch logic is properly isolated in the database layer
• Proper error handling: Maintains the existing error patterns and adds appropriate context
• Tracing instrumentation: All new functions include proper #[tracing::instrument] annotations
• Follows repository conventions: Uses workspace dependencies, proper imports, and adheres to CLAUDE.md guidelines

Observations:

• The macro changes in gasoline-macros/src/lib.rs properly update the Listen trait implementation to handle vectors
• The join_signal! macro was updated correctly to support batch operations (signal.rs:60-69)

Potential Bugs & Issues ⚠️

1. Critical - Empty vector guarantee: In ctx/workflow.rs:706-709, there's an assumption that listen_n returns at least 1 signal:

   signals.into_iter().next().context("must return at least 1 signal")

   However, if the implementation changes or has a bug, this could panic. Consider adding a debug assertion or better error message.

2. Sorting stability: In db/kv/mod.rs:2056-2060, signals are sorted by create_ts then truncated to limit:

   // Sort by ts
   signals.sort_by_key(|key| key.create_ts);
   // Apply limit
   Ok(signals.into_iter().take(limit).collect())

   This is correct, but the comment should clarify that sorting happens before the limit is applied to ensure we get the oldest N signals, not just any N signals sorted.

3. History divergence validation: The compare_signals function in cursor.rs:561+ validates version and event type but doesn't validate the count of signals matches. If replay encounters a different number of signals than history, it might cause subtle bugs. Consider validating signals.names.len() matches expectations.

4. Deprecated event type: The code marks EventType::Signal = 1 as deprecated (event.rs:92) but doesn't provide migration guidance. If there's existing history using the old event type, is there a migration path?

5. Transaction atomicity: In pull_next_signals (db/kv/mod.rs:1984-2054), signals are processed with .buffer_unordered(1024). If any signal processing fails partway through, some signals might have been acked while others weren't. The transaction will roll back, but this could leave the system in an inconsistent state if there are external side effects.

Performance Considerations 🚀

Positives:

• Batch processing: Fetching multiple signals at once reduces database round trips
• Concurrent processing: Using .buffer_unordered(1024) in signal processing (db/kv/mod.rs:2052) is excellent for parallelizing I/O
• Efficient querying: Uses StreamingMode::Exact with limit for efficient pagination

Concerns:

1. Memory allocation: Fetching signals from multiple streams simultaneously (db/kv/mod.rs:1938-1969) could consume significant memory if many signal types have large backlogs. Consider adding memory pressure checks or per-workflow limits.

2. Buffer size: The .buffer_unordered(1024) buffer size seems arbitrary. For workflows with very large signal bodies, this could buffer ~1024 signals in memory simultaneously. Consider making this configurable or documenting the rationale.

3. Metrics overhead: The PR removes the SIGNAL_PULL_DURATION metric (ctx/listen.rs, removed lines) which could make debugging performance issues harder. Was this intentional?

Security Concerns 🔒

No major security issues identified, but some considerations:

1. Resource exhaustion: Without a maximum limit on signal batch size, a malicious actor could potentially cause memory exhaustion by sending many large signals to a single workflow. The limit parameter is caller-controlled - ensure it's bounded at higher levels.

2. Signal ordering: Signals are sorted by create_ts which comes from the client. If timestamp can be manipulated, signal processing order could be affected. Verify that create_ts is set server-side.

Test Coverage 📋

Major Gap: No test files are included in this PR. Given the complexity of the changes, especially around:

• Batch signal processing logic
• History replay with multiple signals
• Edge cases (empty results, limit boundaries)
• Concurrency in buffer_unordered

Recommendations:

• Add unit tests for listen_n with various limits (0, 1, >1)
• Test listen_n_with_timeout with signals arriving before/after timeout
• Test history replay with SignalsEvent
• Test signal ordering with concurrent arrivals
• Test error cases (no signals found, conflicting signals)
• Integration tests for the end-to-end batch listen flow

Additional Observations

1. Documentation: The print_history utility was updated (engine/src/util/wf/mod.rs:470-498) to display batch signals, which is good for debugging.

2. Breaking changes: The Listen trait signature changed from returning Self to Vec<Self>. This is a breaking change for any custom implementations. Ensure all implementations in the codebase are updated.

3. Migration path: The code handles both old (Signal) and new (Signals) event types, which suggests this is an incremental rollout. Document the deprecation timeline.

4. Code clarity: The last_attempt parameter renamed from last_try (ctx/listen.rs:17,23) is clearer. Good improvement!

Recommendations

Before merging:

1. ✅ Add comprehensive test coverage
2. ✅ Add bounds checking on the limit parameter at the API level
3. ✅ Document the breaking Listen trait change in migration notes
4. ✅ Clarify the sorting behavior with inline comments
5. ⚠️ Consider re-adding metrics for batch signal pull duration
6. ⚠️ Add validation for signal count during history replay

Nice to have:

• Benchmark batch vs. single signal performance
• Document the buffer size choice (1024)
• Add examples of using listen_n in workflow code

Conclusion

This is a well-structured PR that adds valuable batch processing capabilities to the workflow engine. The code quality is high and follows project conventions. The main concerns are around test coverage (none provided), potential memory pressure from unbounded batching, and some edge cases in error handling.

The changes are backwards compatible through the wrapper approach, which is excellent. With additional tests and minor refinements around bounds checking and validation, this will be a solid addition to the codebase.

Overall Assessment: Approve with recommendations ✅

Great work on maintaining code consistency and designing a clean API! 🎉

[claude]

PR Review: feat(gas): add batch listen

Summary

This PR introduces batch signal listening functionality to the Gasoline workflow engine, allowing workflows to receive multiple signals at once instead of one at a time. This is a significant architectural change that affects the core signal handling mechanisms.

Code Quality & Architecture

Strengths

1. Well-structured change: The refactoring systematically updates the signal handling from single to batch operations across all layers (API, database, history, cursor).
2. Backward compatibility preserved: The listen() method still works as before by delegating to listen_n(1) and extracting the first signal, ensuring existing code continues to work.
3. Comprehensive implementation: The change properly propagates through all layers - from the trait definition to the database implementation to the history cursor.
4. Good error handling: The code properly handles edge cases like empty signal lists and maintains error context through the stack.

Areas of Concern

1. Breaking API Change in Listen trait (High Severity)

The Listen::listen signature changed from async fn listen(ctx: &mut ListenCtx) -> WorkflowResult to async fn listen(ctx: &mut ListenCtx, limit: usize) -> WorkflowResult<Vec>

Issue: This is a breaking change for any custom implementations of the Listen trait outside the codebase. The #[signal] macro was updated to generate the new signature, but any hand-written implementations will break.

Recommendation: Consider deprecation path or document this as a breaking change in release notes.

2. Inconsistent Error Handling (Medium Severity)

In workflow.rs:703-709, the .context() error message for an empty vec case could be more helpful for debugging.

Recommendation: Use a more specific error or add debug assertions.

3. Removed CustomListener Trait (Medium Severity)

The PR removes the CustomListener trait and custom_listener method without explanation.

Files affected:

• gasoline/src/listen.rs (trait removed)
• gasoline/src/ctx/versioned_workflow.rs:125-135 (method removed)

Recommendation: Document why this was removed.

4. History Storage Complexity (Medium Severity)

The new Signals event type stores multiple signals with indexed storage. The deserialization logic in db/kv/debug.rs has checks that assume indexes arrive in order (0, 1, 2...).

Recommendation: Add documentation about ordering guarantees.

5. Missing Test Coverage (High Severity)

No new tests were added for the batch listen functionality.

Missing test scenarios:

• Multiple signals received in one batch
• Batch size limits (what happens with limit parameter?)
• Order preservation when multiple signals arrive
• Interaction with listen_with_timeout and batches
• The new listen_n and listen_n_with_timeout methods

Recommendation: Add comprehensive tests in tests/workflows/signal_test.rs.

Performance Considerations

Positive Impact

1. Reduced polling overhead: Batch receiving reduces database round-trips
2. Better throughput: Workflows can process multiple signals in one wake cycle

Potential Concerns

1. Memory usage: Batching could increase memory pressure if many large signals are fetched at once. Consider documenting recommended limit values.

Migration & Compatibility

Breaking Changes

1. Listen trait signature changed (affects custom implementations)
2. CustomListener trait removed
3. Database schema evolved (new EventType::Signals = 10)

Questions

1. Database migration: Does the new EventType::Signals require a migration for existing workflows?
2. Backward compatibility: Can old workflow history still be replayed?

Recommendations Summary

Must Fix (High Priority)

1. Add comprehensive test coverage for batch signal functionality
2. Document the breaking changes for Listen trait implementers
3. Clarify database migration requirements and backward compatibility

Should Fix (Medium Priority)

4. Improve error message in listen() method for empty vec case
5. Document removal of CustomListener trait
6. Add doc comments explaining the limit parameter and recommended values
7. Verify ordering guarantees in indexed signal storage

Conclusion

This is a solid architectural improvement that enables important workflow batching capabilities. The implementation is generally well-structured, but needs test coverage, documentation of breaking changes, and clarification on database migration/compatibility.

The code quality is good and follows the existing patterns in the codebase.

Overall Assessment: ⚠️ Approve with Changes Required

Please address the test coverage and breaking change documentation before merging.

[claude]

Code Review: feat(gas): add batch listen

Overview

This PR adds batch listening capability to the Gasoline workflow engine, allowing workflows to receive multiple signals in a single operation. This is a significant enhancement that improves efficiency for workflows that need to process multiple signals.

✅ Strengths

1. Well-designed API changes: The refactoring from single signal to batch signals is clean and maintains backward compatibility through the listen() method which now wraps listen_n(1).

2. Proper backward compatibility: The migration from deprecated EventType::Signal to EventType::Signals is handled gracefully in db/kv/mod.rs:2898-2910, converting old single-signal events to the new batch format.

3. Good data structures: The new SignalsEvent struct uses parallel vectors (names, bodies, signal_ids) which is a reasonable approach for storing batch data.

4. Comprehensive history tracking: The indexed keys (IndexedSignalIdKey, IndexedNameKey, IndexedInputKey) properly track multiple signals per event.

⚠️ Issues & Concerns

1. Critical: Potential data inconsistency in SignalsEvent

Location: engine/packages/gasoline/src/history/event.rs:211-214

The SignalsEvent struct has three parallel vectors but no invariant enforcement:

pub struct SignalsEvent {
    pub names: Vec<String>,
    pub bodies: Vec<Box<serde_json::value::RawValue>>,
}

Problem: There's a missing signal_ids field in the public struct definition, but the debug code references it (db/kv/debug.rs:1542). This suggests either:

• Missing field in the struct definition
• Or the vectors can become desynchronized (different lengths)

Recommendation:

• Add signal_ids: Vec<Id> to the struct
• Add validation ensuring all three vectors have the same length
• Consider using a Vec<Signal> with a proper struct instead of parallel vectors to prevent desynchronization

2. Bug: Incorrect EventType in history insertion

Location: engine/packages/gasoline/src/db/kv/keys/history.rs:1645

pub fn signals_event(
    // ...
) -> Result<()> {
    common(
        subspace,
        tx,
        workflow_id,
        location,
        EventType::Signal,  // ⚠️ Should be EventType::Signals
        version,
        create_ts,
    )
}

Problem: Using deprecated EventType::Signal instead of EventType::Signals for new events. While backward compatibility handles reading old events, new events should use the correct type.

Impact: Could cause issues with event type matching and future migrations.

Fix: Change line 1645 to EventType::Signals

3. Performance: Unbounded buffer in signal processing

Location: engine/packages/gasoline/src/db/kv/mod.rs:2052

.buffer_unordered(1024)

Problem: Hard-coded buffer size of 1024 for parallel signal processing. If limit is less than 1024, this allocates unnecessary resources. If limit is greater, it could cause memory pressure.

Recommendation: Use buffer_unordered(limit.min(1024)) or make it configurable.

4. Logic issue: Sorting after limiting

Location: engine/packages/gasoline/src/db/kv/mod.rs:2056-2060

// Sort by ts
signals.sort_by_key(|key| key.create_ts);

// Apply limit
Ok(signals.into_iter().take(limit).collect())

Problem: The limit is already applied at line 1950 when fetching from the database:

limit: Some(limit),

Then signals are processed with buffer_unordered which can reorder them, sorted by timestamp, and then limited again. This double-limiting with sorting in between might not give you the oldest limit signals across all signal types.

Scenario: If you have 100 signals each of types A and B, and request limit=50, you might get 50 from A and 50 from B (total 100), sort them, then take the first 50 by timestamp. This might not be the intended behavior.

Recommendation: Clarify the intended semantics and potentially fetch with a higher limit, then sort and take.

5. Missing test coverage

Location: No new test files for batch functionality

Problem: The existing test file tests/workflows/signal_test.rs only tests single signal reception. There are no tests for:

• Multiple signals received in one batch
• Mixed signal types
• Batch limits
• listen_n_with_timeout functionality
• Edge cases (empty results, limit=0, etc.)

Recommendation: Add comprehensive tests for the new batch listening features.

📝 Minor Issues

6. Inconsistent naming

• last_try → last_attempt is a good rename, but check if this is used consistently everywhere

7. Documentation

• The new listen_n and listen_n_with_timeout methods lack doc comments
• The limit parameter in the Listen trait should document behavior when limit=0

8. Removed CustomListener without deprecation notice

Location: engine/packages/gasoline/src/ctx/versioned_workflow.rs:125-136 (deleted)

The custom_listener method was removed entirely. If this was part of the public API, consider:

• Adding a deprecation notice in the changelog
• Checking if any code depends on this

🔒 Security

No security concerns identified. The changes maintain proper transaction boundaries and don't introduce obvious vulnerabilities.

🎯 Recommendations Summary

Must Fix:

1. Fix EventType::Signal → EventType::Signals in history insertion (bug)
2. Add signal_ids field to SignalsEvent or clarify the design
3. Add validation for parallel vector consistency

Should Fix:

1. Clarify and fix the sorting/limiting logic for signals
2. Add comprehensive test coverage
3. Adjust buffer_unordered size based on limit
4. Add documentation for new public methods

Nice to Have:

1. Consider refactoring parallel vectors to a proper struct
2. Document breaking changes if CustomListener was public API

Overall Assessment

This is a solid feature addition with good architectural decisions. The main concerns are around data consistency guarantees and the correctness of the event type in new insertions. Once the critical issues are addressed, this will be a valuable enhancement to the workflow engine.

The backward compatibility handling is particularly well done, and the overall code quality follows the project conventions.

[claude]

Pull Request Review: feat(gas): add batch listen

Overview

This PR adds batch listening functionality to the Gasoline workflow engine, allowing workflows to receive multiple signals in a single operation.

Code Quality

Strengths: Well-structured refactoring with consistent naming conventions

Minor Issues:

1. Signal ordering may not preserve strict temporal ordering across different signal types (line 1938-1969)
2. Duplicate name deduplication code in two places

Performance Considerations

1. Buffer size hardcoded to 1024 (line 2052)
2. Fetches up to N*limit signals but only uses limit after sorting (line 2060)
3. Metrics updated individually per signal

Potential Bugs

1. CRITICAL: Index mismatch after sorting - signals enumerated at line 1985, used for history at 2033, but sorted at 2057
2. CRITICAL: listen() may error on empty vec instead of returning None
3. Missing documentation updates

Test Coverage

Missing tests for listen_n, signal ordering, batch history replay, and edge cases

Breaking Changes

• Listen trait changed from Self to Vec
• CustomListener trait removed

Summary

Must Fix: Two critical bugs with index ordering and error handling
Should Fix: Add tests and document breaking changes

This is a solid feature but needs critical bugs fixed before merge.

---

Review generated by Claude Code