feat(realtime): Realtime streams v2 #2632
Conversation
🦋 Changeset detectedLatest commit: 211d804 The changes in this PR will be included in the next version bump. This PR includes changesets to release 24 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
WalkthroughAdds a realtime streams subsystem spanning backend, core, SDK, CLI, and frontend. Introduces DB migrations and TaskRun fields Estimated code review effort🎯 5 (Critical) | ⏱️ ~150 minutes
Pre-merge checks and finishing touches❌ Failed checks (2 warnings)
✅ Passed checks (1 passed)
✨ Finishing touches
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 27
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (8)
apps/webapp/app/models/organization.server.ts (2)
55-69: Remove unusedfeaturesvariable.Since
v3Enabledis now hardcoded totrue, thefeaturesvariable computed on line 55 is no longer used and should be removed.Apply this diff to remove the dead code:
- const features = featuresForUrl(new URL(env.APP_ORIGIN)); - const organization = await prisma.organization.create({ data: { title, slug: uniqueOrgSlug, companySize, maximumConcurrencyLimit: env.DEFAULT_ORG_EXECUTION_CONCURRENCY_LIMIT, members: { create: { userId: userId, role: "ADMIN", }, }, v3Enabled: true, },
30-30: Useenvexport instead of directprocess.envaccess.This line directly accesses
process.env.BLOCKED_USERS, which violates the coding guideline for this file path.As per coding guidelines
Update
env.server.tsto includeBLOCKED_USERSif needed, then access it via theenvexport:- if (typeof process.env.BLOCKED_USERS === "string" && process.env.BLOCKED_USERS.includes(userId)) { + if (typeof env.BLOCKED_USERS === "string" && env.BLOCKED_USERS.includes(userId)) {packages/trigger-sdk/src/v3/index.ts (1)
19-23: Fix type re-export ordering for Context (compile error risk).
export type { Context };appears before theimport type { Context } ..., which can fail. Prefer a direct re-export.Apply this diff:
-export type { Context }; - -import type { Context } from "./shared.js"; +export type { Context } from "./shared.js";The new
export * from "./streams.js";is good.packages/core/src/v3/runMetadata/manager.ts (1)
318-328: Early return may break when value is a ReadableStream. Convert to AsyncIterable.You cast value to AsyncIterable, but ReadableStream isn’t AsyncIterable. Wrap it to avoid runtime issues when runId is undefined.
Apply:
- const $value = value as AsyncIterable<T>; - - if (!this.runId) { - return $value; - } + const toAsyncIterable = (input: AsyncIterable<T> | ReadableStream<T>): AsyncIterable<T> => { + if (Symbol.asyncIterator in (input as any)) { + return input as AsyncIterable<T>; + } + const stream = input as ReadableStream<T>; + return { + async *[Symbol.asyncIterator]() { + const reader = stream.getReader(); + try { + while (true) { + const { done, value } = await reader.read(); + if (done) return; + yield value as T; + } + } finally { + try { + reader.releaseLock(); + } catch {} + } + }, + }; + }; + + if (!this.runId) { + return toAsyncIterable(value); + }If you already have a shared utility for this conversion, prefer importing it instead.
apps/webapp/app/services/realtime/redisRealtimeStreams.server.ts (1)
276-313: Ingest must split by newline to preserve chunk boundaries and correct chunkIndexreader.read() returns arbitrary string segments, not line-delimited chunks. Writing each read as one Redis entry corrupts chunk boundaries and breaks resume (chunkIndex). Accumulate and split on '\n', increment chunkIndex per line.
- const textStream = stream.pipeThrough(new TextDecoderStream()); - const reader = textStream.getReader(); - - while (true) { - const { done, value } = await reader.read(); - - if (done || !value) { - break; - } - - // Write each chunk with its index and clientId - this.logger.debug("[RedisRealtimeStreams][ingestData] Writing chunk", { - streamKey, - runId, - clientId, - chunkIndex: currentChunkIndex, - resumeFromChunk: startChunk, - value, - }); - - await redis.xadd( - streamKey, - "MAXLEN", - "~", - String(env.REALTIME_STREAM_MAX_LENGTH), - "*", - "clientId", - clientId, - "chunkIndex", - currentChunkIndex.toString(), - "data", - value - ); - - currentChunkIndex++; - } + const textStream = stream.pipeThrough(new TextDecoderStream()); + const reader = textStream.getReader(); + let buffer = ""; + + while (true) { + const { done, value } = await reader.read(); + if (done) break; + if (value) buffer += value; + + const lines = buffer.split("\n"); + buffer = lines.pop() ?? ""; + + for (const line of lines) { + const trimmed = line.trim(); + if (trimmed.length === 0) continue; + + this.logger.debug("[RedisRealtimeStreams][ingestData] Writing line", { + streamKey, + runId, + clientId, + chunkIndex: currentChunkIndex, + resumeFromChunk: startChunk, + }); + + await redis.xadd( + streamKey, + "MAXLEN", + "~", + String(env.REALTIME_STREAM_MAX_LENGTH), + "*", + "clientId", + clientId, + "chunkIndex", + currentChunkIndex.toString(), + "data", + trimmed + ); + + currentChunkIndex++; + } + } + + // Flush any trailing data without newline + if (buffer.trim().length > 0) { + await redis.xadd( + streamKey, + "MAXLEN", + "~", + String(env.REALTIME_STREAM_MAX_LENGTH), + "*", + "clientId", + clientId, + "chunkIndex", + currentChunkIndex.toString(), + "data", + buffer + ); + currentChunkIndex++; + }packages/react-hooks/src/hooks/useRealtime.ts (3)
86-98: Fix SWR keying and reset completion when runId changes (prevents mixed/stale state).
- Keys for run/error/complete omit runId, so switching runId within the same hook instance can mix states and suppress onComplete for subsequent runs.
- Also reset
hasCalledOnCompleteRefandcompletewhen runId changes.Apply:
- const { data: run, mutate: mutateRun } = useSWR<RealtimeRun<TTask>>([idKey, "run"], null); + const { data: run, mutate: mutateRun } = useSWR<RealtimeRun<TTask>>([idKey, runId, "run"], null); - const { data: error = undefined, mutate: setError } = useSWR<undefined | Error>( - [idKey, "error"], + const { data: error = undefined, mutate: setError } = useSWR<undefined | Error>( + [idKey, runId, "error"], null ); - const { data: isComplete = false, mutate: setIsComplete } = useSWR<boolean>( - [idKey, "complete"], + const { data: isComplete = false, mutate: setIsComplete } = useSWR<boolean>( + [idKey, runId, "complete"], null );And add a small reset effect:
const hasCalledOnCompleteRef = useRef(false); + useEffect(() => { + hasCalledOnCompleteRef.current = false; + setIsComplete(false); + setError(undefined); + }, [runId]);Also applies to: 147-155
232-258: Isolate per-run stream caches and reset on run change.
streams,run,error, andcompleteSWR keys in useRealtimeRunWithStreams omit runId → cross-run mixing.- Ensure new run starts with fresh stream buffers and completion state.
- const { data: streams, mutate: mutateStreams } = useSWR<StreamResults<TStreams>>( - [idKey, "streams"], + const { data: streams, mutate: mutateStreams } = useSWR<StreamResults<TStreams>>( + [idKey, runId, "streams"], null, { fallbackData: initialStreamsFallback } ); - const { data: run, mutate: mutateRun } = useSWR<RealtimeRun<TTask>>([idKey, "run"], null); + const { data: run, mutate: mutateRun } = useSWR<RealtimeRun<TTask>>([idKey, runId, "run"], null); - const { data: isComplete = false, mutate: setIsComplete } = useSWR<boolean>( - [idKey, "complete"], + const { data: isComplete = false, mutate: setIsComplete } = useSWR<boolean>( + [idKey, runId, "complete"], null ); - const { data: error = undefined, mutate: setError } = useSWR<undefined | Error>( - [idKey, "error"], + const { data: error = undefined, mutate: setError } = useSWR<undefined | Error>( + [idKey, runId, "error"], null );Reset when
runIdchanges:const hasCalledOnCompleteRef = useRef(false); + useEffect(() => { + hasCalledOnCompleteRef.current = false; + setIsComplete(false); + // fresh buffer for new run + mutateStreams({} as StreamResults<TStreams>); + }, [runId]);Also applies to: 311-319
408-423: Key SWR caches by tag/batch to avoid cross-subscription bleed.
- Keys for runs/error in useRealtimeRunsWithTag and useRealtimeBatch omit identifiers (tag/createdAt, batchId).
- This can merge different subscriptions into one cache.
@@ useRealtimeRunsWithTag - const { data: runs, mutate: mutateRuns } = useSWR<RealtimeRun<TTask>[]>([idKey, "run"], null, { + const tagKey = Array.isArray(tag) ? [...tag].sort().join("|") : tag; + const { data: runs, mutate: mutateRuns } = useSWR<RealtimeRun<TTask>[]>([idKey, "runs", tagKey, options?.createdAt], null, { fallbackData: [], }); - const { data: error = undefined, mutate: setError } = useSWR<undefined | Error>( - [idKey, "error"], + const { data: error = undefined, mutate: setError } = useSWR<undefined | Error>( + [idKey, "error", tagKey, options?.createdAt], null ); @@ useRealtimeBatch - const { data: runs, mutate: mutateRuns } = useSWR<RealtimeRun<TTask>[]>([idKey, "run"], null, { + const { data: runs, mutate: mutateRuns } = useSWR<RealtimeRun<TTask>[]>([idKey, "runs", batchId], null, { fallbackData: [], }); - const { data: error = undefined, mutate: setError } = useSWR<undefined | Error>( - [idKey, "error"], + const { data: error = undefined, mutate: setError } = useSWR<undefined | Error>( + [idKey, "error", batchId], null );Also applies to: 506-520
🧹 Nitpick comments (41)
apps/webapp/app/assets/icons/SnakedArrowIcon.tsx (1)
1-20: Extract props type for consistency.The component is well-structured and follows React best practices for icon components. Consider extracting the props definition to an explicit type to align with project conventions of using explicit types over inline definitions.
-export function SnakedArrowIcon({ className }: { className?: string }) { +type SnakedArrowIconProps = { className?: string }; + +export function SnakedArrowIcon({ className }: SnakedArrowIconProps) {apps/webapp/app/assets/icons/MoveToBottomIcon.tsx (1)
1-27: Consider aligning structure with MoveToTopIcon for consistency.The implementation is clean and follows the coding guidelines. However,
MoveToTopIconincludes a<g clipPath>wrapper and<defs>section that are absent here. While the clipPath appears unnecessary (it defines a 24×24 rectangle matching the viewBox, likely an SVG export artifact), the structural inconsistency between paired components could cause confusion.Consider either:
- Adding the clipPath/defs to match
MoveToTopIconfor consistency, or- Removing them from
MoveToTopIconto match this cleaner approach (preferred if they serve no functional purpose).docker/config/nginx.conf (1)
2-2: Consider adding worker_connections for completeness.While an empty
eventsblock is valid, for a more production-representative config, consider addingworker_connections 1024;or similar.-events {} +events { + worker_connections 1024; +}packages/core/src/v3/schemas/common.ts (2)
342-342: Consider constraining realtimeStreamsVersion to valid values.The field accepts any string, but the summary indicates specific values like "v1" (default) and "v2" are used to coordinate between Redis-based and S2-based streaming backends. Using a literal union would prevent invalid values and improve type safety.
Apply this diff to constrain the field to known versions:
- realtimeStreamsVersion: z.string().optional(), + realtimeStreamsVersion: z.enum(["v1", "v2"]).optional(),If additional versions are anticipated, consider defining a shared schema constant like
RealtimeStreamsVersionthat can be reused across the codebase.
342-342: Add documentation for realtimeStreamsVersion.The field lacks a JSDoc comment explaining its purpose or valid values. Documenting this would help developers understand when and how to set this field.
Apply this diff to add documentation:
z.object({ traceContext: z.record(z.unknown()).optional(), + /** Specifies the realtime streaming infrastructure version. Defaults to "v1" (Redis-based). Use "v2" for S2-based streaming. */ realtimeStreamsVersion: z.string().optional(), })packages/trigger-sdk/src/v3/streams.ts (2)
113-155: Harden readStream(): always close span; fix falsy startIndex check.
- If
apiClient.fetchStream()rejects before invoking callbacks, the span won’t end.startIndexuses a falsy check, so0is ignored.Apply this diff:
- return await apiClient.fetchStream(runId, key, { - signal: options?.signal, - timeoutInSeconds: options?.timeoutInSeconds ?? 60, - lastEventId: options?.startIndex ? (options.startIndex - 1).toString() : undefined, - onComplete: () => { - span.end(); - }, - onError: (error) => { - span.recordException(error); - span.setStatus({ code: SpanStatusCode.ERROR }); - span.end(); - }, - }); + try { + return await apiClient.fetchStream(runId, key, { + signal: options?.signal, + timeoutInSeconds: options?.timeoutInSeconds ?? 60, + lastEventId: + options?.startIndex != null ? (options.startIndex - 1).toString() : undefined, + onComplete: () => { + span.end(); + }, + onError: (error) => { + span.recordException(error); + span.setStatus({ code: SpanStatusCode.ERROR }); + span.end(); + }, + }); + } catch (error) { + span.recordException(error as any); + span.setStatus({ code: SpanStatusCode.ERROR }); + span.end(); + throw error; + }
1-13: Type‑only import for AsyncIterableStream.Import
AsyncIterableStreamas a type to avoid bundling it at runtime.- AsyncIterableStream, + type AsyncIterableStream,packages/trigger-sdk/src/v3/shared.ts (1)
1299-1308: Align public wrapper requestOptions types with internals (API parity).Internals accept
TriggerApiRequestOptions(withclientConfig), but public wrappers still typerequestOptionsasApiRequestOptions:
triggerAndWait(...)(Lines 433-455)batchTriggerAndWait(...)(Lines 481-492)This prevents callers from passing
clientConfigto those wrappers.Apply these diffs:
-export function triggerAndWait<TTask extends AnyTask>( +export function triggerAndWait<TTask extends AnyTask>( id: TaskIdentifier<TTask>, payload: TaskPayload<TTask>, options?: TriggerAndWaitOptions, - requestOptions?: ApiRequestOptions + requestOptions?: TriggerApiRequestOptions ): TaskRunPromise<TaskIdentifier<TTask>, TaskOutput<TTask>> {export async function batchTriggerAndWait<TTask extends AnyTask>( id: TaskIdentifier<TTask>, items: Array<BatchItem<TaskPayload<TTask>>>, options?: BatchTriggerAndWaitOptions, - requestOptions?: ApiRequestOptions + requestOptions?: TriggerApiRequestOptions ): Promise<BatchResult<TaskIdentifier<TTask>, TaskOutput<TTask>>> {Would you like a quick repo script to list any remaining exports that still use
ApiRequestOptionswhereclientConfigshould be supported?Also applies to: 1372-1380
packages/core/src/v3/runMetadata/types.ts (1)
33-35: Make StreamInstance generic and iterable (aligns with usage).Today it only guarantees
wait(). Most call sites will also need to iterate. Recommend composing the iterable shape and making it generic.Apply:
-export interface StreamInstance { - wait(): Promise<void>; -} +export interface StreamInstance<T = unknown> extends AsyncIterable<T> { + wait(): Promise<void>; +}Follow-up: update implementers to
implements StreamInstance<T>.apps/webapp/app/env.server.ts (2)
201-201: Guard against zero/negative timeout.Use a minimum to avoid disabled/negative timeouts from misconfiguration.
-REALTIME_STREAMS_INACTIVITY_TIMEOUT_MS: z.coerce.number().int().default(60000), // 1 minute +REALTIME_STREAMS_INACTIVITY_TIMEOUT_MS: z.coerce.number().int().min(1).default(60000), // 1 minute
1206-1215: S2 config: add basic numeric mins and a refine to ensure required vars when v2 is active.
- Prevent accidental zero values; optionally enforce presence of BASIN/ACCESS_TOKEN when REALTIME_STREAM_VERSION === "v2".
-REALTIME_STREAMS_S2_FLUSH_INTERVAL_MS: z.coerce.number().int().default(100), -REALTIME_STREAMS_S2_MAX_RETRIES: z.coerce.number().int().default(10), -REALTIME_STREAMS_S2_WAIT_SECONDS: z.coerce.number().int().default(60), -WAIT_UNTIL_TIMEOUT_MS: z.coerce.number().int().default(600_000), +REALTIME_STREAMS_S2_FLUSH_INTERVAL_MS: z.coerce.number().int().min(1).default(100), +REALTIME_STREAMS_S2_MAX_RETRIES: z.coerce.number().int().min(0).default(10), +REALTIME_STREAMS_S2_WAIT_SECONDS: z.coerce.number().int().min(1).default(60), +WAIT_UNTIL_TIMEOUT_MS: z.coerce.number().int().min(1).default(600_000),Option (near the schema end): add a cross-field check
.superRefine((v, ctx) => { if (v.REALTIME_STREAM_VERSION === "v2") { if (!v.REALTIME_STREAMS_S2_BASIN) ctx.addIssue({ code: z.ZodIssueCode.custom, path:["REALTIME_STREAMS_S2_BASIN"], message:"required when REALTIME_STREAM_VERSION=v2" }); if (!v.REALTIME_STREAMS_S2_ACCESS_TOKEN) ctx.addIssue({ code: z.ZodIssueCode.custom, path:["REALTIME_STREAMS_S2_ACCESS_TOKEN"], message:"required when REALTIME_STREAM_VERSION=v2" }); } })packages/core/src/v3/runMetadata/s2MetadataStream.ts (3)
56-56: Type conformance: implement the generic StreamInstance.If
StreamInstancebecomes generic and iterable, update the implements clause.-export class S2MetadataStream<T = any> implements StreamInstance { +export class S2MetadataStream<T = any> implements StreamInstance<T> {
72-73: Use isomorphic timer type (avoid NodeJS.Timeout).Prevents TS friction in isomorphic builds.
- private flushInterval: NodeJS.Timeout | null = null; + private flushInterval: ReturnType<typeof setInterval> | null = null;
175-218: Retry scope and recursion.
retryCountis a class field shared across flushes; recursion re-entersflush()and adds more promises. Prefer local attempt counting with a loop to avoid cross-flush interference and deep recursion. Also consider a maxBatchSize to bound memory before flush.Would you like me to draft a non-recursive, per-flush retry loop and an optional
maxBatchSizeoption?apps/webapp/app/services/realtime/types.ts (1)
3-7: All implementers and call sites have been updated; style preference (type over interface) remains valid.Breaking changes verified:
S2RealtimeStreamsandRedisRealtimeStreamsboth implement updated signatures withclientId, optionalresumeFromChunk, and newgetLastChunkIndexmethod- Routes and tests use
streamResponse(request, runId, streamId, signal, options)correctly- SSE
lastEventIdmapping verified: "Last-Event-ID" header properly used across packages/core and implemented in s2realtimeStreams and redisRealtimeStreamsStyle: Per guidelines, convert to
typealiases:-export interface StreamIngestor { +export type StreamIngestor = { initializeStream(runId: string, streamId: string): Promise<{ responseHeaders?: Record<string, string> }>; ingestData(stream: ReadableStream<Uint8Array>, runId: string, streamId: string, clientId: string, resumeFromChunk?: number): Promise<Response>; getLastChunkIndex(runId: string, streamId: string, clientId: string): Promise<number>; -} +} -export interface StreamResponder { +export type StreamResponder = { streamResponse(request: Request, runId: string, streamId: string, signal: AbortSignal, options?: StreamResponseOptions): Promise<Response>; -} +}apps/webapp/app/services/realtime/s2realtimeStreams.server.ts (2)
4-4: Prefer isomorphic Web Crypto API over Node.js crypto module.The coding guidelines specify preferring isomorphic code over Node.js specific imports. Replace
randomUUIDfromnode:cryptowith the Web Crypto API'scrypto.randomUUID()which works in both Node.js and browser environments.Apply this diff:
-import { randomUUID } from "node:crypto"; +// Use Web Crypto API (isomorphic)Then update line 73:
- const id = randomUUID(); + const id = crypto.randomUUID();Based on coding guidelines.
200-237: Remove unused method or clarify its purpose.The
s2ReadOncemethod is defined but never called within this file. If it's intended for future use or external consumption, consider adding a comment explaining its purpose. Otherwise, remove it to avoid code clutter.Additionally, line 213 contains a citation artifact
":contentReference[oaicite:9]{index=9}"that should be removed from the comment.If the method is not needed, apply this diff:
- private async s2ReadOnce( - stream: string, - opts: { - seq_num?: number; - timestamp?: number; - tail_offset?: number; - clamp?: boolean; - count?: number; - bytes?: number; - until?: number; - wait?: number; - } - ): Promise<S2ReadResponse> { - // GET /v1/streams/{stream}/records?... (supports wait= for long-poll; linearizable reads). :contentReference[oaicite:9]{index=9} - const qs = new URLSearchParams(); - if (opts.seq_num != null) qs.set("seq_num", String(opts.seq_num)); - if (opts.timestamp != null) qs.set("timestamp", String(opts.timestamp)); - if (opts.tail_offset != null) qs.set("tail_offset", String(opts.tail_offset)); - if (opts.clamp != null) qs.set("clamp", String(opts.clamp)); - if (opts.count != null) qs.set("count", String(opts.count)); - if (opts.bytes != null) qs.set("bytes", String(opts.bytes)); - if (opts.until != null) qs.set("until", String(opts.until)); - if (opts.wait != null) qs.set("wait", String(opts.wait)); - - const res = await fetch(`${this.baseUrl}/streams/${encodeURIComponent(stream)}/records?${qs}`, { - method: "GET", - headers: { - Authorization: `Bearer ${this.token}`, - Accept: "application/json", - "S2-Format": "raw", - }, - }); - if (!res.ok) { - const text = await res.text().catch(() => ""); - throw new Error(`S2 read failed: ${res.status} ${res.statusText} ${text}`); - } - return (await res.json()) as S2ReadResponse; - }Or if keeping it, update the comment on line 213:
- // GET /v1/streams/{stream}/records?... (supports wait= for long-poll; linearizable reads). :contentReference[oaicite:9]{index=9} + // GET /v1/streams/{stream}/records?... (supports wait= for long-poll; linearizable reads)internal-packages/run-engine/src/engine/index.ts (1)
339-396: Plumbed realtimeStreamsVersion looks good; consider narrowing the type.The propagation into TriggerParams and TaskRun.create is correct. To prevent typos at compile time, narrow TriggerParams.realtimeStreamsVersion to a string union (e.g., "v1" | "v2") and let Prisma’s default "v1" apply when undefined.
Also applies to: 471-476
packages/cli-v3/src/entryPoints/dev-run-worker.ts (1)
144-151: Realtime manager initialization looks correct.Base URL selection and debug flag wiring are fine (the final ?? false is redundant but harmless).
- (getEnvVar("TRIGGER_STREAMS_DEBUG") === "1" || getEnvVar("TRIGGER_STREAMS_DEBUG") === "true") ?? - false + ["1", "true"].includes((getEnvVar("TRIGGER_STREAMS_DEBUG") ?? "").toLowerCase())packages/core/src/v3/apiClientManager/index.ts (1)
101-113: Update auth error copy to mention TRIGGER_ACCESS_TOKENMessages still reference TRIGGER_SECRET_KEY only. Recommend including TRIGGER_ACCESS_TOKEN (preferred) to reduce support churn. Example: “Set TRIGGER_API_URL and TRIGGER_ACCESS_TOKEN (or TRIGGER_SECRET_KEY).”
apps/webapp/app/v3/eventRepository/clickhouseEventRepository.server.ts (1)
425-445: Stricter type guards for entityId/entityMetadata; tolerate missing idCurrent code casts entityId/entityMetadata without verifying type; mapping block also requires entityId to be string, dropping metadata otherwise.
Apply:
@@ - ): { entityType: string; entityId?: string; entityMetadata?: string } | undefined { + ): { entityType: string; entityId?: string; entityMetadata?: string } | undefined { @@ - const entityType = attributes[SemanticInternalAttributes.ENTITY_TYPE]; - const entityId = attributes[SemanticInternalAttributes.ENTITY_ID]; - const entityMetadata = attributes[SemanticInternalAttributes.ENTITY_METADATA]; + const entityType = attributes[SemanticInternalAttributes.ENTITY_TYPE]; + const rawEntityId = attributes[SemanticInternalAttributes.ENTITY_ID]; + const rawEntityMetadata = attributes[SemanticInternalAttributes.ENTITY_METADATA]; @@ - return { - entityType, - entityId: entityId as string | undefined, - entityMetadata: entityMetadata as string | undefined, - }; + return { + entityType, + entityId: typeof rawEntityId === "string" ? rawEntityId : undefined, + entityMetadata: typeof rawEntityMetadata === "string" ? rawEntityMetadata : undefined, + };And in span detail mapping, allow entity without id (keep metadata if provided):
@@ - if ( + if ( parsedMetadata && "entity" in parsedMetadata && typeof parsedMetadata.entity === "object" && parsedMetadata.entity && "entityType" in parsedMetadata.entity && - typeof parsedMetadata.entity.entityType === "string" && - "entityId" in parsedMetadata.entity && - typeof parsedMetadata.entity.entityId === "string" + typeof parsedMetadata.entity.entityType === "string" ) { span.entity = { - id: parsedMetadata.entity.entityId, + id: + "entityId" in parsedMetadata.entity && + typeof parsedMetadata.entity.entityId === "string" + ? parsedMetadata.entity.entityId + : undefined, type: parsedMetadata.entity.entityType, metadata: "entityMetadata" in parsedMetadata.entity && parsedMetadata.entity.entityMetadata && typeof parsedMetadata.entity.entityMetadata === "string" ? parsedMetadata.entity.entityMetadata : undefined, }; }This improves resilience without changing persisted shapes. [Based on learnings]
Also applies to: 1109-1110, 1149-1155
apps/webapp/app/routes/resources.orgs.$organizationSlug.projects.$projectParam.env.$envParam.runs.$runParam.spans.$spanParam/route.tsx (1)
83-84: Avoid importing components from route modules; move RealtimeStreamViewer to shared componentImporting from another route binds route bundles together and risks subtle loader side effects. Extract RealtimeStreamViewer into a shared component (e.g., components/runs/RealtimeStreamViewer.tsx) and import from there.
Example:
- import { RealtimeStreamViewer } from "../resources.orgs.$organizationSlug.projects.$projectParam.env.$envParam.runs.$runParam.streams.$streamKey/route"; + import { RealtimeStreamViewer } from "~/components/runs/RealtimeStreamViewer";No behavior change; improves bundling and maintainability. [As per coding guidelines]
Also applies to: 1147-1154
apps/webapp/app/routes/api.v1.tasks.$taskId.trigger.ts (1)
36-37: Threading looks good; consider stricter schema/coercion for versionPropagation to logs and TriggerTaskService is correct.
To reduce invalid inputs, consider constraining/normalizing the header:
- "x-trigger-realtime-streams-version": z.string().nullish(), + "x-trigger-realtime-streams-version": z + .enum(["1", "2"]) // or ["v1","v2"] + .optional() + .nullable(),Or coerce/trim:
- const { ..., "x-trigger-realtime-streams-version": realtimeStreamsVersion } = headers; + const { ..., "x-trigger-realtime-streams-version": realtimeStreamsVersionRaw } = headers; + const realtimeStreamsVersion = + typeof realtimeStreamsVersionRaw === "string" + ? realtimeStreamsVersionRaw.trim() + : undefined;Also applies to: 67-68, 114-114, 129-130
apps/webapp/test/redisRealtimeStreams.test.ts (1)
235-261: Deduplicate SSE parsing in tests with a small helper.Multiple tests reimplement the same parsing; extract to a shared helper for clarity and maintenance.
Example helper (place at top of file or a local test util):
async function readSseEvents( stream: ReadableStream<Uint8Array>, maxEvents: number ): Promise<Array<{ id?: string; data?: string }>> { const reader = stream.getReader(); const decoder = new TextDecoder(); const out: Array<{ id?: string; data?: string }> = []; try { while (out.length < maxEvents) { const { value, done } = await reader.read(); if (done) break; if (!value) continue; const text = decoder.decode(value); for (const evt of text.split("\n\n").filter(Boolean)) { const entry: { id?: string; data?: string } = {}; for (const line of evt.split("\n")) { if (line.startsWith("id: ")) entry.id = line.slice(4).trim(); if (line.startsWith("data: ")) entry.data = line.slice(6).trim(); } if (entry.id || entry.data) out.push(entry); } } } finally { reader.releaseLock(); } return out; }Also applies to: 457-481, 1109-1126, 1199-1231, 1309-1331, 1379-1401, 1411-1414
apps/webapp/app/routes/realtime.v1.streams.$runId.$target.$streamId.ts (1)
57-87: Avoid duplicate streamId entries when creating streams.Using push can add duplicates if PUT is retried. Prefer idempotency: check presence or use a write that guarantees uniqueness.
One approach:
- const updatedRun = await prisma.taskRun.update({ + const updatedRun = await prisma.taskRun.update({ where: { friendlyId: targetId, runtimeEnvironmentId: authentication.environment.id, }, - data: { - realtimeStreams: { - push: params.streamId, - }, - }, + data: { + realtimeStreams: { + set: prisma.raw`ARRAY(SELECT DISTINCT UNNEST(realtime_streams || ${params.streamId}))`, + }, + },Or read-modify-write with a unique Set in application code. Please choose the variant consistent with your DB/provider.
packages/core/src/v3/runMetadata/manager.ts (1)
305-307: Provide a clearer migration error message.Make it actionable for callers adopting the new API.
- throw new Error("This needs to use the new realtime streams API"); + throw new Error( + "fetchStream has been removed. Use realtimeStreams.append(key, source, { target?, signal? }) and consume the returned .stream instead." + );packages/core/src/v3/realtimeStreams/manager.ts (1)
21-26: Map key may collide across runsUsing key alone risks collisions if multiple streams with the same key are appended within a run or future reuse. Consider
${runId}:${key}to avoid accidental overwrites.packages/core/test/metadataStream.test.ts (1)
561-607: Reduce flakiness in backoff-based timing assertionsThe elapsed >= ~5s assertion can be flaky under load. Prefer asserting retry count and order; if timing is required, widen tolerance or control timers with vi.useFakeTimers().
apps/webapp/app/services/realtime/redisRealtimeStreams.server.ts (1)
16-21: Simplify types and remove unused variantStreamChunk defines a "legacy-data" variant that is never enqueued. Either emit it where applicable or remove the variant to reduce cognitive load.
Also applies to: 205-235
packages/cli-v3/src/entryPoints/managed-run-worker.ts (2)
312-314: Pass the same wait‑until timeout into waitForAllStreams()Avoids having two different timeouts (manager default 60s vs TRIGGER_WAIT_UNTIL_TIMEOUT_MS).
-const waitUntilTimeoutInMs = getNumberEnvVar("TRIGGER_WAIT_UNTIL_TIMEOUT_MS", 60_000); +const waitUntilTimeoutInMs = getNumberEnvVar("TRIGGER_WAIT_UNTIL_TIMEOUT_MS", 60_000); const waitUntilManager = new StandardWaitUntilManager(waitUntilTimeoutInMs); @@ waitUntil.register({ - requiresResolving: () => standardRealtimeStreamsManager.hasActiveStreams(), - promise: () => standardRealtimeStreamsManager.waitForAllStreams(), + requiresResolving: () => standardRealtimeStreamsManager.hasActiveStreams(), + promise: () => standardRealtimeStreamsManager.waitForAllStreams(waitUntilTimeoutInMs), });Also applies to: 143-145
135-141: Tighten TRIGGER_STREAMS_DEBUG parsingThe nullish coalescing after a boolean OR is redundant. Slightly clearer parse:
- (getEnvVar("TRIGGER_STREAMS_DEBUG") === "1" || getEnvVar("TRIGGER_STREAMS_DEBUG") === "true") ?? - false + ["1", "true"].includes((getEnvVar("TRIGGER_STREAMS_DEBUG") ?? "").toLowerCase())packages/core/src/v3/runMetadata/metadataStream.ts (2)
211-215: Use a one‑shot abort listener to prevent listener accumulationMultiple retries can add multiple listeners.
- if (this.options.signal) { - this.options.signal.addEventListener("abort", () => { - req.destroy(new Error("Request aborted")); - }); - } + if (this.options.signal) { + this.options.signal.addEventListener( + "abort", + () => req.destroy(new Error("Request aborted")), + { once: true } + ); + }
316-322: Minor cleanups: unused helpers/fields and guideline alignment
- getChunksFromBuffer is unused; remove or wire it up.
- currentChunkIndex appears unused externally.
- Consider consolidating node:http/https to isomorphic fetch for consistency with guidelines.
Would you like me to submit a follow‑up PR that:
- removes dead code (getChunksFromBuffer/currentChunkIndex),
- switches to fetch() with AbortSignal and streaming body?
Also applies to: 337-349, 27-27, 33-33
packages/core/src/v3/apiClient/runStream.ts (1)
501-521: Handle pipeTo rejections and aborts to avoid unhandled promise rejectionsThe background pipeline isn’t awaited; errors will surface as unhandled rejections. Also, writes may race after outer controller closes.
- subscription.subscribe().then((stream) => { - stream + subscription.subscribe().then((stream) => { + const writable = new WritableStream({ + write(chunk) { + // Guard against late writes after close/abort + if (!controller.desiredSize && this.options?.abortController?.signal?.aborted) { + return; + } + controller.enqueue(chunk); + }, + }); + stream .pipeThrough( new TransformStream({ transform(chunk, controller) { controller.enqueue({ type: streamKey, chunk: chunk.chunk as TStreams[typeof streamKey], run, }); }, }) ) - .pipeTo( - new WritableStream({ - write(chunk) { - controller.enqueue(chunk); - }, - }) - ); - }); + .pipeTo(writable) + .catch((err) => { + // Suppress if aborting; otherwise log + if (!this.options.abortController.signal.aborted) { + console.warn("Stream pipeline error:", err); + } + }); + }).catch(() => {});apps/webapp/app/routes/resources.orgs.$organizationSlug.projects.$projectParam.env.$envParam.runs.$runParam.streams.$streamKey/route.tsx (3)
363-371: Use a stable key for list itemsIndex keys cause unnecessary re‑renders and bugs on reconnection. Prefer the SSE id.
- {chunks.map((chunk, index) => ( + {chunks.map((chunk, index) => ( <StreamChunkLine - key={index} + key={chunk.id ?? index} chunk={chunk} lineNumber={firstLineNumber + index} maxLineNumberWidth={maxLineNumberWidth} /> ))}
131-142: Clipboard write should be awaited and errors handledClipboard can fail (permissions, insecure context). Provide feedback.
- const onCopied = useCallback( + const onCopied = useCallback( (event: React.MouseEvent<HTMLButtonElement>) => { event.preventDefault(); event.stopPropagation(); - navigator.clipboard.writeText(getCompactText()); - setCopied(true); - setTimeout(() => { - setCopied(false); - }, 1500); + (async () => { + try { + await navigator.clipboard.writeText(getCompactText()); + setCopied(true); + setTimeout(() => setCopied(false), 1500); + } catch (e) { + console.error("Clipboard error", e); + } + })(); }, [getCompactText] );
447-502: Optional: deduplicate chunks by id to handle reconnect replaysReconnects may replay overlapping events. Avoid duplicates.
useEffect(() => { const abortController = new AbortController(); - let reader: ReadableStreamDefaultReader<SSEStreamPart<unknown>> | null = null; + let reader: ReadableStreamDefaultReader<SSEStreamPart<unknown>> | null = null; + const seen = new Set<string>(); @@ - if (value !== undefined) { - setChunks((prev) => [ - ...prev, - { - id: value.id, - data: value.chunk, - timestamp: value.timestamp, - }, - ]); - } + if (value !== undefined && !seen.has(value.id)) { + seen.add(value.id); + setChunks((prev) => [ + ...prev, + { id: value.id, data: value.chunk, timestamp: value.timestamp }, + ]); + }packages/react-hooks/src/hooks/useRealtime.ts (4)
860-882: Avoid race conditions in throttled flushes by updating refs alongside mutate.
- Current code builds next state from
existing*Ref.currentbut doesn’t update the ref, relying on SWR to propagate. Under tight flushes, this can cause lost or duplicated chunks.@@ processRealtimeRunWithStreams onFlush - // Apply all updates + // Apply all updates for (const [type, chunks] of Object.entries(updatesByType)) { - // @ts-ignore - nextStreamData[type] = [...(existingDataRef.current[type] || []), ...chunks]; + // @ts-expect-error type key narrowing + nextStreamData[type] = [...(existingDataRef.current[type] || []), ...chunks]; } - - mutateStreamData(nextStreamData); + // keep ref in sync to avoid re-reading stale buffers on next flush + existingDataRef.current = nextStreamData; + mutateStreamData(nextStreamData);@@ processRealtimeStream throttle flush - const streamQueue = createThrottledQueue<TPart>(async (parts) => { - mutatePartsData([...existingPartsRef.current, ...parts]); - }, throttleInMs); + const streamQueue = createThrottledQueue<TPart>(async (parts) => { + const next = [...existingPartsRef.current, ...parts]; + existingPartsRef.current = next; + mutatePartsData(next); + }, throttleInMs);Also applies to: 939-941
127-128: Minor cleanup: simplify defaulting and drop no-op.finally.
- Use nullish coalescing for
stopOnCompletion.- Replace
.finally(() => {})withvoid triggerRequest().- typeof options?.stopOnCompletion === "boolean" ? options.stopOnCompletion : true + options?.stopOnCompletion ?? true- triggerRequest().finally(() => {}); + void triggerRequest();Also applies to: 290-293, 166-171, 330-335, 474-479, 571-576, 746-751
855-858: Tighten types to remove@ts-ignorearound stream updates.
- Constrain the stream key to a string and type chunks by key; casts become localized and safer.
- type StreamUpdate = { - type: keyof TStreams; - chunk: any; - }; + type StreamUpdate = { + type: keyof TStreams & string; + chunk: TStreams[keyof TStreams]; + }; @@ - const updatesByType = updates.reduce( - (acc, update) => { + const updatesByType = updates.reduce( + (acc: Partial<Record<keyof TStreams, any[]>>, update) => { if (!acc[update.type]) { acc[update.type] = []; } acc[update.type].push(update.chunk); return acc; }, - {} as Record<keyof TStreams, any[]> + {} as Partial<Record<keyof TStreams, any[]>> ); @@ - for (const [type, chunks] of Object.entries(updatesByType)) { - // @ts-ignore - nextStreamData[type] = [...(existingDataRef.current[type] || []), ...chunks]; + for (const [type, chunks] of Object.entries(updatesByType) as Array<[keyof TStreams & string, any[]]>) { + nextStreamData[type] = [ + ...(((existingDataRef.current as any)[type] as any[]) || []), + ...chunks, + ] as any; } @@ - streamQueue.add({ - type: part.type, - // @ts-ignore - chunk: part.chunk, - }); + streamQueue.add({ type: part.type as keyof TStreams & string, chunk: part.chunk as any });Also applies to: 876-883, 888-893
15-24: Avoid duplicatingenabledin options.
UseRealtimeRunOptionsextendsUseApiClientOptionswhich already includesenabled?. Remove the duplicate to prevent API ambiguity.export type UseRealtimeRunOptions = UseApiClientOptions & { id?: string; - enabled?: boolean; /** * The number of milliseconds to throttle the stream updates. * * @default 16 */ throttleInMs?: number; };As per coding guidelines.
...izationSlug.projects.$projectParam.env.$envParam.runs.$runParam.streams.$streamKey/route.tsx
Show resolved
Hide resolved
...izationSlug.projects.$projectParam.env.$envParam.runs.$runParam.streams.$streamKey/route.tsx
Show resolved
Hide resolved
...izationSlug.projects.$projectParam.env.$envParam.runs.$runParam.streams.$streamKey/route.tsx
Show resolved
Hide resolved
There was a problem hiding this comment.
Actionable comments posted: 7
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
packages/cli-v3/src/entryPoints/managed-run-worker.ts (1)
600-608: Cancel should drain/abort active realtime streams.Currently CANCEL only aborts the run and flushes usage/tracing/metadata. Active streams may keep running. Explicitly wait (and abort on timeout) to avoid leaks.
Apply:
CANCEL: async ({ timeoutInMs }) => { _isCancelled = true; _cancelController.abort("run cancelled"); await callCancelHooks(timeoutInMs); + // Ensure realtime streams finish or get aborted before flushing + try { + await standardRealtimeStreamsManager.waitForAllStreams( + typeof timeoutInMs === "number" ? timeoutInMs : waitUntilTimeoutInMs + ); + } catch (err) { + console.warn("Failed waiting for realtime streams on cancel", err); + } if (_executionMeasurement) { usage.stop(_executionMeasurement); } await flushAll(timeoutInMs); },
♻️ Duplicate comments (4)
packages/core/test/streamsWriterV1.test.ts (1)
792-805: Fix generator-without-yield lint in empty stream testKeep semantics but satisfy the linter with a dead-code yield.
it("should handle empty stream (no chunks)", async () => { async function* generateChunks() { - // Yields nothing - return; + // Yields nothing; keep generator shape for linters + if (false) { + // never executed; placates useYield rule + yield undefined as never; + } }packages/core/src/v3/realtimeStreams/streamsWriterV1.ts (1)
237-266: Propagate buffer task failures to the request loopCheck captured buffer error and fail fast instead of looping indefinitely.
const processStream = async () => { try { let lastSentIndex = startFromChunk - 1; while (true) { + // Surface background errors + if ((this as any)._bufferError) { + req.destroy(); + throw (this as any)._bufferError; + } // Send all chunks that are available in buffer while (lastSentIndex < this.highestBufferedIndex) { lastSentIndex++; const chunk = this.ringBuffer.find((c) => c.index === lastSentIndex); if (chunk) { const stringified = JSON.stringify(chunk.data) + "\n"; req.write(stringified); this.currentChunkIndex = lastSentIndex + 1; } } // If stream is complete and we've sent all buffered chunks, we're done if (this.streamComplete && lastSentIndex >= this.highestBufferedIndex) { req.end(); break; } // Wait a bit for more chunks to be buffered await this.delay(10); } } catch (error) { reject(error); } };apps/webapp/test/redisRealtimeStreams.test.ts (1)
1360-1366: Resolved: correct streamResponse options object.Passing
{ lastEventId }matches the signature and avoids type errors. Looks good.apps/webapp/app/services/realtime/v1StreamsGlobal.server.ts (1)
50-51: Resolved: no silent fallback to v1.Throwing on missing S2 config for v2 prevents empty reads and data loss. Good.
🧹 Nitpick comments (20)
packages/core/src/v3/waitUntil/manager.ts (1)
6-6: Constructor implements centralized timeout strategy.The timeout is now stored at the instance level with a sensible default of 60 seconds, aligning with the PR's centralized timeout approach.
Consider adding validation for the timeout parameter to prevent unexpected behavior:
- constructor(private timeoutInMs: number = 60_000) {} + constructor(private timeoutInMs: number = 60_000) { + if (timeoutInMs <= 0) { + throw new Error("timeoutInMs must be a positive number"); + } + }packages/core/src/v3/apiClient/runStream.ts (2)
387-402: Consider URL encoding the path parameters.The
runIdandstreamKeyare inserted directly into the URL path. If these contain special characters, the URL could be malformed.Apply this diff to add URL encoding:
- const url = `${options?.baseUrl ?? this.baseUrl}/realtime/v1/streams/${runId}/${streamKey}`; + const url = `${options?.baseUrl ?? this.baseUrl}/realtime/v1/streams/${encodeURIComponent(runId)}/${encodeURIComponent(streamKey)}`;
742-757: Consider using Zod for metadata stream validation.The type checking for
metadata.$$streamsis verbose and could be simplified with Zod validation, which is heavily used in packages/core per coding guidelines.Define a schema at the top of the file:
import { z } from "zod"; const MetadataStreamsSchema = z.object({ $$streams: z.array(z.string()).min(1), });Then simplify the function:
function getStreamsFromRunShape(run: AnyRunShape): string[] { - const metadataStreams = - run.metadata && - "$$streams" in run.metadata && - Array.isArray(run.metadata.$$streams) && - run.metadata.$$streams.length > 0 && - run.metadata.$$streams.every((stream) => typeof stream === "string") - ? run.metadata.$$streams - : undefined; + if (run.metadata) { + const result = MetadataStreamsSchema.safeParse(run.metadata); + if (result.success) { + return result.data.$$streams; + } + } - if (metadataStreams) { - return metadataStreams; - } - return run.realtimeStreams; }As per coding guidelines.
packages/core/src/v3/streams/asyncIterableStream.ts (1)
106-135: Harden async-iterable detection and lock releaseUse a callable check for Symbol.asyncIterator and guard releaseLock to avoid edge errors. Optional: cancel the reader when iteration stops early.
export function ensureAsyncIterable<T>( input: AsyncIterable<T> | ReadableStream<T> ): AsyncIterable<T> { // If it's already an AsyncIterable, return it as-is - if (Symbol.asyncIterator in input) { + if (typeof (input as any)[Symbol.asyncIterator] === "function") { return input as AsyncIterable<T>; } @@ const reader = readableStream.getReader(); try { while (true) { const { done, value } = await reader.read(); if (done) { break; } if (value !== undefined) { yield value; } } } finally { - reader.releaseLock(); + try { + reader.releaseLock(); + } catch {} + // Optionally: await reader.cancel().catch(() => {}); } }, }; }packages/core/src/v3/realtimeStreams/streamsWriterV2.ts (4)
72-72: Use env-agnostic timer typeAvoid Node-only
NodeJS.Timeoutfor isomorphic compatibility.- private flushInterval: NodeJS.Timeout | null = null; + private flushInterval: ReturnType<typeof setInterval> | null = null;
160-179: Avoid unbounded prefetch; make tee source backpressure-awareReading the entire AsyncIterable in
startcan buffer unboundedly if consumers stall/abort. Drive reads frompullinstead and support cancellation.- private createTeeStreams() { - const readableSource = new ReadableStream<T>({ - start: async (controller) => { - try { - let count = 0; - - for await (const value of this.options.source) { - controller.enqueue(value); - count++; - } - - controller.close(); - } catch (error) { - controller.error(error); - } - }, - }); - return readableSource.tee(); - } + private createTeeStreams() { + const self = this; + const iterator = this.options.source[Symbol.asyncIterator](); + const readableSource = new ReadableStream<T>({ + async pull(controller) { + if (self.aborted) { + controller.close(); + return; + } + const { value, done } = await iterator.next(); + if (done) { + controller.close(); + return; + } + controller.enqueue(value); + }, + async cancel() { + await iterator.return?.().catch(() => {}); + }, + }); + return readableSource.tee(); + }
181-225: Release reader lock after buffering completesPrevent lingering locks by releasing in a finally block.
private startBuffering(): void { @@ - this.bufferReaderTask = (async () => { + this.bufferReaderTask = (async () => { try { let chunkCount = 0; @@ } - } catch (error) { + } catch (error) { this.logError("[S2MetadataStream] Error in buffering task:", error); throw error; - } + } finally { + try { + this.streamReader?.releaseLock(); + } catch {} + this.streamReader = null; + } })();
295-296: Prune resolved flush promises to cap memoryKeep
flushPromisesbounded by removing settled entries.- this.flushPromises.push(flushPromise); + this.flushPromises.push(flushPromise); + flushPromise.finally(() => { + this.flushPromises = this.flushPromises.filter((p) => p !== flushPromise); + });packages/core/src/v3/realtimeStreams/manager.ts (1)
57-63: Combine abort signals without relying on AbortSignal.anyWhen
AbortSignal.anyis unavailable, the current fallback ignores the user signal. Merge signals manually.- const combinedSignal = options?.signal - ? AbortSignal.any?.([options.signal, abortController.signal]) ?? abortController.signal - : abortController.signal; + const combinedSignal = options?.signal + ? mergeAbortSignals([options.signal, abortController.signal]) + : abortController.signal;Add helper near the bottom of the module:
function mergeAbortSignals(signals: AbortSignal[]): AbortSignal { const controller = new AbortController(); const onAbort = () => controller.abort(); for (const s of signals) { if (s.aborted) return AbortSignal.abort(); s.addEventListener("abort", onAbort, { once: true }); } return controller.signal; }packages/core/src/v3/realtimeStreams/streamsWriterV1.ts (2)
331-341: Minor: exponential backoff caps are good; consider logging retries for observabilityOptional: add debug logs on retries/timeouts to aid support.
357-369: Unused helper
getChunksFromBuffer()isn't used. Remove or integrate to avoid dead code.packages/cli-v3/src/entryPoints/managed-run-worker.ts (1)
135-140: Minor: simplify TRIGGER_STREAMS_DEBUG parsing.The
?? falseis redundant andAbortSignal.anysupport varies. Prefer explicit boolean parsing.Apply:
- (getEnvVar("TRIGGER_STREAMS_DEBUG") === "1" || getEnvVar("TRIGGER_STREAMS_DEBUG") === "true") ?? - false + ["1", "true"].includes((getEnvVar("TRIGGER_STREAMS_DEBUG") ?? "").toLowerCase())apps/webapp/test/redisRealtimeStreams.test.ts (2)
240-261: Harden SSE parsing to handle chunk splits.Decoding and
split("\n\n")per read can drop events split across reads. Buffer between reads.Add a tiny helper and reuse:
function createSSECollector() { let buf = ""; return (chunk: Uint8Array, out: (evt: { id?: string; data?: string }) => void) => { buf += new TextDecoder().decode(chunk); const parts = buf.split("\n\n"); buf = parts.pop() ?? ""; for (const evt of parts) { let id: string | undefined; let data: string | undefined; for (const line of evt.split("\n")) { if (line.startsWith("id: ")) id = line.slice(4).trim(); if (line.startsWith("data: ")) data = line.slice(6); } out({ id, data }); } }; }Use per test:
const collect = createSSECollector(); const events: string[] = []; while (events.length < 3) { const { value, done } = await reader.read(); if (done) break; collect(value!, (e) => e.data && events.push(e.data)); }Also applies to: 462-482, 1199-1231, 1374-1401
1141-1146: Reduce timing brittleness in inactivity test.The 4–8s window is tied to an internal BLOCK=5000ms. If that constant changes, this test flakes.
- Assert only that streamClosed is true and at least one BLOCK cycle elapsed, or
- Expose/read block time from implementation and derive expectations dynamically.
apps/webapp/app/services/realtime/v1StreamsGlobal.server.ts (1)
25-52: Memoize S2RealtimeStreams to avoid per-request re-instantiation.Constructing a new client each call adds overhead. Cache by (basin, streamPrefix).
Example:
+const s2Cache = new Map<string, S2RealtimeStreams>(); export function getRealtimeStreamInstance( environment: AuthenticatedEnvironment, streamVersion: string ): StreamIngestor & StreamResponder { if (streamVersion === "v1") { return v1RealtimeStreams; } else { if (env.REALTIME_STREAMS_S2_BASIN && env.REALTIME_STREAMS_S2_ACCESS_TOKEN) { - return new S2RealtimeStreams({ + const streamPrefix = [ + "org", + environment.organization.id, + "env", + environment.slug, + environment.id, + ].join("/"); + const key = `${env.REALTIME_STREAMS_S2_BASIN}|${streamPrefix}`; + const cached = s2Cache.get(key); + if (cached) return cached; + const inst = new S2RealtimeStreams({ basin: env.REALTIME_STREAMS_S2_BASIN, accessToken: env.REALTIME_STREAMS_S2_ACCESS_TOKEN, - streamPrefix: [ - "org", - environment.organization.id, - "env", - environment.slug, - environment.id, - ].join("/"), + streamPrefix, logLevel: env.REALTIME_STREAMS_S2_LOG_LEVEL, flushIntervalMs: env.REALTIME_STREAMS_S2_FLUSH_INTERVAL_MS, maxRetries: env.REALTIME_STREAMS_S2_MAX_RETRIES, s2WaitSeconds: env.REALTIME_STREAMS_S2_WAIT_SECONDS, }); + s2Cache.set(key, inst); + return inst; } throw new Error("Realtime streams v2 is required for this run but S2 configuration is missing"); } }apps/webapp/app/routes/realtime.v1.streams.$runId.$target.$streamId.ts (2)
121-147: Use the run’s configured version for HEAD; avoid header-driven selection.HEAD with v2 will currently throw. Select the instance from the run’s version (and consider returning 501 for v2).
Add version to select and use it:
@@ findResource select: select: { id: true, friendlyId: true, + realtimeStreamsVersion: true, parentTaskRun: { @@ - const clientId = request.headers.get("X-Client-Id") || "default"; - const streamVersion = request.headers.get("X-Stream-Version") || "v1"; - - const realtimeStream = getRealtimeStreamInstance(authentication.environment, streamVersion); + const clientId = request.headers.get("X-Client-Id") || "default"; + const streamVersion = run.realtimeStreamsVersion || "v1"; + if (streamVersion !== "v1") { + return new Response("HEAD last-chunk is only available for v1", { status: 501 }); + } + const realtimeStream = getRealtimeStreamInstance(authentication.environment, "v1");Also applies to: 170-177
57-87: Potential duplication in realtimeStreams array.Using
pushcan add the samestreamIdmultiple times.Prefer a unique set semantics (e.g., read existing array, de-dup, then
set), or enforce uniqueness at the db layer.apps/webapp/app/services/realtime/redisRealtimeStreams.server.ts (3)
245-251: Add SSE-friendly headers (buffering/CORS/version).Improve proxy compatibility and parity with v2.
Apply:
return new Response(stream, { headers: { "Content-Type": "text/event-stream", "Cache-Control": "no-cache", Connection: "keep-alive", + "X-Accel-Buffering": "no", + "Access-Control-Expose-Headers": "*", + "X-Stream-Version": "v1", }, });
237-241: Use structured logger in cleanup.Avoid bare console in server code.
Apply:
- await redis.quit().catch(console.error); + await redis.quit().catch((err) => + (/* this */ self ?? { logger }).logger.error( + "[RedisRealtimeStreams][streamResponse] Error during cleanup", + { err } + ) + );Note: if
selfnot in scope here, captureconst log = this.logger;before and use it.
288-296: Avoid logging raw chunk data.Raw payloads can be large/sensitive. Log size/indices instead.
Apply:
- this.logger.debug("[RedisRealtimeStreams][ingestData] Writing chunk", { + this.logger.debug("[RedisRealtimeStreams][ingestData] Writing chunk", { streamKey, runId, clientId, chunkIndex: currentChunkIndex, resumeFromChunk: startChunk, - value, + size: value.length, });
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (1)
references/realtime-streams/src/trigger/streams.tsis excluded by!references/**
📒 Files selected for processing (17)
apps/webapp/app/routes/realtime.v1.streams.$runId.$target.$streamId.ts(2 hunks)apps/webapp/app/services/realtime/redisRealtimeStreams.server.ts(6 hunks)apps/webapp/app/services/realtime/v1StreamsGlobal.server.ts(2 hunks)apps/webapp/test/redisRealtimeStreams.test.ts(1 hunks)packages/cli-v3/src/entryPoints/dev-run-worker.ts(5 hunks)packages/cli-v3/src/entryPoints/managed-run-worker.ts(5 hunks)packages/core/src/v3/apiClient/runStream.ts(8 hunks)packages/core/src/v3/apiClientManager/index.ts(1 hunks)packages/core/src/v3/realtimeStreams/manager.ts(1 hunks)packages/core/src/v3/realtimeStreams/streamsWriterV1.ts(1 hunks)packages/core/src/v3/realtimeStreams/streamsWriterV2.ts(1 hunks)packages/core/src/v3/realtimeStreams/types.ts(1 hunks)packages/core/src/v3/streams/asyncIterableStream.ts(1 hunks)packages/core/src/v3/waitUntil/manager.ts(2 hunks)packages/core/src/v3/waitUntil/types.ts(1 hunks)packages/core/test/streamsWriterV1.test.ts(1 hunks)packages/trigger-sdk/src/v3/streams.ts(1 hunks)
🚧 Files skipped from review as they are similar to previous changes (4)
- packages/core/src/v3/apiClientManager/index.ts
- packages/core/src/v3/realtimeStreams/types.ts
- packages/trigger-sdk/src/v3/streams.ts
- packages/cli-v3/src/entryPoints/dev-run-worker.ts
🧰 Additional context used
📓 Path-based instructions (8)
**/*.{ts,tsx}
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
**/*.{ts,tsx}: Always prefer using isomorphic code like fetch, ReadableStream, etc. instead of Node.js specific code
For TypeScript, we usually use types over interfaces
Avoid enums
No default exports, use function declarations
Files:
packages/core/src/v3/waitUntil/types.tspackages/cli-v3/src/entryPoints/managed-run-worker.tsapps/webapp/app/services/realtime/v1StreamsGlobal.server.tspackages/core/src/v3/streams/asyncIterableStream.tspackages/core/src/v3/waitUntil/manager.tspackages/core/src/v3/realtimeStreams/streamsWriterV1.tspackages/core/src/v3/realtimeStreams/streamsWriterV2.tspackages/core/src/v3/realtimeStreams/manager.tspackages/core/src/v3/apiClient/runStream.tspackages/core/test/streamsWriterV1.test.tsapps/webapp/test/redisRealtimeStreams.test.tsapps/webapp/app/routes/realtime.v1.streams.$runId.$target.$streamId.tsapps/webapp/app/services/realtime/redisRealtimeStreams.server.ts
{packages/core,apps/webapp}/**/*.{ts,tsx}
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
We use zod a lot in packages/core and in the webapp
Files:
packages/core/src/v3/waitUntil/types.tsapps/webapp/app/services/realtime/v1StreamsGlobal.server.tspackages/core/src/v3/streams/asyncIterableStream.tspackages/core/src/v3/waitUntil/manager.tspackages/core/src/v3/realtimeStreams/streamsWriterV1.tspackages/core/src/v3/realtimeStreams/streamsWriterV2.tspackages/core/src/v3/realtimeStreams/manager.tspackages/core/src/v3/apiClient/runStream.tspackages/core/test/streamsWriterV1.test.tsapps/webapp/test/redisRealtimeStreams.test.tsapps/webapp/app/routes/realtime.v1.streams.$runId.$target.$streamId.tsapps/webapp/app/services/realtime/redisRealtimeStreams.server.ts
apps/webapp/**/*.{ts,tsx}
📄 CodeRabbit inference engine (.cursor/rules/webapp.mdc)
When importing from @trigger.dev/core in the webapp, never import the root package path; always use one of the documented subpath exports from @trigger.dev/core’s package.json
Files:
apps/webapp/app/services/realtime/v1StreamsGlobal.server.tsapps/webapp/test/redisRealtimeStreams.test.tsapps/webapp/app/routes/realtime.v1.streams.$runId.$target.$streamId.tsapps/webapp/app/services/realtime/redisRealtimeStreams.server.ts
{apps/webapp/app/**/*.server.{ts,tsx},apps/webapp/app/routes/**/*.ts}
📄 CodeRabbit inference engine (.cursor/rules/webapp.mdc)
Access environment variables only via the env export from app/env.server.ts; do not reference process.env directly
Files:
apps/webapp/app/services/realtime/v1StreamsGlobal.server.tsapps/webapp/app/routes/realtime.v1.streams.$runId.$target.$streamId.tsapps/webapp/app/services/realtime/redisRealtimeStreams.server.ts
apps/webapp/app/**/*.ts
📄 CodeRabbit inference engine (.cursor/rules/webapp.mdc)
Modules intended for test consumption under apps/webapp/app/**/*.ts must not read environment variables; accept configuration via options instead
Files:
apps/webapp/app/services/realtime/v1StreamsGlobal.server.tsapps/webapp/app/routes/realtime.v1.streams.$runId.$target.$streamId.tsapps/webapp/app/services/realtime/redisRealtimeStreams.server.ts
**/*.test.{ts,tsx}
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
Our tests are all vitest
Files:
packages/core/test/streamsWriterV1.test.tsapps/webapp/test/redisRealtimeStreams.test.ts
**/*.{test,spec}.{ts,tsx,js,jsx}
📄 CodeRabbit inference engine (AGENTS.md)
**/*.{test,spec}.{ts,tsx,js,jsx}: Unit tests must use Vitest
Tests should avoid mocks or stubs and use helpers from @internal/testcontainers when Redis or Postgres are needed
Test files live beside the files under test and should use descriptive describe and it blocks
Files:
packages/core/test/streamsWriterV1.test.tsapps/webapp/test/redisRealtimeStreams.test.ts
{apps/webapp/**/__tests__/**/*.{ts,tsx},apps/webapp/**/*.{test,spec}.{ts,tsx}}
📄 CodeRabbit inference engine (.cursor/rules/webapp.mdc)
{apps/webapp/**/__tests__/**/*.{ts,tsx},apps/webapp/**/*.{test,spec}.{ts,tsx}}: Do not import app/env.server.ts into tests, either directly or indirectly
Tests should only import classes/functions from files under apps/webapp/app/**/*.ts
Files:
apps/webapp/test/redisRealtimeStreams.test.ts
🧬 Code graph analysis (11)
packages/core/src/v3/waitUntil/types.ts (1)
packages/core/src/v3/waitUntil/index.ts (1)
WaitUntilManager(39-41)
packages/cli-v3/src/entryPoints/managed-run-worker.ts (5)
packages/core/src/v3/apiClientManager-api.ts (1)
apiClientManager(5-5)packages/core/src/v3/run-metadata-api.ts (1)
runMetadata(5-5)packages/core/src/v3/realtimeStreams/manager.ts (1)
StandardRealtimeStreamsManager(16-139)packages/core/src/v3/realtime-streams-api.ts (1)
realtimeStreams(5-5)packages/core/src/v3/waitUntil/manager.ts (1)
StandardWaitUntilManager(3-40)
apps/webapp/app/services/realtime/v1StreamsGlobal.server.ts (2)
apps/webapp/app/services/realtime/types.ts (2)
StreamIngestor(2-17)StreamResponder(25-33)apps/webapp/app/services/realtime/s2realtimeStreams.server.ts (1)
S2RealtimeStreams(32-246)
packages/core/src/v3/realtimeStreams/streamsWriterV1.ts (1)
packages/core/src/v3/realtimeStreams/types.ts (1)
StreamsWriter(23-25)
packages/core/src/v3/realtimeStreams/streamsWriterV2.ts (1)
packages/core/src/v3/realtimeStreams/types.ts (1)
StreamsWriter(23-25)
packages/core/src/v3/realtimeStreams/manager.ts (6)
packages/core/src/v3/realtimeStreams/types.ts (3)
RealtimeStreamsManager(10-16)RealtimeAppendStreamOptions(4-8)RealtimeStreamInstance(18-21)packages/core/src/v3/apiClient/index.ts (1)
headers(1202-1213)packages/core/src/v3/streams/asyncIterableStream.ts (3)
ensureAsyncIterable(107-135)AsyncIterableStream(1-1)createAsyncIterableStreamFromAsyncIterable(52-97)packages/core/src/v3/realtimeStreams/streamsWriterV1.ts (1)
StreamsWriterV1(26-466)packages/core/src/v3/realtimeStreams/streamsWriterV2.ts (1)
StreamsWriterV2(56-392)packages/core/src/v3/task-context-api.ts (1)
taskContext(5-5)
packages/core/src/v3/apiClient/runStream.ts (1)
packages/core/src/v3/apiClient/index.ts (4)
SSEStreamPart(146-146)SSEStreamSubscription(136-136)headers(1202-1213)AnyRunShape(139-139)
packages/core/test/streamsWriterV1.test.ts (1)
packages/core/src/v3/realtimeStreams/streamsWriterV1.ts (1)
StreamsWriterV1(26-466)
apps/webapp/test/redisRealtimeStreams.test.ts (2)
internal-packages/testcontainers/src/index.ts (1)
redisTest(167-167)apps/webapp/app/services/realtime/redisRealtimeStreams.server.ts (1)
RedisRealtimeStreams(23-413)
apps/webapp/app/routes/realtime.v1.streams.$runId.$target.$streamId.ts (4)
apps/webapp/app/services/realtime/v1StreamsGlobal.server.ts (1)
getRealtimeStreamInstance(25-52)apps/webapp/app/routes/realtime.v1.streams.$runId.$streamId.ts (1)
loader(11-71)apps/webapp/app/services/routeBuilders/apiBuilder.server.ts (1)
createLoaderApiRoute(97-287)apps/webapp/app/db.server.ts (1)
$replica(103-106)
apps/webapp/app/services/realtime/redisRealtimeStreams.server.ts (3)
packages/core/src/logger.ts (3)
Logger(19-139)LogLevel(15-15)error(69-79)apps/webapp/app/services/realtime/types.ts (3)
StreamIngestor(2-17)StreamResponder(25-33)StreamResponseOptions(19-22)apps/webapp/app/env.server.ts (1)
env(1219-1219)
🪛 Biome (2.1.2)
packages/core/test/streamsWriterV1.test.ts
[error] 792-795: This generator function doesn't contain yield.
(lint/correctness/useYield)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (23)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (1, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (8, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (7, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (7, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (8, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (3, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (5, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (4, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (6, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (4, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (1, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (2, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (2, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (5, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (3, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (6, 8)
- GitHub Check: units / packages / 🧪 Unit Tests: Packages (1, 1)
- GitHub Check: e2e / 🧪 CLI v3 tests (ubuntu-latest - pnpm)
- GitHub Check: e2e / 🧪 CLI v3 tests (windows-latest - pnpm)
- GitHub Check: e2e / 🧪 CLI v3 tests (ubuntu-latest - npm)
- GitHub Check: e2e / 🧪 CLI v3 tests (windows-latest - npm)
- GitHub Check: typecheck / typecheck
- GitHub Check: Analyze (javascript-typescript)
🔇 Additional comments (12)
packages/core/src/v3/waitUntil/types.ts (2)
3-3: Type signature refined for deferred promise timeout.The updated signature now explicitly passes
timeoutInMsto deferred promise functions, making the timeout contract clearer.
8-8: API change verified – all call sites properly updated.All references to
blockUntilSettled()in the codebase confirm the timeout parameter has been removed and call sites updated accordingly. The sole call site atpackages/core/src/v3/workers/taskExecutor.ts:1099correctly invokes the method with no arguments, matching the new interface signature.packages/core/src/v3/waitUntil/manager.ts (1)
16-31: Implementation correctly uses instance-level timeout.The refactored
blockUntilSettledmethod properly uses the stored timeout for both deferred promise invocation (line 22) and the race timeout (line 27). Memory management is sound with promises being cleared after settlement.packages/core/src/v3/apiClient/runStream.ts (8)
1-19: LGTM!The import additions are appropriate for the streaming infrastructure enhancements. All imported modules are actively used in the file.
51-51: LGTM!The
realtimeStreamsfield addition aligns with the PR objectives for Streams 2.0.
156-179: LGTM!The new types and interface changes provide a solid foundation for the enhanced streaming API with resume support and timeout handling.
202-213: LGTM!The subscribe method correctly returns a ReadableStream and properly wires up the connection and cancellation logic.
344-375: LGTM!The retry logic with exponential backoff is well-implemented, properly handles abort signals, and includes sensible max retry limits.
765-765: LGTM!The radix parameter has been correctly added to
parseInt, addressing the previous review comment.
759-771: LGTM!The Redis stream ID parsing is correct with proper fallback handling and NaN checks.
590-590: LGTM!The
realtimeStreamsfield is correctly added with a safe fallback to an empty array.apps/webapp/app/services/realtime/redisRealtimeStreams.server.ts (1)
50-81: LGTM on unified inactivity threshold and pings.Threshold computed once and reused in both branches; periodic pings help keep connections alive.
Also applies to: 162-178
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (2)
packages/core/src/v3/realtimeStreams/manager.ts (2)
76-86: Consider hoisting p-limit import to module or class level.The dynamic import
(await import("p-limit")).defaulton line 82 runs every time a v2 stream is appended. While module caching prevents repeated loads, hoisting the import to module initialization would eliminate the async overhead on each call.Add at the top of the file after imports:
// Lazy-load p-limit once for all v2 streams let pLimitPromise: Promise<typeof import("p-limit").default> | undefined; function getPLimit() { if (!pLimitPromise) { pLimitPromise = import("p-limit").then(m => m.default); } return pLimitPromise; }Then on line 82:
- limiter: (await import("p-limit")).default, + limiter: await getPLimit(),
175-200: Consider parseInt radix and v1 fallback logging.Two minor improvements:
- Lines 197-198: Use explicit radix in
parseIntfor clarity and to prevent unexpected octal/hex interpretation.- Lines 186-188: The silent fallback to v1 when headers are missing could mask server misconfiguration. Consider logging a warning.
Apply this diff:
return { version: "v2", accessToken, basin, - flushIntervalMs: flushIntervalMs ? parseInt(flushIntervalMs) : undefined, - maxRetries: maxRetries ? parseInt(maxRetries) : undefined, + flushIntervalMs: flushIntervalMs ? parseInt(flushIntervalMs, 10) : undefined, + maxRetries: maxRetries ? parseInt(maxRetries, 10) : undefined, };Optionally add logging at line 187:
if (!accessToken || !basin) { + if (version === "v2") { + console.warn("V2 realtime stream headers missing, falling back to v1"); + } return { version: "v1" }; }
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (2)
packages/core/src/v3/realtimeStreams/manager.ts(1 hunks)packages/core/test/runStream.test.ts(10 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
- packages/core/test/runStream.test.ts
🧰 Additional context used
📓 Path-based instructions (2)
**/*.{ts,tsx}
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
**/*.{ts,tsx}: Always prefer using isomorphic code like fetch, ReadableStream, etc. instead of Node.js specific code
For TypeScript, we usually use types over interfaces
Avoid enums
No default exports, use function declarations
Files:
packages/core/src/v3/realtimeStreams/manager.ts
{packages/core,apps/webapp}/**/*.{ts,tsx}
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
We use zod a lot in packages/core and in the webapp
Files:
packages/core/src/v3/realtimeStreams/manager.ts
🧬 Code graph analysis (1)
packages/core/src/v3/realtimeStreams/manager.ts (6)
packages/core/src/v3/realtimeStreams/types.ts (3)
RealtimeStreamsManager(10-16)RealtimeAppendStreamOptions(4-8)RealtimeStreamInstance(18-21)packages/core/src/v3/apiClient/index.ts (1)
headers(1202-1213)packages/core/src/v3/streams/asyncIterableStream.ts (3)
ensureAsyncIterable(107-135)AsyncIterableStream(1-1)createAsyncIterableStreamFromAsyncIterable(52-97)packages/core/src/v3/realtimeStreams/streamsWriterV1.ts (1)
StreamsWriterV1(26-466)packages/core/src/v3/realtimeStreams/streamsWriterV2.ts (1)
StreamsWriterV2(56-392)packages/core/src/v3/task-context-api.ts (1)
taskContext(5-5)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (23)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (7, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (6, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (2, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (6, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (8, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (5, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (3, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (7, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (5, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (1, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (8, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (3, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (4, 8)
- GitHub Check: e2e / 🧪 CLI v3 tests (windows-latest - pnpm)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (2, 8)
- GitHub Check: e2e / 🧪 CLI v3 tests (windows-latest - npm)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (4, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (1, 8)
- GitHub Check: e2e / 🧪 CLI v3 tests (ubuntu-latest - npm)
- GitHub Check: e2e / 🧪 CLI v3 tests (ubuntu-latest - pnpm)
- GitHub Check: units / packages / 🧪 Unit Tests: Packages (1, 1)
- GitHub Check: typecheck / typecheck
- GitHub Check: Analyze (javascript-typescript)
🔇 Additional comments (6)
packages/core/src/v3/realtimeStreams/manager.ts (6)
1-14: LGTM: Imports are clean and necessary.All imports are used in the implementation.
16-26: LGTM: Set-based tracking prevents stream collisions.The switch from Map to Set correctly addresses the previous key collision issue, allowing multiple concurrent streams with the same key.
28-30: LGTM: Simple state management methods.Both
reset()andhasActiveStreams()are implemented correctly.Also applies to: 103-105
32-101: LGTM: append() logic is well-structured.The method correctly:
- Normalizes input sources (line 38) ✓
- Resolves target run IDs with clear error messages (lines 40-46) ✓
- Forwards server tuning parameters to v2 streams (lines 84-85) ✓
- Tracks streams with proper cleanup (lines 88-93) ✓
Signal handling and dynamic import issues are addressed in separate comments.
107-140: LGTM: waitForAllStreams() correctly handles timeout and propagates errors.The implementation properly:
- Uses
Promise.allso stream rejections propagate immediately- Detects timeout via sentinel pattern
- Aborts active streams on timeout
- Provides clear error messages
This correctly addresses the previous review feedback.
143-161: LGTM: getRunIdForOptions() handles all target resolution cases.The target resolution logic correctly handles special values ("parent", "root", "self"), custom run IDs, and the default current run fallback.
ericallam
changed the title
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (1)
packages/core/src/v3/apiClient/index.ts (1)
1110-1132: Consider adding explicit return type for clarity.The
createStreammethod implementation is correct and follows the expected RESTful pattern. The method collects response headers which is useful for accessing stream metadata.For improved code clarity and maintainability, consider adding an explicit return type:
async createStream( runId: string, target: string, streamId: string, requestOptions?: ZodFetchOptions - ) { + ): Promise<CreateStreamResponseBody & { headers: Record<string, string> }> { return zodfetch(
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
packages/core/src/v3/apiClient/index.ts(9 hunks)
🧰 Additional context used
📓 Path-based instructions (2)
**/*.{ts,tsx}
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
**/*.{ts,tsx}: Always prefer using isomorphic code like fetch, ReadableStream, etc. instead of Node.js specific code
For TypeScript, we usually use types over interfaces
Avoid enums
No default exports, use function declarations
Files:
packages/core/src/v3/apiClient/index.ts
{packages/core,apps/webapp}/**/*.{ts,tsx}
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
We use zod a lot in packages/core and in the webapp
Files:
packages/core/src/v3/apiClient/index.ts
🧬 Code graph analysis (1)
packages/core/src/v3/apiClient/index.ts (5)
packages/core/src/v3/apiClientManager/types.ts (1)
ApiClientConfiguration(3-19)packages/core/src/v3/streams/asyncIterableStream.ts (1)
AsyncIterableStream(1-1)packages/core/src/v3/apiClient/runStream.ts (2)
SSEStreamSubscriptionFactory(378-403)SSEStreamPart(175-179)packages/core/src/v3/schemas/api.ts (2)
CreateStreamResponseBody(1301-1303)CreateStreamResponseBody(1304-1304)packages/core/src/v3/utils/getEnv.ts (1)
getEnvVar(11-13)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (23)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (6, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (7, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (3, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (5, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (1, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (5, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (2, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (8, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (4, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (8, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (7, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (3, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (2, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (1, 8)
- GitHub Check: units / packages / 🧪 Unit Tests: Packages (1, 1)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (6, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (4, 8)
- GitHub Check: e2e / 🧪 CLI v3 tests (windows-latest - npm)
- GitHub Check: e2e / 🧪 CLI v3 tests (ubuntu-latest - pnpm)
- GitHub Check: e2e / 🧪 CLI v3 tests (ubuntu-latest - npm)
- GitHub Check: e2e / 🧪 CLI v3 tests (windows-latest - pnpm)
- GitHub Check: typecheck / typecheck
- GitHub Check: Analyze (javascript-typescript)
🔇 Additional comments (5)
packages/core/src/v3/apiClient/index.ts (5)
17-17: LGTM: New imports and exports support streams v2 functionality.The imports for
CreateStreamResponseBody,SSEStreamSubscription,SSEStreamPart,ApiClientConfiguration, andgetEnvVarare all appropriately used in the new streaming features. Re-exportingSSEStreamSubscriptionandSSEStreamPartproperly exposes these types to API client consumers.Also applies to: 73-73, 77-77, 89-90, 137-137, 147-147
133-135: LGTM: Future flags implementation is backward compatible.The
ApiClientFutureFlagstype and its integration into theApiClientconstructor is well-designed. The optional parameter with a default empty object ensures backward compatibility, and theunstable_prefix clearly communicates the experimental nature of the v2 realtime streams feature.Also applies to: 159-159, 166-167, 173-173
1078-1108: LGTM: Stream transformation logic is correct with expected type limitations.The
fetchStreamenhancements properly wire through new options (timeoutInSeconds,onComplete,onError,lastEventId) to the subscription factory. The transformation fromSSEStreamParttoTextracts thechunkfield as expected. Note that sinceSSEStreamPart.chunkis typed asunknown, this relies on the caller to ensure type correctness at runtime, which is acceptable for a generic streaming API.
120-120: No action required - theclientConfigfield is properly used.The
clientConfigfield is consumed byapiClientManager.clientOrThrow()which accepts it as an optional configuration parameter. The method uses it to overridebaseURLandaccessTokensettings. Multiple callsites inpackages/trigger-sdk/src/v3/shared.tspassrequestOptions?.clientConfigto this method, and it's used by end-user code (e.g., inreferences/realtime-streams/src/app/actions.ts). The field has a clear, intentional purpose for configuration management.
1196-1204: Clarify the intent of multiple environment variable names for the v2 realtime streams feature.The code supports four environment variable patterns (
TRIGGER_V2_REALTIME_STREAMSandTRIGGER_REALTIME_STREAMS_V2, each with"1"or"true"values) alongside a programmatic flag. These appear intentional for an experimental feature (prefixedunstable_), but the multiple patterns suggest either:
- A migration path that should be documented
- Configuration flexibility that could be simplified
Consolidate to a single environment variable name unless active migration support or multiple deployment scenarios require the flexibility.
ericallam
force-pushed
the
feat/realtime-streams-2
branch
from
ed1f590 to
0a80d52
Compare
There was a problem hiding this comment.
Actionable comments posted: 4
♻️ Duplicate comments (10)
docker/docker-compose.yml (1)
157-166: Add networks configuration to nginx-h2 service.The
nginx-h2service is missing thenetworksconfiguration needed to communicate with other services onapp_network. Without this, nginx cannot reach the upstream application and the proxy will fail. (This issue was flagged in a previous review and remains unresolved.)Apply this diff:
nginx-h2: image: nginx:1.27 container_name: nginx-h2 restart: unless-stopped ports: - "8443:8443" volumes: - ./config/nginx.conf:/etc/nginx/nginx.conf:ro - ./config/certs:/etc/nginx/certs:ro + networks: + - app_networkpackages/trigger-sdk/src/v3/streams.ts (1)
76-81: AbortError branch leaks span; end before rethrow.The AbortError handling throws without calling
span.end(), leaving a dangling span.Apply this diff:
} catch (error) { // if the error is a signal abort error, we need to end the span but not record an exception if (error instanceof Error && error.name === "AbortError") { + span.end(); throw error; }packages/react-hooks/src/hooks/useRealtime.ts (1)
935-935: Fix falsystartIndexhandling.Explicitly passing
startIndex={0}should start from the beginning, but the current truthy check treats 0 as “unset”, so we never send the requiredlastEventId. Please switch to an explicit null/undefined check before subtracting.- lastEventId: startIndex ? (startIndex - 1).toString() : undefined, + lastEventId: startIndex != null ? String(startIndex - 1) : undefined,packages/core/src/v3/apiClient/runStream.ts (2)
283-285: Validate batch data structure with Zod.The type assertion remains unsafe and could cause runtime errors if the response format doesn't match expectations. As per coding guidelines, Zod validation should be used.
Apply this diff to add safe validation:
+import { z } from "zod"; + +const BatchDataSchema = z.object({ + records: z.array( + z.object({ + body: z.string(), + seq_num: z.number(), + timestamp: z.number(), + }) + ), +});Then update the transform:
} else { if (chunk.event === "batch") { - const data = safeParseJSON(chunk.data) as { - records: Array<{ body: string; seq_num: number; timestamp: number }>; - }; + const parsed = safeParseJSON(chunk.data); + const result = BatchDataSchema.safeParse(parsed); + + if (!result.success) { + console.error("Invalid batch data format:", result.error); + return; + } + + const data = result.data; for (const record of data.records) {As per coding guidelines
501-521: Add error handling for stream subscription and processing.The promise chain lacks error handling. If
subscription.subscribe()rejects or stream processing fails, it will cause an unhandled promise rejection.Apply this diff to add error handling:
// Start stream processing in the background subscription.subscribe().then((stream) => { stream .pipeThrough( new TransformStream({ transform(chunk, controller) { controller.enqueue({ type: streamKey, chunk: chunk.chunk as TStreams[typeof streamKey], run, }); }, }) ) .pipeTo( new WritableStream({ write(chunk) { controller.enqueue(chunk); }, }) - ); - }); + ) + .catch((error) => { + console.error(`Error processing stream ${streamKey}:`, error); + }); + }).catch((error) => { + console.error(`Error subscribing to stream ${streamKey}:`, error); + activeStreams.delete(streamKey); + });apps/webapp/app/routes/realtime.v1.streams.$runId.$streamId.ts (1)
46-59: Fix Timeout-Seconds parsing to prevent NaN bypass.The truthy check on line 47 allows
NaNto slip through validation. WhenparseIntreturnsNaN, it's falsy, so the subsequentisNaNcheck on line 49 is skipped, andtimeoutInSecondsbecomesundefined. However, invalid inputs like non-numeric strings should be rejected with a 400 error.Apply this fix:
- const timeoutInSecondsRaw = request.headers.get("Timeout-Seconds") ?? undefined; - const timeoutInSeconds = timeoutInSecondsRaw ? parseInt(timeoutInSecondsRaw) : undefined; - - if (timeoutInSeconds && isNaN(timeoutInSeconds)) { - return new Response("Invalid timeout seconds", { status: 400 }); - } - - if (timeoutInSeconds && timeoutInSeconds < 1) { - return new Response("Timeout seconds must be greater than 0", { status: 400 }); - } - - if (timeoutInSeconds && timeoutInSeconds > 600) { - return new Response("Timeout seconds must be less than 600", { status: 400 }); - } + const timeoutInSecondsRaw = request.headers.get("Timeout-Seconds"); + let timeoutInSeconds: number | undefined; + if (timeoutInSecondsRaw !== null) { + const n = Number(timeoutInSecondsRaw.trim()); + if (!Number.isFinite(n)) { + return new Response("Invalid timeout seconds", { status: 400 }); + } + if (n < 1) { + return new Response("Timeout seconds must be greater than 0", { status: 400 }); + } + if (n > 600) { + return new Response("Timeout seconds must be less than 600", { status: 400 }); + } + timeoutInSeconds = n; + }apps/webapp/app/routes/realtime.v1.streams.$runId.$target.$streamId.ts (2)
96-106: Strengthen X-Resume-From-Chunk validation to reject fractional values.The current validation checks
isNaN(parsed) || parsed < 0, but doesn't verify the value is an integer. Fractional values like "5.7" will pass and could cause issues downstream.Apply this fix:
const resumeFromChunk = request.headers.get("X-Resume-From-Chunk"); let resumeFromChunkNumber: number | undefined = undefined; if (resumeFromChunk) { const parsed = parseInt(resumeFromChunk, 10); - if (isNaN(parsed) || parsed < 0) { + if (!Number.isInteger(parsed) || parsed < 0) { return new Response(`Invalid X-Resume-From-Chunk header value: ${resumeFromChunk}`, { status: 400, }); } resumeFromChunkNumber = parsed; }
88-116: Enforce v1-only for server-side ingest to prevent runtime errors.The action path accepts
X-Stream-Versionfrom the client (line 90) and uses it to select the stream instance (line 108). WhenstreamVersionis "v2",S2RealtimeStreams.ingestDatawill throw "S2 streams are written to S2 via the client, not from the server". It's better to fail fast with a clear 400 error.Apply this fix:
// Extract client ID from header, default to "default" if not provided const clientId = request.headers.get("X-Client-Id") || "default"; - const streamVersion = request.headers.get("X-Stream-Version") || "v1"; + const streamVersion = (request.headers.get("X-Stream-Version") || "v1").toLowerCase(); + + if (streamVersion !== "v1") { + return new Response("Server-side ingest is only supported for v1 streams", { status: 400 }); + } if (!request.body) { return new Response("No body provided", { status: 400 }); } const resumeFromChunk = request.headers.get("X-Resume-From-Chunk"); let resumeFromChunkNumber: number | undefined = undefined; if (resumeFromChunk) { const parsed = parseInt(resumeFromChunk, 10); if (!Number.isInteger(parsed) || parsed < 0) { return new Response(`Invalid X-Resume-From-Chunk header value: ${resumeFromChunk}`, { status: 400, }); } resumeFromChunkNumber = parsed; } - const realtimeStream = getRealtimeStreamInstance(authentication.environment, streamVersion); + const realtimeStream = getRealtimeStreamInstance(authentication.environment, "v1");packages/core/src/v3/realtimeStreams/types.ts (1)
18-21: Fix invalid accessor declarationTypeScript interfaces can't declare accessors; this getter makes the file fail to compile. Expose
streamas a readonly property instead so the public surface stays the same.Apply this diff:
@@ export interface RealtimeStreamInstance<T> { wait(): Promise<void>; - get stream(): AsyncIterableStream<T>; + readonly stream: AsyncIterableStream<T>; }packages/core/src/v3/realtimeStreams/manager.ts (1)
60-62: Restore caller abort support when AbortSignal.any is unavailable.If the runtime doesn’t ship
AbortSignal.any(still common in older browsers and long-term-support Node builds), this fallback drops the caller’s signal entirely, so their cancellation never reaches the stream upload. Please reintroduce the manual combiner we discussed so both signals propagate regardless of platform support. (developer.mozilla.org)- const combinedSignal = options?.signal - ? AbortSignal.any?.([options.signal, abortController.signal]) ?? abortController.signal - : abortController.signal; + let combinedSignal: AbortSignal = abortController.signal; + if (options?.signal) { + if (typeof AbortSignal.any === "function") { + combinedSignal = AbortSignal.any([options.signal, abortController.signal]); + } else { + const combo = new AbortController(); + const relay = (reason?: any) => combo.abort(reason); + options.signal.addEventListener("abort", () => relay(options.signal!.reason), { + once: true, + }); + abortController.signal.addEventListener( + "abort", + () => relay(abortController.signal.reason), + { once: true } + ); + combinedSignal = combo.signal; + } + }
🧹 Nitpick comments (4)
apps/webapp/app/assets/icons/StreamsIcon.tsx (2)
1-1: Broaden props to standard SVG props + spread them.Expose full SVG attributes (onClick, style, data-*, etc.) and keep named export. This also aligns with TS best practices in the webapp.
Apply:
+import type { ComponentProps } from "react"; + -export function StreamsIcon({ className }: { className?: string }) { +type StreamsIconProps = Omit<ComponentProps<"svg">, "viewBox" | "xmlns"> & { + title?: string; +}; + +export function StreamsIcon({ className, title, ...props }: StreamsIconProps) { return ( -<svg className={className} viewBox="0 0 24 24" fill="none" xmlns="http://www.w3.org/2000/svg"> +<svg + className={className} + viewBox="0 0 24 24" + fill="none" + role="img" + aria-hidden={title ? undefined : true} + aria-label={title} + focusable="false" + {...props} +>
3-7: Optional a11y polish: use <title> with aria-labelledby.If you prefer explicit labelling over aria-label, add a <title> and id wiring.
Example (to layer on top of previous diff):
-<svg +const titleId = title ? "streams-icon-title" : undefined; +return ( +<svg ... - aria-label={title} + aria-labelledby={titleId} > + {title ? <title id={titleId}>{title}</title> : null} <path ... /> <path ... /> <path ... /> </svg> +)apps/webapp/app/services/realtime/v1StreamsGlobal.server.ts (1)
33-47: Consider memoizing S2RealtimeStreams instances to reduce overhead.A new
S2RealtimeStreamsinstance is created on every call whenstreamVersion !== "v1". Since the instance holds connection state and configuration, recreating it repeatedly is inefficient.Memoize instances keyed by
(basin, streamPrefix):const s2InstanceCache = new Map<string, S2RealtimeStreams>(); export function getRealtimeStreamInstance( environment: AuthenticatedEnvironment, streamVersion: string ): StreamIngestor & StreamResponder { if (streamVersion === "v1") { return v1RealtimeStreams; } else { if (env.REALTIME_STREAMS_S2_BASIN && env.REALTIME_STREAMS_S2_ACCESS_TOKEN) { const streamPrefix = [ "org", environment.organization.id, "env", environment.slug, environment.id, ].join("/"); const cacheKey = `${env.REALTIME_STREAMS_S2_BASIN}:${streamPrefix}`; if (!s2InstanceCache.has(cacheKey)) { s2InstanceCache.set(cacheKey, new S2RealtimeStreams({ basin: env.REALTIME_STREAMS_S2_BASIN, accessToken: env.REALTIME_STREAMS_S2_ACCESS_TOKEN, streamPrefix, logLevel: env.REALTIME_STREAMS_S2_LOG_LEVEL, flushIntervalMs: env.REALTIME_STREAMS_S2_FLUSH_INTERVAL_MS, maxRetries: env.REALTIME_STREAMS_S2_MAX_RETRIES, s2WaitSeconds: env.REALTIME_STREAMS_S2_WAIT_SECONDS, })); } return s2InstanceCache.get(cacheKey)!; } throw new Error("Realtime streams v2 is required for this run but S2 configuration is missing"); } }apps/webapp/app/routes/resources.orgs.$organizationSlug.projects.$projectParam.env.$envParam.runs.$runParam.spans.$spanParam/route.tsx (1)
1146-1154: Reset RealtimeStreamViewer when stream changesSwitching between span entities that are both
realtime-streamwill reuse the sameRealtimeStreamViewerinstance, so the prior stream’s chunks, view mode, and scroll state linger until the hook tears down. Keying the component byrunId/streamKeygives us a guaranteed clean slate whenever the selection changes.- return ( - <RealtimeStreamViewer - runId={span.entity.object.runId} - streamKey={span.entity.object.streamKey} - metadata={span.entity.object.metadata} - /> - ); + return ( + <RealtimeStreamViewer + key={`${span.entity.object.runId}:${span.entity.object.streamKey}`} + runId={span.entity.object.runId} + streamKey={span.entity.object.streamKey} + metadata={span.entity.object.metadata} + /> + );
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (31)
pnpm-lock.yamlis excluded by!**/pnpm-lock.yamlreferences/hello-world/src/trigger/realtime.tsis excluded by!references/**references/realtime-streams/.gitignoreis excluded by!references/**references/realtime-streams/PERFORMANCE_TESTING.mdis excluded by!references/**references/realtime-streams/README.mdis excluded by!references/**references/realtime-streams/TESTING.mdis excluded by!references/**references/realtime-streams/next.config.tsis excluded by!references/**references/realtime-streams/package.jsonis excluded by!references/**references/realtime-streams/postcss.config.mjsis excluded by!references/**references/realtime-streams/public/file.svgis excluded by!**/*.svg,!references/**references/realtime-streams/public/globe.svgis excluded by!**/*.svg,!references/**references/realtime-streams/public/next.svgis excluded by!**/*.svg,!references/**references/realtime-streams/public/vercel.svgis excluded by!**/*.svg,!references/**references/realtime-streams/public/window.svgis excluded by!**/*.svg,!references/**references/realtime-streams/src/app/actions.tsis excluded by!references/**references/realtime-streams/src/app/chat/[runId]/page.tsxis excluded by!references/**references/realtime-streams/src/app/favicon.icois excluded by!**/*.ico,!references/**references/realtime-streams/src/app/globals.cssis excluded by!references/**references/realtime-streams/src/app/layout.tsxis excluded by!references/**references/realtime-streams/src/app/page.tsxis excluded by!references/**references/realtime-streams/src/app/performance/[runId]/page.tsxis excluded by!references/**references/realtime-streams/src/app/runs/[runId]/page.tsxis excluded by!references/**references/realtime-streams/src/components/ai-chat-button.tsxis excluded by!references/**references/realtime-streams/src/components/ai-chat.tsxis excluded by!references/**references/realtime-streams/src/components/performance-monitor.tsxis excluded by!references/**references/realtime-streams/src/components/streams.tsxis excluded by!references/**references/realtime-streams/src/components/trigger-button.tsxis excluded by!references/**references/realtime-streams/src/trigger/ai-chat.tsis excluded by!references/**references/realtime-streams/src/trigger/streams.tsis excluded by!references/**references/realtime-streams/trigger.config.tsis excluded by!references/**references/realtime-streams/tsconfig.jsonis excluded by!references/**
📒 Files selected for processing (77)
.changeset/quiet-falcons-approve.md(1 hunks)apps/webapp/app/assets/icons/ListBulletIcon.tsx(1 hunks)apps/webapp/app/assets/icons/MoveToBottomIcon.tsx(1 hunks)apps/webapp/app/assets/icons/SnakedArrowIcon.tsx(1 hunks)apps/webapp/app/assets/icons/StreamsIcon.tsx(1 hunks)apps/webapp/app/components/runs/v3/RunIcon.tsx(2 hunks)apps/webapp/app/env.server.ts(2 hunks)apps/webapp/app/models/organization.server.ts(1 hunks)apps/webapp/app/presenters/v3/SpanPresenter.server.ts(2 hunks)apps/webapp/app/routes/api.v1.tasks.$taskId.trigger.ts(4 hunks)apps/webapp/app/routes/realtime.v1.streams.$runId.$streamId.ts(2 hunks)apps/webapp/app/routes/realtime.v1.streams.$runId.$target.$streamId.ts(2 hunks)apps/webapp/app/routes/resources.orgs.$organizationSlug.projects.$projectParam.env.$envParam.runs.$runParam.spans.$spanParam/route.tsx(8 hunks)apps/webapp/app/routes/resources.orgs.$organizationSlug.projects.$projectParam.env.$envParam.runs.$runParam.streams.$streamKey/route.tsx(1 hunks)apps/webapp/app/runEngine/services/triggerTask.server.ts(1 hunks)apps/webapp/app/services/realtime/redisRealtimeStreams.server.ts(6 hunks)apps/webapp/app/services/realtime/relayRealtimeStreams.server.ts(0 hunks)apps/webapp/app/services/realtime/s2realtimeStreams.server.ts(1 hunks)apps/webapp/app/services/realtime/types.ts(1 hunks)apps/webapp/app/services/realtime/v1StreamsGlobal.server.ts(2 hunks)apps/webapp/app/services/realtimeClient.server.ts(1 hunks)apps/webapp/app/utils/pathBuilder.ts(1 hunks)apps/webapp/app/v3/environmentVariables/environmentVariablesRepository.server.ts(1 hunks)apps/webapp/app/v3/eventRepository/clickhouseEventRepository.server.ts(3 hunks)apps/webapp/app/v3/eventRepository/eventRepository.server.ts(1 hunks)apps/webapp/app/v3/eventRepository/eventRepository.types.ts(1 hunks)apps/webapp/app/v3/services/replayTaskRun.server.ts(1 hunks)apps/webapp/app/v3/services/triggerTask.server.ts(1 hunks)apps/webapp/package.json(2 hunks)apps/webapp/prisma/seed.ts(0 hunks)apps/webapp/prisma/seedCloud.ts(0 hunks)apps/webapp/seed.mts(1 hunks)apps/webapp/test/redisRealtimeStreams.test.ts(1 hunks)docker/config/nginx.conf(1 hunks)docker/config/toxiproxy.json(1 hunks)docker/docker-compose.yml(1 hunks)internal-packages/database/prisma/migrations/20251020121543_add_realtime_streams_version_to_task_run/migration.sql(1 hunks)internal-packages/database/prisma/migrations/20251020163612_add_realtime_streams_to_task_run/migration.sql(1 hunks)internal-packages/database/prisma/schema.prisma(1 hunks)internal-packages/run-engine/src/engine/index.ts(2 hunks)internal-packages/run-engine/src/engine/systems/runAttemptSystem.ts(2 hunks)internal-packages/run-engine/src/engine/types.ts(1 hunks)packages/cli-v3/src/entryPoints/dev-run-worker.ts(5 hunks)packages/cli-v3/src/entryPoints/managed-run-worker.ts(5 hunks)packages/core/package.json(1 hunks)packages/core/src/v3/apiClient/index.ts(9 hunks)packages/core/src/v3/apiClient/runStream.ts(8 hunks)packages/core/src/v3/apiClientManager/index.ts(1 hunks)packages/core/src/v3/apiClientManager/types.ts(2 hunks)packages/core/src/v3/index.ts(1 hunks)packages/core/src/v3/realtime-streams-api.ts(1 hunks)packages/core/src/v3/realtimeStreams/index.ts(1 hunks)packages/core/src/v3/realtimeStreams/manager.ts(1 hunks)packages/core/src/v3/realtimeStreams/noopManager.ts(1 hunks)packages/core/src/v3/realtimeStreams/streamsWriterV1.ts(1 hunks)packages/core/src/v3/realtimeStreams/streamsWriterV2.ts(1 hunks)packages/core/src/v3/realtimeStreams/types.ts(1 hunks)packages/core/src/v3/runMetadata/manager.ts(4 hunks)packages/core/src/v3/runMetadata/metadataStream.ts(0 hunks)packages/core/src/v3/schemas/api.ts(2 hunks)packages/core/src/v3/schemas/common.ts(1 hunks)packages/core/src/v3/semanticInternalAttributes.ts(1 hunks)packages/core/src/v3/streams/asyncIterableStream.ts(1 hunks)packages/core/src/v3/utils/globals.ts(2 hunks)packages/core/src/v3/waitUntil/index.ts(2 hunks)packages/core/src/v3/waitUntil/manager.ts(2 hunks)packages/core/src/v3/waitUntil/types.ts(1 hunks)packages/core/src/v3/workers/index.ts(1 hunks)packages/core/src/v3/workers/taskExecutor.ts(1 hunks)packages/core/test/runStream.test.ts(10 hunks)packages/core/test/streamsWriterV1.test.ts(1 hunks)packages/react-hooks/src/hooks/useRealtime.ts(4 hunks)packages/trigger-sdk/src/v3/index.ts(1 hunks)packages/trigger-sdk/src/v3/metadata.ts(2 hunks)packages/trigger-sdk/src/v3/shared.ts(9 hunks)packages/trigger-sdk/src/v3/streams.ts(1 hunks)turbo.json(1 hunks)
💤 Files with no reviewable changes (4)
- packages/core/src/v3/runMetadata/metadataStream.ts
- apps/webapp/prisma/seedCloud.ts
- apps/webapp/app/services/realtime/relayRealtimeStreams.server.ts
- apps/webapp/prisma/seed.ts
🚧 Files skipped from review as they are similar to previous changes (34)
- internal-packages/run-engine/src/engine/types.ts
- packages/core/src/v3/index.ts
- apps/webapp/app/assets/icons/SnakedArrowIcon.tsx
- packages/trigger-sdk/src/v3/index.ts
- docker/config/nginx.conf
- packages/core/src/v3/waitUntil/index.ts
- packages/core/src/v3/realtimeStreams/streamsWriterV2.ts
- internal-packages/run-engine/src/engine/index.ts
- packages/core/src/v3/realtimeStreams/noopManager.ts
- packages/core/src/v3/schemas/api.ts
- turbo.json
- apps/webapp/app/presenters/v3/SpanPresenter.server.ts
- packages/trigger-sdk/src/v3/metadata.ts
- internal-packages/database/prisma/migrations/20251020163612_add_realtime_streams_to_task_run/migration.sql
- internal-packages/database/prisma/schema.prisma
- apps/webapp/app/runEngine/services/triggerTask.server.ts
- apps/webapp/app/v3/services/replayTaskRun.server.ts
- apps/webapp/app/v3/eventRepository/clickhouseEventRepository.server.ts
- packages/core/src/v3/schemas/common.ts
- apps/webapp/app/v3/services/triggerTask.server.ts
- apps/webapp/app/v3/eventRepository/eventRepository.server.ts
- apps/webapp/app/env.server.ts
- packages/core/src/v3/streams/asyncIterableStream.ts
- apps/webapp/app/utils/pathBuilder.ts
- packages/core/package.json
- internal-packages/run-engine/src/engine/systems/runAttemptSystem.ts
- apps/webapp/app/components/runs/v3/RunIcon.tsx
- apps/webapp/app/services/realtimeClient.server.ts
- apps/webapp/app/routes/resources.orgs.$organizationSlug.projects.$projectParam.env.$envParam.runs.$runParam.streams.$streamKey/route.tsx
- apps/webapp/app/assets/icons/MoveToBottomIcon.tsx
- apps/webapp/app/assets/icons/ListBulletIcon.tsx
- packages/core/src/v3/workers/taskExecutor.ts
- packages/core/src/v3/waitUntil/types.ts
- packages/core/src/v3/apiClientManager/types.ts
🧰 Additional context used
📓 Path-based instructions (8)
**/*.{ts,tsx}
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
**/*.{ts,tsx}: Always prefer using isomorphic code like fetch, ReadableStream, etc. instead of Node.js specific code
For TypeScript, we usually use types over interfaces
Avoid enums
No default exports, use function declarations
Files:
packages/core/src/v3/utils/globals.tspackages/core/src/v3/apiClientManager/index.tsapps/webapp/app/routes/realtime.v1.streams.$runId.$streamId.tspackages/core/src/v3/workers/index.tspackages/core/src/v3/realtime-streams-api.tspackages/core/test/streamsWriterV1.test.tspackages/core/src/v3/waitUntil/manager.tspackages/core/src/v3/realtimeStreams/manager.tspackages/core/src/v3/semanticInternalAttributes.tsapps/webapp/app/v3/environmentVariables/environmentVariablesRepository.server.tsapps/webapp/app/routes/api.v1.tasks.$taskId.trigger.tspackages/core/src/v3/realtimeStreams/streamsWriterV1.tsapps/webapp/app/routes/realtime.v1.streams.$runId.$target.$streamId.tsapps/webapp/test/redisRealtimeStreams.test.tspackages/core/src/v3/runMetadata/manager.tspackages/trigger-sdk/src/v3/streams.tsapps/webapp/app/routes/resources.orgs.$organizationSlug.projects.$projectParam.env.$envParam.runs.$runParam.spans.$spanParam/route.tsxapps/webapp/app/services/realtime/types.tspackages/core/src/v3/realtimeStreams/types.tspackages/trigger-sdk/src/v3/shared.tsapps/webapp/app/models/organization.server.tsapps/webapp/app/v3/eventRepository/eventRepository.types.tspackages/cli-v3/src/entryPoints/managed-run-worker.tspackages/core/src/v3/apiClient/runStream.tsapps/webapp/app/services/realtime/v1StreamsGlobal.server.tsapps/webapp/app/assets/icons/StreamsIcon.tsxpackages/react-hooks/src/hooks/useRealtime.tspackages/core/test/runStream.test.tsapps/webapp/app/services/realtime/s2realtimeStreams.server.tspackages/cli-v3/src/entryPoints/dev-run-worker.tspackages/core/src/v3/realtimeStreams/index.tspackages/core/src/v3/apiClient/index.tsapps/webapp/app/services/realtime/redisRealtimeStreams.server.ts
{packages/core,apps/webapp}/**/*.{ts,tsx}
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
We use zod a lot in packages/core and in the webapp
Files:
packages/core/src/v3/utils/globals.tspackages/core/src/v3/apiClientManager/index.tsapps/webapp/app/routes/realtime.v1.streams.$runId.$streamId.tspackages/core/src/v3/workers/index.tspackages/core/src/v3/realtime-streams-api.tspackages/core/test/streamsWriterV1.test.tspackages/core/src/v3/waitUntil/manager.tspackages/core/src/v3/realtimeStreams/manager.tspackages/core/src/v3/semanticInternalAttributes.tsapps/webapp/app/v3/environmentVariables/environmentVariablesRepository.server.tsapps/webapp/app/routes/api.v1.tasks.$taskId.trigger.tspackages/core/src/v3/realtimeStreams/streamsWriterV1.tsapps/webapp/app/routes/realtime.v1.streams.$runId.$target.$streamId.tsapps/webapp/test/redisRealtimeStreams.test.tspackages/core/src/v3/runMetadata/manager.tsapps/webapp/app/routes/resources.orgs.$organizationSlug.projects.$projectParam.env.$envParam.runs.$runParam.spans.$spanParam/route.tsxapps/webapp/app/services/realtime/types.tspackages/core/src/v3/realtimeStreams/types.tsapps/webapp/app/models/organization.server.tsapps/webapp/app/v3/eventRepository/eventRepository.types.tspackages/core/src/v3/apiClient/runStream.tsapps/webapp/app/services/realtime/v1StreamsGlobal.server.tsapps/webapp/app/assets/icons/StreamsIcon.tsxpackages/core/test/runStream.test.tsapps/webapp/app/services/realtime/s2realtimeStreams.server.tspackages/core/src/v3/realtimeStreams/index.tspackages/core/src/v3/apiClient/index.tsapps/webapp/app/services/realtime/redisRealtimeStreams.server.ts
apps/webapp/**/*.{ts,tsx}
📄 CodeRabbit inference engine (.cursor/rules/webapp.mdc)
When importing from @trigger.dev/core in the webapp, never import the root package path; always use one of the documented subpath exports from @trigger.dev/core’s package.json
Files:
apps/webapp/app/routes/realtime.v1.streams.$runId.$streamId.tsapps/webapp/app/v3/environmentVariables/environmentVariablesRepository.server.tsapps/webapp/app/routes/api.v1.tasks.$taskId.trigger.tsapps/webapp/app/routes/realtime.v1.streams.$runId.$target.$streamId.tsapps/webapp/test/redisRealtimeStreams.test.tsapps/webapp/app/routes/resources.orgs.$organizationSlug.projects.$projectParam.env.$envParam.runs.$runParam.spans.$spanParam/route.tsxapps/webapp/app/services/realtime/types.tsapps/webapp/app/models/organization.server.tsapps/webapp/app/v3/eventRepository/eventRepository.types.tsapps/webapp/app/services/realtime/v1StreamsGlobal.server.tsapps/webapp/app/assets/icons/StreamsIcon.tsxapps/webapp/app/services/realtime/s2realtimeStreams.server.tsapps/webapp/app/services/realtime/redisRealtimeStreams.server.ts
{apps/webapp/app/**/*.server.{ts,tsx},apps/webapp/app/routes/**/*.ts}
📄 CodeRabbit inference engine (.cursor/rules/webapp.mdc)
Access environment variables only via the env export from app/env.server.ts; do not reference process.env directly
Files:
apps/webapp/app/routes/realtime.v1.streams.$runId.$streamId.tsapps/webapp/app/v3/environmentVariables/environmentVariablesRepository.server.tsapps/webapp/app/routes/api.v1.tasks.$taskId.trigger.tsapps/webapp/app/routes/realtime.v1.streams.$runId.$target.$streamId.tsapps/webapp/app/models/organization.server.tsapps/webapp/app/services/realtime/v1StreamsGlobal.server.tsapps/webapp/app/services/realtime/s2realtimeStreams.server.tsapps/webapp/app/services/realtime/redisRealtimeStreams.server.ts
apps/webapp/app/**/*.ts
📄 CodeRabbit inference engine (.cursor/rules/webapp.mdc)
Modules intended for test consumption under apps/webapp/app/**/*.ts must not read environment variables; accept configuration via options instead
Files:
apps/webapp/app/routes/realtime.v1.streams.$runId.$streamId.tsapps/webapp/app/v3/environmentVariables/environmentVariablesRepository.server.tsapps/webapp/app/routes/api.v1.tasks.$taskId.trigger.tsapps/webapp/app/routes/realtime.v1.streams.$runId.$target.$streamId.tsapps/webapp/app/services/realtime/types.tsapps/webapp/app/models/organization.server.tsapps/webapp/app/v3/eventRepository/eventRepository.types.tsapps/webapp/app/services/realtime/v1StreamsGlobal.server.tsapps/webapp/app/services/realtime/s2realtimeStreams.server.tsapps/webapp/app/services/realtime/redisRealtimeStreams.server.ts
**/*.test.{ts,tsx}
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
Our tests are all vitest
Files:
packages/core/test/streamsWriterV1.test.tsapps/webapp/test/redisRealtimeStreams.test.tspackages/core/test/runStream.test.ts
**/*.{test,spec}.{ts,tsx,js,jsx}
📄 CodeRabbit inference engine (AGENTS.md)
**/*.{test,spec}.{ts,tsx,js,jsx}: Unit tests must use Vitest
Tests should avoid mocks or stubs and use helpers from @internal/testcontainers when Redis or Postgres are needed
Test files live beside the files under test and should use descriptive describe and it blocks
Files:
packages/core/test/streamsWriterV1.test.tsapps/webapp/test/redisRealtimeStreams.test.tspackages/core/test/runStream.test.ts
{apps/webapp/**/__tests__/**/*.{ts,tsx},apps/webapp/**/*.{test,spec}.{ts,tsx}}
📄 CodeRabbit inference engine (.cursor/rules/webapp.mdc)
{apps/webapp/**/__tests__/**/*.{ts,tsx},apps/webapp/**/*.{test,spec}.{ts,tsx}}: Do not import app/env.server.ts into tests, either directly or indirectly
Tests should only import classes/functions from files under apps/webapp/app/**/*.ts
Files:
apps/webapp/test/redisRealtimeStreams.test.ts
🧠 Learnings (29)
📚 Learning: 2025-07-18T17:49:24.468Z
Learnt from: CR
PR: triggerdotdev/trigger.dev#0
File: .github/copilot-instructions.md:0-0
Timestamp: 2025-07-18T17:49:24.468Z
Learning: Applies to internal-packages/database/**/*.{ts,tsx} : We use prisma in internal-packages/database for our database interactions using PostgreSQL
Applied to files:
apps/webapp/seed.mtsapps/webapp/package.json
📚 Learning: 2025-08-18T10:07:17.368Z
Learnt from: CR
PR: triggerdotdev/trigger.dev#0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-08-18T10:07:17.368Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Import Trigger.dev APIs from "trigger.dev/sdk/v3" when writing tasks or related utilities
Applied to files:
packages/core/src/v3/utils/globals.tsapps/webapp/app/routes/api.v1.tasks.$taskId.trigger.tspackages/trigger-sdk/src/v3/streams.tspackages/trigger-sdk/src/v3/shared.tspackages/cli-v3/src/entryPoints/managed-run-worker.tspackages/cli-v3/src/entryPoints/dev-run-worker.ts
📚 Learning: 2025-08-29T10:06:49.293Z
Learnt from: CR
PR: triggerdotdev/trigger.dev#0
File: .cursor/rules/webapp.mdc:0-0
Timestamp: 2025-08-29T10:06:49.293Z
Learning: Applies to apps/webapp/**/*.{ts,tsx} : When importing from trigger.dev/core in the webapp, never import the root package path; always use one of the documented subpath exports from trigger.dev/core’s package.json
Applied to files:
packages/core/src/v3/utils/globals.tsapps/webapp/package.json
📚 Learning: 2025-08-29T10:06:49.293Z
Learnt from: CR
PR: triggerdotdev/trigger.dev#0
File: .cursor/rules/webapp.mdc:0-0
Timestamp: 2025-08-29T10:06:49.293Z
Learning: Maintain service/configuration separation (e.g., create testable service modules and separate configuration modules like realtimeClient.server.ts vs realtimeClientGlobal.server.ts)
Applied to files:
packages/core/src/v3/realtime-streams-api.ts
📚 Learning: 2025-07-18T17:49:24.468Z
Learnt from: CR
PR: triggerdotdev/trigger.dev#0
File: .github/copilot-instructions.md:0-0
Timestamp: 2025-07-18T17:49:24.468Z
Learning: Applies to **/*.test.{ts,tsx} : Our tests are all vitest
Applied to files:
packages/core/test/streamsWriterV1.test.tsapps/webapp/test/redisRealtimeStreams.test.tsapps/webapp/package.jsonpackages/core/test/runStream.test.ts
📚 Learning: 2025-07-18T17:49:24.468Z
Learnt from: CR
PR: triggerdotdev/trigger.dev#0
File: .github/copilot-instructions.md:0-0
Timestamp: 2025-07-18T17:49:24.468Z
Learning: Applies to **/*.{ts,tsx} : Always prefer using isomorphic code like fetch, ReadableStream, etc. instead of Node.js specific code
Applied to files:
packages/core/src/v3/realtimeStreams/manager.tspackages/core/src/v3/realtimeStreams/types.tspackages/core/src/v3/apiClient/runStream.ts
📚 Learning: 2025-10-08T11:48:12.327Z
Learnt from: nicktrn
PR: triggerdotdev/trigger.dev#2593
File: packages/core/src/v3/workers/warmStartClient.ts:168-170
Timestamp: 2025-10-08T11:48:12.327Z
Learning: The trigger.dev runners execute only in Node 21 and 22 environments, so modern Node.js APIs like AbortSignal.any (introduced in v20.3.0) are supported.
Applied to files:
packages/core/src/v3/realtimeStreams/manager.ts
📚 Learning: 2025-08-29T10:06:49.293Z
Learnt from: CR
PR: triggerdotdev/trigger.dev#0
File: .cursor/rules/webapp.mdc:0-0
Timestamp: 2025-08-29T10:06:49.293Z
Learning: Applies to {apps/webapp/app/**/*.server.{ts,tsx},apps/webapp/app/routes/**/*.ts} : Access environment variables only via the env export from app/env.server.ts; do not reference process.env directly
Applied to files:
apps/webapp/app/v3/environmentVariables/environmentVariablesRepository.server.tsapps/webapp/app/services/realtime/v1StreamsGlobal.server.ts
📚 Learning: 2025-08-19T09:49:07.011Z
Learnt from: julienvanbeveren
PR: triggerdotdev/trigger.dev#2417
File: apps/webapp/app/routes/api.v1.projects.$projectRef.envvars.$slug.import.ts:56-61
Timestamp: 2025-08-19T09:49:07.011Z
Learning: In the Trigger.dev codebase, environment variables should default to `isSecret: false` when not explicitly marked as secrets in the syncEnvVars functionality. This is the intended behavior for both regular variables and parent variables.
Applied to files:
apps/webapp/app/v3/environmentVariables/environmentVariablesRepository.server.ts
📚 Learning: 2025-08-18T10:07:17.368Z
Learnt from: CR
PR: triggerdotdev/trigger.dev#0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-08-18T10:07:17.368Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Export every task (including subtasks) defined with task(), schedules.task(), or schemaTask()
Applied to files:
apps/webapp/app/routes/api.v1.tasks.$taskId.trigger.ts
📚 Learning: 2025-08-18T10:07:17.368Z
Learnt from: CR
PR: triggerdotdev/trigger.dev#0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-08-18T10:07:17.368Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Do not use client.defineJob or any deprecated v2 patterns (e.g., eventTrigger) when defining tasks
Applied to files:
apps/webapp/app/routes/api.v1.tasks.$taskId.trigger.tspackages/trigger-sdk/src/v3/streams.tspackages/trigger-sdk/src/v3/shared.ts
📚 Learning: 2025-08-18T10:07:17.368Z
Learnt from: CR
PR: triggerdotdev/trigger.dev#0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-08-18T10:07:17.368Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : For idempotent child-task invocations, create and pass idempotencyKey (and optional TTL) when calling trigger()/batchTrigger() from tasks
Applied to files:
apps/webapp/app/routes/api.v1.tasks.$taskId.trigger.tspackages/trigger-sdk/src/v3/shared.ts
📚 Learning: 2025-08-18T10:07:17.368Z
Learnt from: CR
PR: triggerdotdev/trigger.dev#0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-08-18T10:07:17.368Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Define tasks using task({ id, run, ... }) with a unique id per project
Applied to files:
apps/webapp/app/routes/api.v1.tasks.$taskId.trigger.tspackages/trigger-sdk/src/v3/streams.tspackages/trigger-sdk/src/v3/shared.ts
📚 Learning: 2025-08-18T10:07:17.368Z
Learnt from: CR
PR: triggerdotdev/trigger.dev#0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-08-18T10:07:17.368Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Use schemaTask({ schema, run, ... }) to validate payloads when input validation is required
Applied to files:
apps/webapp/app/routes/api.v1.tasks.$taskId.trigger.tspackages/trigger-sdk/src/v3/shared.ts
📚 Learning: 2025-08-18T10:07:17.368Z
Learnt from: CR
PR: triggerdotdev/trigger.dev#0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-08-18T10:07:17.368Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Use schedules.task(...) for scheduled (cron) tasks; do not implement schedules as plain task() with external cron logic
Applied to files:
apps/webapp/app/routes/api.v1.tasks.$taskId.trigger.tspackages/trigger-sdk/src/v3/shared.ts
📚 Learning: 2025-08-18T10:07:17.368Z
Learnt from: CR
PR: triggerdotdev/trigger.dev#0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-08-18T10:07:17.368Z
Learning: Applies to trigger.config.ts : Configure global task lifecycle hooks (onStart/onSuccess/onFailure) only within trigger.config.ts if needed, not within arbitrary files
Applied to files:
apps/webapp/app/routes/api.v1.tasks.$taskId.trigger.tspackages/trigger-sdk/src/v3/shared.ts
📚 Learning: 2025-08-18T10:07:17.368Z
Learnt from: CR
PR: triggerdotdev/trigger.dev#0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-08-18T10:07:17.368Z
Learning: Applies to trigger.config.ts : Declare build options and extensions (external, jsx, conditions, extensions) via the build block in trigger.config.ts rather than custom scripts
Applied to files:
apps/webapp/app/routes/api.v1.tasks.$taskId.trigger.tspackages/trigger-sdk/src/v3/shared.ts
📚 Learning: 2025-08-18T10:07:17.368Z
Learnt from: CR
PR: triggerdotdev/trigger.dev#0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-08-18T10:07:17.368Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Use triggerAndWait() only from within a task context (not from generic app code) and handle result.ok or use unwrap() with error handling
Applied to files:
apps/webapp/app/routes/api.v1.tasks.$taskId.trigger.tspackages/trigger-sdk/src/v3/shared.ts
📚 Learning: 2025-08-29T15:49:22.406Z
Learnt from: CR
PR: triggerdotdev/trigger.dev#0
File: AGENTS.md:0-0
Timestamp: 2025-08-29T15:49:22.406Z
Learning: Applies to **/*.{test,spec}.{ts,tsx,js,jsx} : Tests should avoid mocks or stubs and use helpers from internal/testcontainers when Redis or Postgres are needed
Applied to files:
apps/webapp/test/redisRealtimeStreams.test.ts
📚 Learning: 2024-10-22T10:50:41.311Z
Learnt from: nicktrn
PR: triggerdotdev/trigger.dev#1424
File: packages/core/src/v3/errors.ts:155-189
Timestamp: 2024-10-22T10:50:41.311Z
Learning: When using `assertExhaustive` in a `switch` statement in TypeScript (e.g., in the `shouldRetryError` function in `packages/core/src/v3/errors.ts`), and it throws an error, it's acceptable not to add a `return` statement afterward, as control flow will not proceed beyond the `assertExhaustive` call.
Applied to files:
packages/trigger-sdk/src/v3/streams.ts
📚 Learning: 2025-07-18T17:49:24.468Z
Learnt from: CR
PR: triggerdotdev/trigger.dev#0
File: .github/copilot-instructions.md:0-0
Timestamp: 2025-07-18T17:49:24.468Z
Learning: Applies to {packages/core,apps/webapp}/**/*.{ts,tsx} : We use zod a lot in packages/core and in the webapp
Applied to files:
apps/webapp/package.json
📚 Learning: 2025-08-29T15:49:22.406Z
Learnt from: CR
PR: triggerdotdev/trigger.dev#0
File: AGENTS.md:0-0
Timestamp: 2025-08-29T15:49:22.406Z
Learning: Applies to **/*.{test,spec}.{ts,tsx,js,jsx} : Unit tests must use Vitest
Applied to files:
apps/webapp/package.jsonpackages/core/test/runStream.test.ts
📚 Learning: 2025-08-18T10:07:17.368Z
Learnt from: CR
PR: triggerdotdev/trigger.dev#0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-08-18T10:07:17.368Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : When triggering a task multiple times in a loop from inside another task, use batchTrigger()/batchTriggerAndWait() instead of per-item trigger() calls
Applied to files:
packages/trigger-sdk/src/v3/shared.ts
📚 Learning: 2025-08-18T10:07:17.368Z
Learnt from: CR
PR: triggerdotdev/trigger.dev#0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-08-18T10:07:17.368Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Use metadata API (metadata.current/get/set/append/stream, etc.) only inside run functions or lifecycle hooks
Applied to files:
packages/trigger-sdk/src/v3/shared.tspackages/cli-v3/src/entryPoints/managed-run-worker.tspackages/cli-v3/src/entryPoints/dev-run-worker.ts
📚 Learning: 2025-08-18T10:07:17.368Z
Learnt from: CR
PR: triggerdotdev/trigger.dev#0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-08-18T10:07:17.368Z
Learning: When triggering tasks from backend code, prefer tasks.trigger/tasks.batchTrigger/batch.trigger with type-only imports for type safety
Applied to files:
packages/trigger-sdk/src/v3/shared.ts
📚 Learning: 2025-09-03T14:34:41.781Z
Learnt from: myftija
PR: triggerdotdev/trigger.dev#2464
File: apps/webapp/app/routes/_app.orgs.$organizationSlug.projects.$projectParam.env.$envParam.settings/route.tsx:357-371
Timestamp: 2025-09-03T14:34:41.781Z
Learning: When using Zod's safeParse, the .data property is undefined when parsing fails, but TypeScript may still complain about accessing .data without checking .success first. The suggested approach of checking .success before accessing .data improves type safety and code clarity.
Applied to files:
packages/core/src/v3/apiClient/runStream.ts
📚 Learning: 2025-05-27T19:30:34.004Z
Learnt from: ericallam
PR: triggerdotdev/trigger.dev#2086
File: internal-packages/database/prisma/migrations/20250511145836_runtime_environment_add_is_branchable_environment/migration.sql:1-3
Timestamp: 2025-05-27T19:30:34.004Z
Learning: In modern PostgreSQL versions (11+), adding a column with a constant default value (like DEFAULT false, DEFAULT 0, DEFAULT 'text') does NOT require a table rewrite. PostgreSQL stores the default value in the catalog and applies it virtually when reading rows. Only non-constant defaults or more complex scenarios require table rewrites. Avoid suggesting multi-step migrations for simple constant defaults.
Applied to files:
internal-packages/database/prisma/migrations/20251020121543_add_realtime_streams_version_to_task_run/migration.sql
📚 Learning: 2025-08-18T10:07:17.368Z
Learnt from: CR
PR: triggerdotdev/trigger.dev#0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-08-18T10:07:17.368Z
Learning: For Realtime subscriptions or React hooks, provide a Public Access Token and scope it appropriately (e.g., via TriggerAuthContext)
Applied to files:
packages/react-hooks/src/hooks/useRealtime.ts
📚 Learning: 2025-08-29T15:49:22.406Z
Learnt from: CR
PR: triggerdotdev/trigger.dev#0
File: AGENTS.md:0-0
Timestamp: 2025-08-29T15:49:22.406Z
Learning: Applies to **/*.{test,spec}.{ts,tsx,js,jsx} : Test files live beside the files under test and should use descriptive describe and it blocks
Applied to files:
packages/core/test/runStream.test.ts
🧬 Code graph analysis (26)
apps/webapp/seed.mts (1)
apps/webapp/app/models/organization.server.ts (1)
createOrganization(20-77)
packages/core/src/v3/utils/globals.ts (2)
packages/core/src/v3/realtimeStreams/types.ts (1)
RealtimeStreamsManager(10-16)packages/core/src/v3/realtimeStreams/index.ts (1)
RealtimeStreamsManager(30-32)
packages/core/src/v3/apiClientManager/index.ts (2)
packages/core/src/v3/apiClient/index.ts (1)
ApiClient(155-1221)packages/core/src/v3/apiClientManager/types.ts (1)
ApiClientConfiguration(3-19)
apps/webapp/app/routes/realtime.v1.streams.$runId.$streamId.ts (1)
apps/webapp/app/services/realtime/v1StreamsGlobal.server.ts (1)
getRealtimeStreamInstance(25-52)
packages/core/src/v3/realtime-streams-api.ts (1)
packages/core/src/v3/realtimeStreams/index.ts (1)
RealtimeStreamsAPI(13-41)
packages/core/test/streamsWriterV1.test.ts (1)
packages/core/src/v3/realtimeStreams/streamsWriterV1.ts (1)
StreamsWriterV1(26-466)
packages/core/src/v3/realtimeStreams/manager.ts (6)
packages/core/src/v3/realtimeStreams/types.ts (3)
RealtimeStreamsManager(10-16)RealtimeAppendStreamOptions(4-8)RealtimeStreamInstance(18-21)packages/core/src/v3/apiClient/index.ts (1)
headers(1209-1220)packages/core/src/v3/streams/asyncIterableStream.ts (3)
ensureAsyncIterable(107-135)AsyncIterableStream(1-1)createAsyncIterableStreamFromAsyncIterable(52-97)packages/core/src/v3/realtimeStreams/streamsWriterV1.ts (1)
StreamsWriterV1(26-466)packages/core/src/v3/realtimeStreams/streamsWriterV2.ts (1)
StreamsWriterV2(56-392)packages/core/src/v3/task-context-api.ts (1)
taskContext(5-5)
apps/webapp/app/v3/environmentVariables/environmentVariablesRepository.server.ts (1)
apps/webapp/app/env.server.ts (1)
env(1219-1219)
packages/core/src/v3/realtimeStreams/streamsWriterV1.ts (1)
packages/core/src/v3/realtimeStreams/types.ts (1)
StreamsWriter(23-25)
apps/webapp/app/routes/realtime.v1.streams.$runId.$target.$streamId.ts (3)
apps/webapp/app/services/realtime/v1StreamsGlobal.server.ts (1)
getRealtimeStreamInstance(25-52)apps/webapp/app/services/routeBuilders/apiBuilder.server.ts (1)
createLoaderApiRoute(97-287)apps/webapp/app/db.server.ts (1)
$replica(103-106)
apps/webapp/test/redisRealtimeStreams.test.ts (2)
internal-packages/testcontainers/src/index.ts (1)
redisTest(167-167)apps/webapp/app/services/realtime/redisRealtimeStreams.server.ts (1)
RedisRealtimeStreams(23-413)
packages/core/src/v3/runMetadata/manager.ts (1)
packages/core/src/v3/realtime-streams-api.ts (1)
realtimeStreams(5-5)
packages/trigger-sdk/src/v3/streams.ts (10)
packages/core/src/v3/apiClient/index.ts (2)
ApiRequestOptions(141-141)mergeRequestOptions(1390-1406)packages/core/src/v3/streams/asyncIterableStream.ts (1)
AsyncIterableStream(1-1)packages/core/src/v3/realtimeStreams/manager.ts (1)
append(32-101)packages/core/src/v3/realtimeStreams/index.ts (1)
append(34-40)packages/core/src/v3/realtimeStreams/noopManager.ts (1)
append(12-23)packages/core/src/v3/semanticInternalAttributes.ts (1)
SemanticInternalAttributes(1-68)packages/core/src/v3/realtime-streams-api.ts (1)
realtimeStreams(5-5)packages/core/src/v3/apiClientManager-api.ts (1)
apiClientManager(5-5)packages/core/src/v3/realtimeStreams/types.ts (1)
RealtimeAppendStreamOptions(4-8)packages/core/src/v3/task-context-api.ts (1)
taskContext(5-5)
apps/webapp/app/routes/resources.orgs.$organizationSlug.projects.$projectParam.env.$envParam.runs.$runParam.spans.$spanParam/route.tsx (1)
apps/webapp/app/routes/resources.orgs.$organizationSlug.projects.$projectParam.env.$envParam.runs.$runParam.streams.$streamKey/route.tsx (1)
RealtimeStreamViewer(96-386)
apps/webapp/app/services/realtime/types.ts (2)
packages/core/src/v3/runMetadata/manager.ts (1)
stream(297-303)packages/core/src/v3/apiClient/stream.ts (1)
stream(180-182)
packages/core/src/v3/realtimeStreams/types.ts (3)
packages/core/src/v3/apiClient/core.ts (1)
AnyZodFetchOptions(41-41)packages/core/src/v3/realtimeStreams/index.ts (1)
RealtimeStreamsManager(30-32)packages/core/src/v3/streams/asyncIterableStream.ts (1)
AsyncIterableStream(1-1)
packages/trigger-sdk/src/v3/shared.ts (2)
packages/core/src/v3/apiClientManager-api.ts (1)
apiClientManager(5-5)packages/core/src/v3/apiClient/index.ts (1)
TriggerApiRequestOptions(118-121)
packages/cli-v3/src/entryPoints/managed-run-worker.ts (6)
packages/core/src/v3/runMetadata/manager.ts (1)
StandardMetadataManager(12-452)packages/core/src/v3/apiClientManager-api.ts (1)
apiClientManager(5-5)packages/core/src/v3/run-metadata-api.ts (1)
runMetadata(5-5)packages/core/src/v3/realtimeStreams/manager.ts (1)
StandardRealtimeStreamsManager(16-141)packages/core/src/v3/realtime-streams-api.ts (1)
realtimeStreams(5-5)packages/core/src/v3/waitUntil/manager.ts (1)
StandardWaitUntilManager(3-40)
apps/webapp/app/services/realtime/v1StreamsGlobal.server.ts (3)
apps/webapp/app/env.server.ts (1)
env(1219-1219)apps/webapp/app/services/realtime/types.ts (2)
StreamIngestor(2-17)StreamResponder(25-33)apps/webapp/app/services/realtime/s2realtimeStreams.server.ts (1)
S2RealtimeStreams(32-246)
packages/react-hooks/src/hooks/useRealtime.ts (3)
packages/react-hooks/src/hooks/useApiClient.ts (2)
UseApiClientOptions(9-25)useApiClient(47-67)packages/core/src/v3/apiClient/index.ts (1)
ApiClient(155-1221)packages/react-hooks/src/utils/throttle.ts (1)
createThrottledQueue(8-58)
packages/core/test/runStream.test.ts (1)
packages/core/src/v3/apiClient/runStream.ts (1)
SSEStreamPart(175-179)
apps/webapp/app/services/realtime/s2realtimeStreams.server.ts (1)
apps/webapp/app/services/realtime/types.ts (3)
StreamResponder(25-33)StreamIngestor(2-17)StreamResponseOptions(19-22)
packages/cli-v3/src/entryPoints/dev-run-worker.ts (7)
packages/core/src/v3/runMetadata/manager.ts (1)
StandardMetadataManager(12-452)packages/core/src/v3/apiClientManager-api.ts (1)
apiClientManager(5-5)packages/core/src/v3/run-metadata-api.ts (1)
runMetadata(5-5)packages/core/src/v3/realtimeStreams/manager.ts (1)
StandardRealtimeStreamsManager(16-141)packages/core/src/v3/utils/getEnv.ts (2)
getEnvVar(11-13)getNumberEnvVar(15-29)packages/core/src/v3/realtime-streams-api.ts (1)
realtimeStreams(5-5)packages/core/src/v3/waitUntil/manager.ts (1)
StandardWaitUntilManager(3-40)
packages/core/src/v3/realtimeStreams/index.ts (3)
packages/core/src/v3/realtimeStreams/noopManager.ts (1)
NoopRealtimeStreamsManager(11-24)packages/core/src/v3/realtimeStreams/types.ts (3)
RealtimeStreamsManager(10-16)RealtimeAppendStreamOptions(4-8)RealtimeStreamInstance(18-21)packages/core/src/v3/utils/globals.ts (2)
registerGlobal(22-38)getGlobal(40-44)
packages/core/src/v3/apiClient/index.ts (4)
packages/core/src/v3/apiClientManager/index.ts (2)
ApiClientConfiguration(100-102)accessToken(37-45)packages/core/src/v3/apiClientManager/types.ts (1)
ApiClientConfiguration(3-19)packages/core/src/v3/schemas/api.ts (2)
CreateStreamResponseBody(1301-1303)CreateStreamResponseBody(1304-1304)packages/core/src/v3/utils/getEnv.ts (1)
getEnvVar(11-13)
apps/webapp/app/services/realtime/redisRealtimeStreams.server.ts (2)
packages/core/src/logger.ts (3)
Logger(19-139)LogLevel(15-15)error(69-79)apps/webapp/app/services/realtime/types.ts (3)
StreamIngestor(2-17)StreamResponder(25-33)StreamResponseOptions(19-22)
🪛 Biome (2.1.2)
packages/core/test/streamsWriterV1.test.ts
[error] 792-795: This generator function doesn't contain yield.
(lint/correctness/useYield)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (23)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (6, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (8, 8)
- GitHub Check: units / packages / 🧪 Unit Tests: Packages (1, 1)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (6, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (7, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (2, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (3, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (4, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (5, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (1, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (8, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (5, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (3, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (1, 8)
- GitHub Check: e2e / 🧪 CLI v3 tests (ubuntu-latest - pnpm)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (4, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (7, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (2, 8)
- GitHub Check: e2e / 🧪 CLI v3 tests (windows-latest - npm)
- GitHub Check: e2e / 🧪 CLI v3 tests (ubuntu-latest - npm)
- GitHub Check: typecheck / typecheck
- GitHub Check: e2e / 🧪 CLI v3 tests (windows-latest - pnpm)
- GitHub Check: Analyze (javascript-typescript)
ericallam
force-pushed
the
feat/realtime-streams-2
branch
from
0a80d52 to
667bf8d
Compare
There was a problem hiding this comment.
Actionable comments posted: 5
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
apps/webapp/app/models/organization.server.ts (1)
55-55: Remove unused variable.The
featuresvariable is declared but no longer used after hardcodingv3Enabled: trueon line 69.Apply this diff to remove the unused code:
- const features = featuresForUrl(new URL(env.APP_ORIGIN)); - const organization = await prisma.organization.create({apps/webapp/app/routes/realtime.v1.streams.$runId.$target.$streamId.ts (1)
21-40: Fetch realtimeStreamsVersion to ensure version consistency.The query doesn't select the
realtimeStreamsVersionfield, but the POST handler (lines 87-117) needs this to determine the correct stream backend. Currently, the code trusts the client-providedX-Stream-Versionheader (line 90), which can mismatch the run's actual version and route data to the wrong backend.Apply this diff to fetch the version:
select: { id: true, friendlyId: true, + realtimeStreamsVersion: true, parentTaskRun: { select: { friendlyId: true,
♻️ Duplicate comments (7)
docker/docker-compose.yml (1)
157-166: Missing networks configuration on nginx-h2 service.The
nginx-h2service lacks the requirednetworksconfiguration needed to communicate with other services onapp_network. Without this, nginx cannot reach upstream application services and real-time streaming proxying will fail.Apply this diff to add the networks configuration:
nginx-h2: image: nginx:1.27 container_name: nginx-h2 restart: unless-stopped ports: - "8443:8443" volumes: - ./config/nginx.conf:/etc/nginx/nginx.conf:ro - ./config/certs:/etc/nginx/certs:ro + networks: + - app_networkpackages/core/test/streamsWriterV1.test.ts (1)
792-795: Satisfy generator-without-yield lint rule.The empty generator function triggers a lint error because it never yields. Add a dead-code yield to satisfy the linter while maintaining the intended behavior of testing empty streams.
Apply this diff:
async function* generateChunks() { - // Yields nothing + // Yields nothing; keep generator shape for API compatibility + if (false) { + yield undefined as never; + } return; }apps/webapp/app/routes/resources.orgs.$organizationSlug.projects.$projectParam.env.$envParam.runs.$runParam.streams.$streamKey/route.tsx (2)
200-204: Fix invalid ScrollBehavior value.The
scrollIntoViewcall usesbehavior: "instant", which is not a valid ScrollBehavior value in the DOM specification. Use"auto"or"smooth"instead.Apply this diff:
if (isAtBottom && bottomRef.current) { - bottomRef.current.scrollIntoView({ behavior: "instant", block: "end" }); + bottomRef.current.scrollIntoView({ behavior: "auto", block: "end" }); }
455-455: Fix startIndex falsy check that drops lastEventId when startIndex is 0.When
startIndexis0, the current falsy check treats it as undefined and omits thelastEventIdheader. This breaks resume functionality when starting from the beginning of the stream.Apply this diff:
- lastEventId: startIndex ? (startIndex - 1).toString() : undefined, + lastEventId: startIndex !== undefined ? (startIndex - 1).toString() : undefined,apps/webapp/app/routes/realtime.v1.streams.$runId.$streamId.ts (1)
46-58: Fix Timeout-Seconds validation (truthy guard bug remains)The truthy checks still let
"0"and"NaN"slip through, so callers can request a zero-second timeout or send garbage and reachstreamResponse, which breaks the promised 1–600 s bounds. Parse explicitly, enforce finiteness, and clamp the range before using it.Apply this diff:
- const timeoutInSecondsRaw = request.headers.get("Timeout-Seconds") ?? undefined; - const timeoutInSeconds = timeoutInSecondsRaw ? parseInt(timeoutInSecondsRaw) : undefined; - - if (timeoutInSeconds && isNaN(timeoutInSeconds)) { - return new Response("Invalid timeout seconds", { status: 400 }); - } - - if (timeoutInSeconds && timeoutInSeconds < 1) { - return new Response("Timeout seconds must be greater than 0", { status: 400 }); - } - - if (timeoutInSeconds && timeoutInSeconds > 600) { - return new Response("Timeout seconds must be less than 600", { status: 400 }); - } + const timeoutInSecondsRaw = request.headers.get("Timeout-Seconds"); + let timeoutInSeconds: number | undefined; + if (timeoutInSecondsRaw !== null) { + const n = Number(timeoutInSecondsRaw.trim()); + if (!Number.isFinite(n)) { + return new Response("Invalid timeout seconds", { status: 400 }); + } + if (n < 1) { + return new Response("Timeout seconds must be greater than 0", { status: 400 }); + } + if (n > 600) { + return new Response("Timeout seconds must be less than 600", { status: 400 }); + } + timeoutInSeconds = n; + }packages/core/src/v3/realtimeStreams/streamsWriterV1.ts (1)
80-103: Stop buffer task rejections from leaking to the processIf the source stream errors (or the task aborts), the async IIFE rethrows the failure. Nothing awaits
bufferReaderTask, so Node flags an unhandled rejection and the reader lock never gets released, leaving the sender loop spinning forever. Capture the error, release the reader, and let the send loop surface it so we can abort cleanly.@@ - private bufferReaderTask: Promise<void> | null = null; - private streamComplete = false; + private bufferReaderTask: Promise<void> | null = null; + private streamComplete = false; + private bufferError: unknown = null; @@ - this.bufferReaderTask = (async () => { - try { - let chunkIndex = 0; - while (true) { - const { done, value } = await this.streamReader!.read(); - - if (done) { - this.streamComplete = true; - break; - } - - // Add to ring buffer - this.addToRingBuffer(chunkIndex, value); - this.highestBufferedIndex = chunkIndex; - chunkIndex++; - } - } catch (error) { - throw error; - } - })(); + this.bufferReaderTask = (async () => { + try { + let chunkIndex = 0; + while (true) { + const { done, value } = await this.streamReader!.read(); + + if (done) { + this.streamComplete = true; + break; + } + + // Add to ring buffer + this.addToRingBuffer(chunkIndex, value); + this.highestBufferedIndex = chunkIndex; + chunkIndex++; + } + } catch (error) { + this.bufferError = error; + this.streamComplete = true; + } finally { + try { + this.streamReader?.releaseLock(); + } catch { + // ignore + } + this.streamReader = null; + } + })(); + this.bufferReaderTask.catch(() => {}); @@ - while (lastSentIndex < this.highestBufferedIndex) { + while (lastSentIndex < this.highestBufferedIndex) { lastSentIndex++; const chunk = this.ringBuffer.find((c) => c.index === lastSentIndex); @@ - if (this.streamComplete && lastSentIndex >= this.highestBufferedIndex) { + if (this.bufferError) { + throw this.bufferError instanceof Error + ? this.bufferError + : new Error(String(this.bufferError)); + } + + if (this.streamComplete && lastSentIndex >= this.highestBufferedIndex) { req.end(); break; }packages/core/src/v3/apiClient/runStream.ts (1)
500-521: Handle subscription/pipe failures for stream fan-outWe still launch the subscription in the background with no rejection handling. If
subscribe()or the downstreampipeTofails, the promise rejects with no consumer, crashing the process and leavingactiveStreamspolluted. Propagate the failure and clean up the bookkeeping.const subscription = this.options.streamFactory.createSubscription( run.id, streamKey, { baseUrl: this.options.client?.baseUrl, } ); - // Start stream processing in the background - subscription.subscribe().then((stream) => { - stream - .pipeThrough( - new TransformStream({ - transform(chunk, controller) { - controller.enqueue({ - type: streamKey, - chunk: chunk.chunk as TStreams[typeof streamKey], - run, - }); - }, - }) - ) - .pipeTo( - new WritableStream({ - write(chunk) { - controller.enqueue(chunk); - }, - }) - ); - }); + subscription + .subscribe() + .then((stream) => { + stream + .pipeThrough( + new TransformStream({ + transform(chunk, controller) { + controller.enqueue({ + type: streamKey, + chunk: chunk.chunk as TStreams[typeof streamKey], + run, + }); + }, + }) + ) + .pipeTo( + new WritableStream({ + write(chunk) { + controller.enqueue(chunk); + }, + }) + ) + .catch((error) => { + activeStreams.delete(streamKey); + controller.error(error); + }); + }) + .catch((error) => { + activeStreams.delete(streamKey); + controller.error(error); + });
🧹 Nitpick comments (3)
apps/webapp/app/models/organization.server.ts (1)
30-30: Environment variable access violates coding guidelines.This line directly accesses
process.env.BLOCKED_USERSinstead of using theenvexport fromapp/env.server.ts. While this is existing code (not introduced in this PR), it should be refactored for consistency.As per coding guidelines:
Apply this diff to align with the coding guidelines:
- if (typeof process.env.BLOCKED_USERS === "string" && process.env.BLOCKED_USERS.includes(userId)) { + if (typeof env.BLOCKED_USERS === "string" && env.BLOCKED_USERS.includes(userId)) {Note: Ensure
BLOCKED_USERSis exported from~/env.server.ts.apps/webapp/app/routes/realtime.v1.streams.$runId.$target.$streamId.ts (1)
150-152: Remove unreachable null check.The
createLoaderApiRoutebuilder already validates thatresourceis non-null before invoking the handler (seeapiBuilder.server.tslines 185-195), so this null check is unreachable dead code.Apply this diff to remove the dead code:
async ({ request, params, resource: run, authentication }) => { - if (!run) { - return new Response("Run not found", { status: 404 }); - } - const targetId =packages/core/src/v3/realtimeStreams/index.ts (1)
26-32: Consider explicit return type documentation.The setGlobalManager and #getManager methods work correctly, but consider adding JSDoc comments to clarify the boolean return value from setGlobalManager indicates success/failure of registration.
+ /** + * Registers a global manager for realtime streams. + * @returns true if registration succeeded, false if a manager was already registered + */ setGlobalManager(manager: RealtimeStreamsManager): boolean { return registerGlobal(API_NAME, manager); }
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (31)
pnpm-lock.yamlis excluded by!**/pnpm-lock.yamlreferences/hello-world/src/trigger/realtime.tsis excluded by!references/**references/realtime-streams/.gitignoreis excluded by!references/**references/realtime-streams/PERFORMANCE_TESTING.mdis excluded by!references/**references/realtime-streams/README.mdis excluded by!references/**references/realtime-streams/TESTING.mdis excluded by!references/**references/realtime-streams/next.config.tsis excluded by!references/**references/realtime-streams/package.jsonis excluded by!references/**references/realtime-streams/postcss.config.mjsis excluded by!references/**references/realtime-streams/public/file.svgis excluded by!**/*.svg,!references/**references/realtime-streams/public/globe.svgis excluded by!**/*.svg,!references/**references/realtime-streams/public/next.svgis excluded by!**/*.svg,!references/**references/realtime-streams/public/vercel.svgis excluded by!**/*.svg,!references/**references/realtime-streams/public/window.svgis excluded by!**/*.svg,!references/**references/realtime-streams/src/app/actions.tsis excluded by!references/**references/realtime-streams/src/app/chat/[runId]/page.tsxis excluded by!references/**references/realtime-streams/src/app/favicon.icois excluded by!**/*.ico,!references/**references/realtime-streams/src/app/globals.cssis excluded by!references/**references/realtime-streams/src/app/layout.tsxis excluded by!references/**references/realtime-streams/src/app/page.tsxis excluded by!references/**references/realtime-streams/src/app/performance/[runId]/page.tsxis excluded by!references/**references/realtime-streams/src/app/runs/[runId]/page.tsxis excluded by!references/**references/realtime-streams/src/components/ai-chat-button.tsxis excluded by!references/**references/realtime-streams/src/components/ai-chat.tsxis excluded by!references/**references/realtime-streams/src/components/performance-monitor.tsxis excluded by!references/**references/realtime-streams/src/components/streams.tsxis excluded by!references/**references/realtime-streams/src/components/trigger-button.tsxis excluded by!references/**references/realtime-streams/src/trigger/ai-chat.tsis excluded by!references/**references/realtime-streams/src/trigger/streams.tsis excluded by!references/**references/realtime-streams/trigger.config.tsis excluded by!references/**references/realtime-streams/tsconfig.jsonis excluded by!references/**
📒 Files selected for processing (77)
.changeset/quiet-falcons-approve.md(1 hunks)apps/webapp/app/assets/icons/ListBulletIcon.tsx(1 hunks)apps/webapp/app/assets/icons/MoveToBottomIcon.tsx(1 hunks)apps/webapp/app/assets/icons/SnakedArrowIcon.tsx(1 hunks)apps/webapp/app/assets/icons/StreamsIcon.tsx(1 hunks)apps/webapp/app/components/runs/v3/RunIcon.tsx(2 hunks)apps/webapp/app/env.server.ts(2 hunks)apps/webapp/app/models/organization.server.ts(1 hunks)apps/webapp/app/presenters/v3/SpanPresenter.server.ts(2 hunks)apps/webapp/app/routes/api.v1.tasks.$taskId.trigger.ts(4 hunks)apps/webapp/app/routes/realtime.v1.streams.$runId.$streamId.ts(2 hunks)apps/webapp/app/routes/realtime.v1.streams.$runId.$target.$streamId.ts(2 hunks)apps/webapp/app/routes/resources.orgs.$organizationSlug.projects.$projectParam.env.$envParam.runs.$runParam.spans.$spanParam/route.tsx(8 hunks)apps/webapp/app/routes/resources.orgs.$organizationSlug.projects.$projectParam.env.$envParam.runs.$runParam.streams.$streamKey/route.tsx(1 hunks)apps/webapp/app/runEngine/services/triggerTask.server.ts(1 hunks)apps/webapp/app/services/realtime/redisRealtimeStreams.server.ts(6 hunks)apps/webapp/app/services/realtime/relayRealtimeStreams.server.ts(0 hunks)apps/webapp/app/services/realtime/s2realtimeStreams.server.ts(1 hunks)apps/webapp/app/services/realtime/types.ts(1 hunks)apps/webapp/app/services/realtime/v1StreamsGlobal.server.ts(2 hunks)apps/webapp/app/services/realtimeClient.server.ts(1 hunks)apps/webapp/app/utils/pathBuilder.ts(1 hunks)apps/webapp/app/v3/environmentVariables/environmentVariablesRepository.server.ts(1 hunks)apps/webapp/app/v3/eventRepository/clickhouseEventRepository.server.ts(3 hunks)apps/webapp/app/v3/eventRepository/eventRepository.server.ts(1 hunks)apps/webapp/app/v3/eventRepository/eventRepository.types.ts(1 hunks)apps/webapp/app/v3/services/replayTaskRun.server.ts(1 hunks)apps/webapp/app/v3/services/triggerTask.server.ts(1 hunks)apps/webapp/package.json(2 hunks)apps/webapp/prisma/seed.ts(0 hunks)apps/webapp/prisma/seedCloud.ts(0 hunks)apps/webapp/seed.mts(1 hunks)apps/webapp/test/redisRealtimeStreams.test.ts(1 hunks)docker/config/nginx.conf(1 hunks)docker/config/toxiproxy.json(1 hunks)docker/docker-compose.yml(1 hunks)internal-packages/database/prisma/migrations/20251020121543_add_realtime_streams_version_to_task_run/migration.sql(1 hunks)internal-packages/database/prisma/migrations/20251020163612_add_realtime_streams_to_task_run/migration.sql(1 hunks)internal-packages/database/prisma/schema.prisma(1 hunks)internal-packages/run-engine/src/engine/index.ts(2 hunks)internal-packages/run-engine/src/engine/systems/runAttemptSystem.ts(2 hunks)internal-packages/run-engine/src/engine/types.ts(1 hunks)packages/cli-v3/src/entryPoints/dev-run-worker.ts(5 hunks)packages/cli-v3/src/entryPoints/managed-run-worker.ts(5 hunks)packages/core/package.json(1 hunks)packages/core/src/v3/apiClient/index.ts(9 hunks)packages/core/src/v3/apiClient/runStream.ts(8 hunks)packages/core/src/v3/apiClientManager/index.ts(1 hunks)packages/core/src/v3/apiClientManager/types.ts(2 hunks)packages/core/src/v3/index.ts(1 hunks)packages/core/src/v3/realtime-streams-api.ts(1 hunks)packages/core/src/v3/realtimeStreams/index.ts(1 hunks)packages/core/src/v3/realtimeStreams/manager.ts(1 hunks)packages/core/src/v3/realtimeStreams/noopManager.ts(1 hunks)packages/core/src/v3/realtimeStreams/streamsWriterV1.ts(1 hunks)packages/core/src/v3/realtimeStreams/streamsWriterV2.ts(1 hunks)packages/core/src/v3/realtimeStreams/types.ts(1 hunks)packages/core/src/v3/runMetadata/manager.ts(4 hunks)packages/core/src/v3/runMetadata/metadataStream.ts(0 hunks)packages/core/src/v3/schemas/api.ts(2 hunks)packages/core/src/v3/schemas/common.ts(1 hunks)packages/core/src/v3/semanticInternalAttributes.ts(1 hunks)packages/core/src/v3/streams/asyncIterableStream.ts(1 hunks)packages/core/src/v3/utils/globals.ts(2 hunks)packages/core/src/v3/waitUntil/index.ts(2 hunks)packages/core/src/v3/waitUntil/manager.ts(2 hunks)packages/core/src/v3/waitUntil/types.ts(1 hunks)packages/core/src/v3/workers/index.ts(1 hunks)packages/core/src/v3/workers/taskExecutor.ts(1 hunks)packages/core/test/runStream.test.ts(10 hunks)packages/core/test/streamsWriterV1.test.ts(1 hunks)packages/react-hooks/src/hooks/useRealtime.ts(4 hunks)packages/trigger-sdk/src/v3/index.ts(1 hunks)packages/trigger-sdk/src/v3/metadata.ts(2 hunks)packages/trigger-sdk/src/v3/shared.ts(9 hunks)packages/trigger-sdk/src/v3/streams.ts(1 hunks)turbo.json(1 hunks)
💤 Files with no reviewable changes (4)
- apps/webapp/app/services/realtime/relayRealtimeStreams.server.ts
- apps/webapp/prisma/seedCloud.ts
- packages/core/src/v3/runMetadata/metadataStream.ts
- apps/webapp/prisma/seed.ts
🚧 Files skipped from review as they are similar to previous changes (41)
- apps/webapp/app/v3/environmentVariables/environmentVariablesRepository.server.ts
- docker/config/toxiproxy.json
- packages/core/src/v3/workers/taskExecutor.ts
- packages/core/src/v3/utils/globals.ts
- apps/webapp/app/assets/icons/MoveToBottomIcon.tsx
- apps/webapp/app/presenters/v3/SpanPresenter.server.ts
- apps/webapp/app/v3/services/triggerTask.server.ts
- apps/webapp/app/assets/icons/SnakedArrowIcon.tsx
- packages/core/src/v3/waitUntil/index.ts
- apps/webapp/app/services/realtimeClient.server.ts
- packages/core/package.json
- packages/core/src/v3/waitUntil/types.ts
- packages/core/src/v3/schemas/api.ts
- packages/core/src/v3/apiClientManager/index.ts
- packages/core/src/v3/realtime-streams-api.ts
- internal-packages/database/prisma/migrations/20251020121543_add_realtime_streams_version_to_task_run/migration.sql
- packages/core/src/v3/realtimeStreams/noopManager.ts
- packages/core/src/v3/realtimeStreams/manager.ts
- packages/core/src/v3/semanticInternalAttributes.ts
- packages/core/src/v3/index.ts
- packages/core/src/v3/apiClientManager/types.ts
- apps/webapp/app/assets/icons/StreamsIcon.tsx
- apps/webapp/seed.mts
- apps/webapp/app/services/realtime/v1StreamsGlobal.server.ts
- packages/core/src/v3/workers/index.ts
- packages/core/src/v3/realtimeStreams/types.ts
- apps/webapp/package.json
- apps/webapp/app/utils/pathBuilder.ts
- internal-packages/run-engine/src/engine/index.ts
- internal-packages/database/prisma/schema.prisma
- turbo.json
- apps/webapp/app/v3/eventRepository/eventRepository.server.ts
- packages/core/src/v3/schemas/common.ts
- internal-packages/database/prisma/migrations/20251020163612_add_realtime_streams_to_task_run/migration.sql
- packages/core/src/v3/realtimeStreams/streamsWriterV2.ts
- packages/react-hooks/src/hooks/useRealtime.ts
- packages/trigger-sdk/src/v3/streams.ts
- apps/webapp/app/runEngine/services/triggerTask.server.ts
- docker/config/nginx.conf
- packages/trigger-sdk/src/v3/shared.ts
- apps/webapp/app/v3/services/replayTaskRun.server.ts
🧰 Additional context used
📓 Path-based instructions (8)
**/*.{ts,tsx}
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
**/*.{ts,tsx}: Always prefer using isomorphic code like fetch, ReadableStream, etc. instead of Node.js specific code
For TypeScript, we usually use types over interfaces
Avoid enums
No default exports, use function declarations
Files:
apps/webapp/app/components/runs/v3/RunIcon.tsxpackages/core/src/v3/streams/asyncIterableStream.tsapps/webapp/app/v3/eventRepository/eventRepository.types.tsapps/webapp/app/routes/api.v1.tasks.$taskId.trigger.tsapps/webapp/test/redisRealtimeStreams.test.tspackages/trigger-sdk/src/v3/index.tsapps/webapp/app/routes/realtime.v1.streams.$runId.$streamId.tsapps/webapp/app/routes/resources.orgs.$organizationSlug.projects.$projectParam.env.$envParam.runs.$runParam.spans.$spanParam/route.tsxpackages/core/src/v3/waitUntil/manager.tspackages/core/test/runStream.test.tspackages/cli-v3/src/entryPoints/managed-run-worker.tspackages/core/test/streamsWriterV1.test.tsapps/webapp/app/services/realtime/redisRealtimeStreams.server.tspackages/core/src/v3/runMetadata/manager.tsinternal-packages/run-engine/src/engine/systems/runAttemptSystem.tspackages/core/src/v3/realtimeStreams/streamsWriterV1.tsapps/webapp/app/services/realtime/s2realtimeStreams.server.tsapps/webapp/app/routes/realtime.v1.streams.$runId.$target.$streamId.tspackages/core/src/v3/apiClient/runStream.tsapps/webapp/app/models/organization.server.tsapps/webapp/app/services/realtime/types.tsapps/webapp/app/v3/eventRepository/clickhouseEventRepository.server.tspackages/core/src/v3/realtimeStreams/index.tsapps/webapp/app/routes/resources.orgs.$organizationSlug.projects.$projectParam.env.$envParam.runs.$runParam.streams.$streamKey/route.tsxapps/webapp/app/assets/icons/ListBulletIcon.tsxinternal-packages/run-engine/src/engine/types.tsapps/webapp/app/env.server.tspackages/cli-v3/src/entryPoints/dev-run-worker.tspackages/trigger-sdk/src/v3/metadata.tspackages/core/src/v3/apiClient/index.ts
{packages/core,apps/webapp}/**/*.{ts,tsx}
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
We use zod a lot in packages/core and in the webapp
Files:
apps/webapp/app/components/runs/v3/RunIcon.tsxpackages/core/src/v3/streams/asyncIterableStream.tsapps/webapp/app/v3/eventRepository/eventRepository.types.tsapps/webapp/app/routes/api.v1.tasks.$taskId.trigger.tsapps/webapp/test/redisRealtimeStreams.test.tsapps/webapp/app/routes/realtime.v1.streams.$runId.$streamId.tsapps/webapp/app/routes/resources.orgs.$organizationSlug.projects.$projectParam.env.$envParam.runs.$runParam.spans.$spanParam/route.tsxpackages/core/src/v3/waitUntil/manager.tspackages/core/test/runStream.test.tspackages/core/test/streamsWriterV1.test.tsapps/webapp/app/services/realtime/redisRealtimeStreams.server.tspackages/core/src/v3/runMetadata/manager.tspackages/core/src/v3/realtimeStreams/streamsWriterV1.tsapps/webapp/app/services/realtime/s2realtimeStreams.server.tsapps/webapp/app/routes/realtime.v1.streams.$runId.$target.$streamId.tspackages/core/src/v3/apiClient/runStream.tsapps/webapp/app/models/organization.server.tsapps/webapp/app/services/realtime/types.tsapps/webapp/app/v3/eventRepository/clickhouseEventRepository.server.tspackages/core/src/v3/realtimeStreams/index.tsapps/webapp/app/routes/resources.orgs.$organizationSlug.projects.$projectParam.env.$envParam.runs.$runParam.streams.$streamKey/route.tsxapps/webapp/app/assets/icons/ListBulletIcon.tsxapps/webapp/app/env.server.tspackages/core/src/v3/apiClient/index.ts
apps/webapp/**/*.{ts,tsx}
📄 CodeRabbit inference engine (.cursor/rules/webapp.mdc)
When importing from @trigger.dev/core in the webapp, never import the root package path; always use one of the documented subpath exports from @trigger.dev/core’s package.json
Files:
apps/webapp/app/components/runs/v3/RunIcon.tsxapps/webapp/app/v3/eventRepository/eventRepository.types.tsapps/webapp/app/routes/api.v1.tasks.$taskId.trigger.tsapps/webapp/test/redisRealtimeStreams.test.tsapps/webapp/app/routes/realtime.v1.streams.$runId.$streamId.tsapps/webapp/app/routes/resources.orgs.$organizationSlug.projects.$projectParam.env.$envParam.runs.$runParam.spans.$spanParam/route.tsxapps/webapp/app/services/realtime/redisRealtimeStreams.server.tsapps/webapp/app/services/realtime/s2realtimeStreams.server.tsapps/webapp/app/routes/realtime.v1.streams.$runId.$target.$streamId.tsapps/webapp/app/models/organization.server.tsapps/webapp/app/services/realtime/types.tsapps/webapp/app/v3/eventRepository/clickhouseEventRepository.server.tsapps/webapp/app/routes/resources.orgs.$organizationSlug.projects.$projectParam.env.$envParam.runs.$runParam.streams.$streamKey/route.tsxapps/webapp/app/assets/icons/ListBulletIcon.tsxapps/webapp/app/env.server.ts
apps/webapp/app/**/*.ts
📄 CodeRabbit inference engine (.cursor/rules/webapp.mdc)
Modules intended for test consumption under apps/webapp/app/**/*.ts must not read environment variables; accept configuration via options instead
Files:
apps/webapp/app/v3/eventRepository/eventRepository.types.tsapps/webapp/app/routes/api.v1.tasks.$taskId.trigger.tsapps/webapp/app/routes/realtime.v1.streams.$runId.$streamId.tsapps/webapp/app/services/realtime/redisRealtimeStreams.server.tsapps/webapp/app/services/realtime/s2realtimeStreams.server.tsapps/webapp/app/routes/realtime.v1.streams.$runId.$target.$streamId.tsapps/webapp/app/models/organization.server.tsapps/webapp/app/services/realtime/types.tsapps/webapp/app/v3/eventRepository/clickhouseEventRepository.server.tsapps/webapp/app/env.server.ts
{apps/webapp/app/**/*.server.{ts,tsx},apps/webapp/app/routes/**/*.ts}
📄 CodeRabbit inference engine (.cursor/rules/webapp.mdc)
Access environment variables only via the env export from app/env.server.ts; do not reference process.env directly
Files:
apps/webapp/app/routes/api.v1.tasks.$taskId.trigger.tsapps/webapp/app/routes/realtime.v1.streams.$runId.$streamId.tsapps/webapp/app/services/realtime/redisRealtimeStreams.server.tsapps/webapp/app/services/realtime/s2realtimeStreams.server.tsapps/webapp/app/routes/realtime.v1.streams.$runId.$target.$streamId.tsapps/webapp/app/models/organization.server.tsapps/webapp/app/v3/eventRepository/clickhouseEventRepository.server.tsapps/webapp/app/env.server.ts
**/*.test.{ts,tsx}
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
Our tests are all vitest
Files:
apps/webapp/test/redisRealtimeStreams.test.tspackages/core/test/runStream.test.tspackages/core/test/streamsWriterV1.test.ts
{apps/webapp/**/__tests__/**/*.{ts,tsx},apps/webapp/**/*.{test,spec}.{ts,tsx}}
📄 CodeRabbit inference engine (.cursor/rules/webapp.mdc)
{apps/webapp/**/__tests__/**/*.{ts,tsx},apps/webapp/**/*.{test,spec}.{ts,tsx}}: Do not import app/env.server.ts into tests, either directly or indirectly
Tests should only import classes/functions from files under apps/webapp/app/**/*.ts
Files:
apps/webapp/test/redisRealtimeStreams.test.ts
**/*.{test,spec}.{ts,tsx,js,jsx}
📄 CodeRabbit inference engine (AGENTS.md)
**/*.{test,spec}.{ts,tsx,js,jsx}: Unit tests must use Vitest
Tests should avoid mocks or stubs and use helpers from @internal/testcontainers when Redis or Postgres are needed
Test files live beside the files under test and should use descriptive describe and it blocks
Files:
apps/webapp/test/redisRealtimeStreams.test.tspackages/core/test/runStream.test.tspackages/core/test/streamsWriterV1.test.ts
🧠 Learnings (24)
📚 Learning: 2025-08-18T10:07:17.368Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-08-18T10:07:17.368Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Do not use client.defineJob or any deprecated v2 patterns (e.g., eventTrigger) when defining tasks
Applied to files:
apps/webapp/app/routes/api.v1.tasks.$taskId.trigger.tspackages/trigger-sdk/src/v3/index.tsinternal-packages/run-engine/src/engine/types.tspackages/core/src/v3/apiClient/index.ts
📚 Learning: 2025-08-18T10:07:17.368Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-08-18T10:07:17.368Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Import Trigger.dev APIs from "trigger.dev/sdk/v3" when writing tasks or related utilities
Applied to files:
apps/webapp/app/routes/api.v1.tasks.$taskId.trigger.tspackages/trigger-sdk/src/v3/index.tspackages/cli-v3/src/entryPoints/managed-run-worker.tsinternal-packages/run-engine/src/engine/types.tspackages/cli-v3/src/entryPoints/dev-run-worker.tspackages/trigger-sdk/src/v3/metadata.tspackages/core/src/v3/apiClient/index.ts
📚 Learning: 2025-08-18T10:07:17.368Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-08-18T10:07:17.368Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Export every task (including subtasks) defined with task(), schedules.task(), or schemaTask()
Applied to files:
apps/webapp/app/routes/api.v1.tasks.$taskId.trigger.tspackages/trigger-sdk/src/v3/index.ts
📚 Learning: 2025-08-18T10:07:17.368Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-08-18T10:07:17.368Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Define tasks using task({ id, run, ... }) with a unique id per project
Applied to files:
apps/webapp/app/routes/api.v1.tasks.$taskId.trigger.ts
📚 Learning: 2025-08-18T10:07:17.368Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-08-18T10:07:17.368Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Use schemaTask({ schema, run, ... }) to validate payloads when input validation is required
Applied to files:
apps/webapp/app/routes/api.v1.tasks.$taskId.trigger.ts
📚 Learning: 2025-08-18T10:07:17.368Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-08-18T10:07:17.368Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : For idempotent child-task invocations, create and pass idempotencyKey (and optional TTL) when calling trigger()/batchTrigger() from tasks
Applied to files:
apps/webapp/app/routes/api.v1.tasks.$taskId.trigger.ts
📚 Learning: 2025-08-18T10:07:17.368Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-08-18T10:07:17.368Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Use schedules.task(...) for scheduled (cron) tasks; do not implement schedules as plain task() with external cron logic
Applied to files:
apps/webapp/app/routes/api.v1.tasks.$taskId.trigger.ts
📚 Learning: 2025-08-18T10:07:17.368Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-08-18T10:07:17.368Z
Learning: Applies to trigger.config.ts : Configure global task lifecycle hooks (onStart/onSuccess/onFailure) only within trigger.config.ts if needed, not within arbitrary files
Applied to files:
apps/webapp/app/routes/api.v1.tasks.$taskId.trigger.ts
📚 Learning: 2025-08-18T10:07:17.368Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-08-18T10:07:17.368Z
Learning: Applies to trigger.config.ts : Declare build options and extensions (external, jsx, conditions, extensions) via the build block in trigger.config.ts rather than custom scripts
Applied to files:
apps/webapp/app/routes/api.v1.tasks.$taskId.trigger.tsinternal-packages/run-engine/src/engine/types.ts
📚 Learning: 2025-08-18T10:07:17.368Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-08-18T10:07:17.368Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Inside tasks, prefer logger.debug/log/info/warn/error over ad-hoc console logging for structured logs
Applied to files:
apps/webapp/app/routes/api.v1.tasks.$taskId.trigger.ts
📚 Learning: 2025-08-29T15:49:22.406Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-08-29T15:49:22.406Z
Learning: Applies to **/*.{test,spec}.{ts,tsx,js,jsx} : Tests should avoid mocks or stubs and use helpers from internal/testcontainers when Redis or Postgres are needed
Applied to files:
apps/webapp/test/redisRealtimeStreams.test.ts
📚 Learning: 2025-07-18T17:49:24.468Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .github/copilot-instructions.md:0-0
Timestamp: 2025-07-18T17:49:24.468Z
Learning: Applies to **/*.test.{ts,tsx} : Our tests are all vitest
Applied to files:
apps/webapp/test/redisRealtimeStreams.test.tspackages/core/test/runStream.test.tspackages/core/test/streamsWriterV1.test.ts
📚 Learning: 2025-08-29T10:06:49.293Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/webapp.mdc:0-0
Timestamp: 2025-08-29T10:06:49.293Z
Learning: Applies to apps/webapp/**/*.{ts,tsx} : When importing from trigger.dev/core in the webapp, never import the root package path; always use one of the documented subpath exports from trigger.dev/core’s package.json
Applied to files:
packages/trigger-sdk/src/v3/index.ts
📚 Learning: 2025-07-18T17:49:24.468Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .github/copilot-instructions.md:0-0
Timestamp: 2025-07-18T17:49:24.468Z
Learning: Applies to **/*.{ts,tsx} : No default exports, use function declarations
Applied to files:
packages/trigger-sdk/src/v3/index.ts
📚 Learning: 2025-07-18T17:49:24.468Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .github/copilot-instructions.md:0-0
Timestamp: 2025-07-18T17:49:24.468Z
Learning: Applies to **/*.{ts,tsx} : Always prefer using isomorphic code like fetch, ReadableStream, etc. instead of Node.js specific code
Applied to files:
packages/trigger-sdk/src/v3/index.tspackages/core/src/v3/apiClient/runStream.ts
📚 Learning: 2025-08-29T15:49:22.406Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-08-29T15:49:22.406Z
Learning: Applies to **/*.{test,spec}.{ts,tsx,js,jsx} : Unit tests must use Vitest
Applied to files:
packages/core/test/runStream.test.ts
📚 Learning: 2025-08-29T15:49:22.406Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-08-29T15:49:22.406Z
Learning: Applies to **/*.{test,spec}.{ts,tsx,js,jsx} : Test files live beside the files under test and should use descriptive describe and it blocks
Applied to files:
packages/core/test/runStream.test.ts
📚 Learning: 2025-08-18T10:07:17.368Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-08-18T10:07:17.368Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Use metadata API (metadata.current/get/set/append/stream, etc.) only inside run functions or lifecycle hooks
Applied to files:
packages/cli-v3/src/entryPoints/managed-run-worker.tspackages/cli-v3/src/entryPoints/dev-run-worker.tspackages/trigger-sdk/src/v3/metadata.ts
📚 Learning: 2025-09-03T14:34:41.781Z
Learnt from: myftija
Repo: triggerdotdev/trigger.dev PR: 2464
File: apps/webapp/app/routes/_app.orgs.$organizationSlug.projects.$projectParam.env.$envParam.settings/route.tsx:357-371
Timestamp: 2025-09-03T14:34:41.781Z
Learning: When using Zod's safeParse, the .data property is undefined when parsing fails, but TypeScript may still complain about accessing .data without checking .success first. The suggested approach of checking .success before accessing .data improves type safety and code clarity.
Applied to files:
packages/core/src/v3/apiClient/runStream.ts
📚 Learning: 2025-08-29T10:06:49.293Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/webapp.mdc:0-0
Timestamp: 2025-08-29T10:06:49.293Z
Learning: Applies to {apps/webapp/app/**/*.server.{ts,tsx},apps/webapp/app/routes/**/*.ts} : Access environment variables only via the env export from app/env.server.ts; do not reference process.env directly
Applied to files:
apps/webapp/app/env.server.ts
📚 Learning: 2025-08-29T10:06:49.293Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/webapp.mdc:0-0
Timestamp: 2025-08-29T10:06:49.293Z
Learning: Applies to {apps/webapp/**/__tests__/**/*.{ts,tsx},apps/webapp/**/*.{test,spec}.{ts,tsx}} : Do not import app/env.server.ts into tests, either directly or indirectly
Applied to files:
apps/webapp/app/env.server.ts
📚 Learning: 2025-08-29T10:06:49.293Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/webapp.mdc:0-0
Timestamp: 2025-08-29T10:06:49.293Z
Learning: Applies to apps/webapp/app/**/*.ts : Modules intended for test consumption under apps/webapp/app/**/*.ts must not read environment variables; accept configuration via options instead
Applied to files:
apps/webapp/app/env.server.ts
📚 Learning: 2025-08-14T12:13:20.455Z
Learnt from: myftija
Repo: triggerdotdev/trigger.dev PR: 2392
File: packages/cli-v3/src/utilities/gitMeta.ts:195-218
Timestamp: 2025-08-14T12:13:20.455Z
Learning: In the GitMeta schema (packages/core/src/v3/schemas/common.ts), all fields are intentionally optional to handle partial data from various deployment contexts (local, GitHub Actions, GitHub App). Functions like getGitHubAppMeta() are designed to work with missing environment variables rather than validate their presence.
Applied to files:
apps/webapp/app/env.server.ts
📚 Learning: 2025-10-08T11:48:12.327Z
Learnt from: nicktrn
Repo: triggerdotdev/trigger.dev PR: 2593
File: packages/core/src/v3/workers/warmStartClient.ts:168-170
Timestamp: 2025-10-08T11:48:12.327Z
Learning: The trigger.dev runners execute only in Node 21 and 22 environments, so modern Node.js APIs like AbortSignal.any (introduced in v20.3.0) are supported.
Applied to files:
packages/trigger-sdk/src/v3/metadata.ts
🧬 Code graph analysis (19)
apps/webapp/app/components/runs/v3/RunIcon.tsx (1)
apps/webapp/app/assets/icons/StreamsIcon.tsx (1)
StreamsIcon(1-9)
apps/webapp/test/redisRealtimeStreams.test.ts (2)
internal-packages/testcontainers/src/index.ts (1)
redisTest(167-167)apps/webapp/app/services/realtime/redisRealtimeStreams.server.ts (1)
RedisRealtimeStreams(23-413)
apps/webapp/app/routes/realtime.v1.streams.$runId.$streamId.ts (1)
apps/webapp/app/services/realtime/v1StreamsGlobal.server.ts (1)
getRealtimeStreamInstance(25-52)
apps/webapp/app/routes/resources.orgs.$organizationSlug.projects.$projectParam.env.$envParam.runs.$runParam.spans.$spanParam/route.tsx (1)
apps/webapp/app/routes/resources.orgs.$organizationSlug.projects.$projectParam.env.$envParam.runs.$runParam.streams.$streamKey/route.tsx (1)
RealtimeStreamViewer(96-386)
packages/core/test/runStream.test.ts (2)
packages/core/src/v3/apiClient/index.ts (1)
SSEStreamPart(147-147)packages/core/src/v3/apiClient/runStream.ts (1)
SSEStreamPart(175-179)
packages/cli-v3/src/entryPoints/managed-run-worker.ts (6)
packages/core/src/v3/runMetadata/manager.ts (1)
StandardMetadataManager(12-452)packages/core/src/v3/apiClientManager-api.ts (1)
apiClientManager(5-5)packages/core/src/v3/run-metadata-api.ts (1)
runMetadata(5-5)packages/core/src/v3/realtimeStreams/manager.ts (1)
StandardRealtimeStreamsManager(16-141)packages/core/src/v3/realtime-streams-api.ts (1)
realtimeStreams(5-5)packages/core/src/v3/waitUntil/manager.ts (1)
StandardWaitUntilManager(3-40)
packages/core/test/streamsWriterV1.test.ts (1)
packages/core/src/v3/realtimeStreams/streamsWriterV1.ts (1)
StreamsWriterV1(26-466)
apps/webapp/app/services/realtime/redisRealtimeStreams.server.ts (2)
packages/core/src/logger.ts (3)
Logger(19-139)LogLevel(15-15)error(69-79)apps/webapp/app/services/realtime/types.ts (3)
StreamIngestor(2-17)StreamResponder(25-33)StreamResponseOptions(19-22)
packages/core/src/v3/runMetadata/manager.ts (1)
packages/core/src/v3/realtime-streams-api.ts (1)
realtimeStreams(5-5)
packages/core/src/v3/realtimeStreams/streamsWriterV1.ts (1)
packages/core/src/v3/realtimeStreams/types.ts (1)
StreamsWriter(23-25)
apps/webapp/app/services/realtime/s2realtimeStreams.server.ts (1)
apps/webapp/app/services/realtime/types.ts (3)
StreamResponder(25-33)StreamIngestor(2-17)StreamResponseOptions(19-22)
apps/webapp/app/routes/realtime.v1.streams.$runId.$target.$streamId.ts (4)
apps/webapp/app/services/realtime/v1StreamsGlobal.server.ts (1)
getRealtimeStreamInstance(25-52)apps/webapp/app/routes/realtime.v1.streams.$runId.$streamId.ts (1)
loader(11-71)apps/webapp/app/services/routeBuilders/apiBuilder.server.ts (1)
createLoaderApiRoute(97-287)apps/webapp/app/db.server.ts (1)
$replica(103-106)
apps/webapp/app/services/realtime/types.ts (2)
packages/core/src/v3/runMetadata/manager.ts (1)
stream(297-303)packages/core/src/v3/apiClient/stream.ts (1)
stream(180-182)
apps/webapp/app/v3/eventRepository/clickhouseEventRepository.server.ts (1)
packages/core/src/v3/semanticInternalAttributes.ts (1)
SemanticInternalAttributes(1-68)
packages/core/src/v3/realtimeStreams/index.ts (3)
packages/core/src/v3/realtimeStreams/noopManager.ts (1)
NoopRealtimeStreamsManager(11-24)packages/core/src/v3/realtimeStreams/types.ts (3)
RealtimeStreamsManager(10-16)RealtimeAppendStreamOptions(4-8)RealtimeStreamInstance(18-21)packages/core/src/v3/utils/globals.ts (2)
registerGlobal(22-38)getGlobal(40-44)
apps/webapp/app/routes/resources.orgs.$organizationSlug.projects.$projectParam.env.$envParam.runs.$runParam.streams.$streamKey/route.tsx (8)
apps/webapp/app/services/session.server.ts (1)
requireUserId(25-35)apps/webapp/app/utils/pathBuilder.ts (1)
v3RunStreamParamsSchema(43-45)apps/webapp/app/db.server.ts (1)
$replica(103-106)apps/webapp/app/services/realtime/v1StreamsGlobal.server.ts (1)
getRealtimeStreamInstance(25-52)apps/webapp/app/hooks/useOrganizations.ts (1)
useOrganization(39-43)apps/webapp/app/hooks/useProject.tsx (1)
useProject(20-24)apps/webapp/app/hooks/useEnvironment.tsx (1)
useEnvironment(19-23)packages/core/src/v3/apiClient/runStream.ts (2)
SSEStreamPart(175-179)SSEStreamSubscription(182-376)
packages/cli-v3/src/entryPoints/dev-run-worker.ts (6)
packages/core/src/v3/apiClientManager-api.ts (1)
apiClientManager(5-5)packages/core/src/v3/run-metadata-api.ts (1)
runMetadata(5-5)packages/core/src/v3/realtimeStreams/manager.ts (1)
StandardRealtimeStreamsManager(16-141)packages/core/src/v3/utils/getEnv.ts (2)
getEnvVar(11-13)getNumberEnvVar(15-29)packages/core/src/v3/realtime-streams-api.ts (1)
realtimeStreams(5-5)packages/core/src/v3/waitUntil/manager.ts (1)
StandardWaitUntilManager(3-40)
packages/trigger-sdk/src/v3/metadata.ts (5)
packages/core/src/v3/runMetadata/manager.ts (1)
stream(297-303)packages/core/src/v3/apiClient/stream.ts (1)
stream(180-182)packages/core/src/v3/runMetadata/noopManager.ts (1)
stream(19-21)packages/core/src/v3/runMetadata/index.ts (1)
stream(80-86)packages/trigger-sdk/src/v3/streams.ts (1)
streams(158-161)
packages/core/src/v3/apiClient/index.ts (4)
packages/core/src/v3/apiClientManager/types.ts (1)
ApiClientConfiguration(3-19)packages/core/src/v3/apiClient/runStream.ts (2)
SSEStreamSubscriptionFactory(378-403)SSEStreamPart(175-179)packages/core/src/v3/schemas/api.ts (2)
CreateStreamResponseBody(1310-1312)CreateStreamResponseBody(1313-1313)packages/core/src/v3/utils/getEnv.ts (1)
getEnvVar(11-13)
🪛 Biome (2.1.2)
packages/core/test/streamsWriterV1.test.ts
[error] 792-795: This generator function doesn't contain yield.
(lint/correctness/useYield)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (23)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (5, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (6, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (4, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (8, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (7, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (3, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (3, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (2, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (7, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (4, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (8, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (5, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (1, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (6, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (1, 8)
- GitHub Check: units / packages / 🧪 Unit Tests: Packages (1, 1)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (2, 8)
- GitHub Check: typecheck / typecheck
- GitHub Check: e2e / 🧪 CLI v3 tests (windows-latest - npm)
- GitHub Check: e2e / 🧪 CLI v3 tests (ubuntu-latest - npm)
- GitHub Check: e2e / 🧪 CLI v3 tests (ubuntu-latest - pnpm)
- GitHub Check: e2e / 🧪 CLI v3 tests (windows-latest - pnpm)
- GitHub Check: Analyze (javascript-typescript)
There was a problem hiding this comment.
Actionable comments posted: 1
♻️ Duplicate comments (2)
packages/react-hooks/src/hooks/useRealtime.ts (2)
1026-1068: Fix startIndex handling to allow explicit 0 value.Line 1043 uses
startIndex ? ...which treats0as falsy. When a user explicitly passesstartIndex: 0to read from the beginning, this will be ignored and behave as ifstartIndexwas undefined. This breaks the documented API behavior.const stream = await apiClient.fetchStream<TPart>(runId, streamKey, { signal: abortControllerRef.current?.signal, timeoutInSeconds, - lastEventId: startIndex ? (startIndex - 1).toString() : undefined, + lastEventId: startIndex != null ? (startIndex - 1).toString() : undefined, });
794-843: Extract specific option values to prevent unnecessary re-subscriptions.Including the entire
optsobject in the dependency array (line 843) will causetriggerRequestto be recreated wheneveroptsis a new object reference—even if the actual values haven't changed. This is common when passing inline option objects (as shown in the JSDoc examples), and will cause the subscription to restart on every render.Extract the specific values before the callbacks:
const apiClient = useApiClient(opts); + const timeoutInSeconds = opts?.timeoutInSeconds; + const startIndex = opts?.startIndex; + const throttleInMs = opts?.throttleInMs ?? 16; + const onDataCallback = opts?.onData; + + const onData = useCallback( + (data: TPart) => { + if (onDataCallback) { + onDataCallback(data); + } + }, + [onDataCallback] + ); - const onData = useCallback( - (data: TPart) => { - if (opts?.onData) { - opts.onData(data); - } - }, - [opts?.onData] - ); - - const apiClient = useApiClient(opts); const triggerRequest = useCallback(async () => { try { if (!runId || !apiClient) { return; } const abortController = new AbortController(); abortControllerRef.current = abortController; await processRealtimeStream<TPart>( runId, streamKey, apiClient, mutateParts, partsRef, setError, onData, abortControllerRef, - opts?.timeoutInSeconds, - opts?.startIndex, - opts?.throttleInMs ?? 16 + timeoutInSeconds, + startIndex, + throttleInMs ); } catch (err) { // Ignore abort errors as they are expected. if ((err as any).name === "AbortError") { abortControllerRef.current = null; return; } setError(err as Error); } finally { if (abortControllerRef.current) { abortControllerRef.current = null; } // Mark the subscription as complete setIsComplete(true); } - }, [runId, streamKey, mutateParts, partsRef, abortControllerRef, apiClient, setError, onData, opts]); + }, [runId, streamKey, mutateParts, partsRef, abortControllerRef, apiClient, setError, onData, timeoutInSeconds, startIndex, throttleInMs]);
🧹 Nitpick comments (1)
packages/react-hooks/src/hooks/useRealtime.ts (1)
845-862: Consider includingstreamKeyin effect deps for clarity (optional).While the current implementation is functionally correct (changes to
streamKeytrigger the effect viatriggerRequest), React best practices recommend including all conceptually-relevant dependencies directly in the dependency array for clarity and maintainability.return () => { stop(); }; - }, [runId, stop, opts?.enabled, triggerRequest]); + }, [runId, streamKey, stop, opts?.enabled, triggerRequest]);
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (4)
references/realtime-streams/src/components/ai-chat.tsxis excluded by!references/**references/realtime-streams/src/components/streams.tsxis excluded by!references/**references/realtime-streams/src/trigger/ai-chat.tsis excluded by!references/**references/realtime-streams/src/trigger/streams.tsis excluded by!references/**
📒 Files selected for processing (2)
packages/react-hooks/src/hooks/useRealtime.ts(4 hunks)packages/trigger-sdk/src/v3/streams.ts(1 hunks)
🧰 Additional context used
📓 Path-based instructions (1)
**/*.{ts,tsx}
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
**/*.{ts,tsx}: Always prefer using isomorphic code like fetch, ReadableStream, etc. instead of Node.js specific code
For TypeScript, we usually use types over interfaces
Avoid enums
No default exports, use function declarations
Files:
packages/react-hooks/src/hooks/useRealtime.tspackages/trigger-sdk/src/v3/streams.ts
🧠 Learnings (3)
📚 Learning: 2025-08-18T10:07:17.368Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-08-18T10:07:17.368Z
Learning: For Realtime subscriptions or React hooks, provide a Public Access Token and scope it appropriately (e.g., via TriggerAuthContext)
Applied to files:
packages/react-hooks/src/hooks/useRealtime.ts
📚 Learning: 2025-08-18T10:07:17.368Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-08-18T10:07:17.368Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Use metadata API (metadata.current/get/set/append/stream, etc.) only inside run functions or lifecycle hooks
Applied to files:
packages/react-hooks/src/hooks/useRealtime.ts
📚 Learning: 2025-08-18T10:07:17.368Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-08-18T10:07:17.368Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Import Trigger.dev APIs from "trigger.dev/sdk/v3" when writing tasks or related utilities
Applied to files:
packages/trigger-sdk/src/v3/streams.ts
🧬 Code graph analysis (2)
packages/react-hooks/src/hooks/useRealtime.ts (3)
packages/react-hooks/src/hooks/useApiClient.ts (2)
UseApiClientOptions(9-25)useApiClient(47-67)packages/core/src/v3/apiClient/index.ts (1)
ApiClient(155-1221)packages/react-hooks/src/utils/throttle.ts (1)
createThrottledQueue(8-58)
packages/trigger-sdk/src/v3/streams.ts (10)
packages/core/src/v3/apiClient/index.ts (2)
ApiRequestOptions(141-141)mergeRequestOptions(1390-1406)packages/core/src/v3/streams/asyncIterableStream.ts (1)
AsyncIterableStream(1-1)packages/core/src/v3/realtimeStreams/manager.ts (1)
append(32-101)packages/core/src/v3/realtimeStreams/index.ts (1)
append(34-40)packages/core/src/v3/semanticInternalAttributes.ts (1)
SemanticInternalAttributes(1-68)packages/core/src/v3/index.ts (1)
SemanticInternalAttributes(24-24)packages/core/src/v3/realtime-streams-api.ts (1)
realtimeStreams(5-5)packages/core/src/v3/apiClientManager-api.ts (1)
apiClientManager(5-5)packages/core/src/v3/realtimeStreams/types.ts (1)
RealtimeAppendStreamOptions(4-8)packages/core/src/v3/task-context-api.ts (1)
taskContext(5-5)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (23)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (1, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (7, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (6, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (8, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (4, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (5, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (3, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (3, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (2, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (8, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (7, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (5, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (6, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (4, 8)
- GitHub Check: e2e / 🧪 CLI v3 tests (ubuntu-latest - npm)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (1, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (2, 8)
- GitHub Check: units / packages / 🧪 Unit Tests: Packages (1, 1)
- GitHub Check: e2e / 🧪 CLI v3 tests (windows-latest - pnpm)
- GitHub Check: e2e / 🧪 CLI v3 tests (ubuntu-latest - pnpm)
- GitHub Check: e2e / 🧪 CLI v3 tests (windows-latest - npm)
- GitHub Check: typecheck / typecheck
- GitHub Check: Analyze (javascript-typescript)
🔇 Additional comments (10)
packages/trigger-sdk/src/v3/streams.ts (6)
1-14: LGTM!Imports are comprehensive and correctly structured for the streams SDK module.
16-58: LGTM!Type definitions are clear, well-documented, and provide appropriate flexibility for stream operations.
145-228: LGTM!The
appendimplementation correctly handles:
- Overload parameter resolution
- Span lifecycle management via
instance.wait().finally()- Special AbortError handling (span ends without exception)
- Comprehensive error recording for non-abort errors
233-255: LGTM!
ReadStreamOptionstype is well-documented with clear defaults and appropriate configuration options.
333-354: LGTM!Clean overload resolution and delegation to
readStreamImpl. The pattern is consistent with theappendfunction.
400-423: LGTM!Clean namespace export and helper function. The
getRunIdForOptionshelper correctly resolves special target values ("parent","root","self") and provides appropriate fallback behavior.packages/react-hooks/src/hooks/useRealtime.ts (4)
18-23: LGTM!Clear documentation for the throttling parameter with appropriate default value.
291-291: LGTM!Correctly applies the throttle parameter with the default value.
581-620: LGTM!Type definitions are well-structured and comprehensively documented. The hook properly extends
UseApiClientOptionsto support authentication via access tokens.
622-752: LGTM!Excellent API design with clear overloads and comprehensive documentation. The overload resolution cleanly handles both use cases.
ericallam
force-pushed
the
feat/realtime-streams-2
branch
from
2da5fe2 to
e8c35a1
Compare
There was a problem hiding this comment.
Actionable comments posted: 3
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
apps/webapp/app/services/realtime/redisRealtimeStreams.server.ts (1)
263-296: Fix resume chunk indexing to avoid duplicate chunkIndex valuesOn resumptions we should emit the next chunk index, but the current code restarts at the last acknowledged index. This duplicates chunkIndex values, confusing clients that rely on monotonic indexes (e.g. resume logic, reconciliation). Increment the starting index before writing.
Apply this diff:
- const startChunk = resumeFromChunk ?? 0; - // Start counting from the resume point, not from 0 - let currentChunkIndex = startChunk; + const resumeIndex = resumeFromChunk; + // Start from the chunk immediately after the resume point (or 0 when starting fresh) + let currentChunkIndex = typeof resumeIndex === "number" ? resumeIndex + 1 : 0; @@ - resumeFromChunk: startChunk, + resumeFromChunk: resumeIndex ?? null,
♻️ Duplicate comments (6)
apps/webapp/app/routes/realtime.v1.streams.$runId.$target.$streamId.ts (4)
21-39: Pull realtime stream version from the DB, not the client header.We never select
realtimeStreamsVersion, so the handler can’t rely on the authoritative run metadata and falls back to the untrustedX-Stream-Versionheader later in the file. That lets clients route v2 traffic into the v1 backend (and vice versa), which corrupts streams and triggers 500s when S2 runs hit the Redis path. Extend thisselectto includerealtimeStreamsVersionso the action can enforce the correct backend.
95-123: Reject non‑v1 server ingest and stop trustingX-Stream-Version.On the POST path we still read
X-Stream-Versiondirectly and pass it togetRealtimeStreamInstance(). S2 (v2) ingest throws (ingestDatais unimplemented), so any v2 client request returns 500. Worse, a client can lie about the version and wedge data into the wrong backend. Userun.realtimeStreamsVersioninstead and short-circuit unless it’s"v1"(return 400 otherwise) before instantiating the ingestor.Apply something along these lines:
- const clientId = request.headers.get("X-Client-Id") || "default"; - const streamVersion = request.headers.get("X-Stream-Version") || "v1"; + const clientId = request.headers.get("X-Client-Id") || "default"; + const streamVersion = run.realtimeStreamsVersion ?? "v1"; + if (streamVersion !== "v1") { + return new Response("Server-side ingest is only supported for v1 streams", { status: 400 }); + }
139-152: Loader also needsrealtimeStreamsVersion.The loader’s
findResourcequery mirrors the action’s select but still omitsrealtimeStreamsVersion. Without it the HEAD handler can’t fall back to the authoritative version and again trusts the header later, leading to the same cross-version routing bugs. Include the field here too so the loader can enforce the correct backend.
177-187: Use the run’s stream version in the HEAD handler.For HEAD requests we again use
X-Stream-Version, which lets clients poke the wrong backend and crashes whengetLastChunkIndexis invoked on the S2 implementation (it throws). Replace this withrun.realtimeStreamsVersion ?? "v1"so we always hit the correct manager.packages/core/src/v3/realtimeStreams/manager.ts (1)
46-48: User's abort signal is ignored whenAbortSignal.anyis unavailable.When
AbortSignal.anyis undefined, the optional chaining producesundefinedand the fallback uses onlyabortController.signal, silently droppingoptions?.signal. Users in older environments cannot abort their streams.Apply this diff to manually combine signals when
anyis unavailable:// Create an AbortController for this stream const abortController = new AbortController(); // Chain with user-provided signal if present - const combinedSignal = options?.signal - ? AbortSignal.any?.([options.signal, abortController.signal]) ?? abortController.signal - : abortController.signal; + let combinedSignal: AbortSignal; + if (options?.signal) { + if (AbortSignal.any) { + combinedSignal = AbortSignal.any([options.signal, abortController.signal]); + } else { + // Manual combination: forward abort from user signal to our controller + combinedSignal = abortController.signal; + options.signal.addEventListener("abort", () => abortController.abort(), { once: true }); + } + } else { + combinedSignal = abortController.signal; + }packages/react-hooks/src/hooks/useRealtime.ts (1)
1069-1069: Fix: startIndex of 0 is treated as falsy.
startIndex ? ...treats0as falsy, so an explicitstartIndex: 0incorrectly produceslastEventId: undefinedinstead of"-1". Users cannot explicitly start from the beginning.Apply this diff:
- lastEventId: startIndex ? (startIndex - 1).toString() : undefined, + lastEventId: startIndex != null ? (startIndex - 1).toString() : undefined,
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (7)
pnpm-lock.yamlis excluded by!**/pnpm-lock.yamlreferences/realtime-streams/src/app/streams.tsis excluded by!references/**references/realtime-streams/src/components/ai-chat.tsxis excluded by!references/**references/realtime-streams/src/components/performance-monitor.tsxis excluded by!references/**references/realtime-streams/src/components/streams.tsxis excluded by!references/**references/realtime-streams/src/trigger/ai-chat.tsis excluded by!references/**references/realtime-streams/src/trigger/streams.tsis excluded by!references/**
📒 Files selected for processing (18)
apps/webapp/app/routes/realtime.v1.streams.$runId.$target.$streamId.append.ts(1 hunks)apps/webapp/app/routes/realtime.v1.streams.$runId.$target.$streamId.ts(2 hunks)apps/webapp/app/services/realtime/redisRealtimeStreams.server.ts(6 hunks)apps/webapp/app/services/realtime/s2realtimeStreams.server.ts(1 hunks)apps/webapp/app/services/realtime/types.ts(1 hunks)packages/core/package.json(1 hunks)packages/core/src/v3/apiClient/index.ts(10 hunks)packages/core/src/v3/realtimeStreams/index.ts(1 hunks)packages/core/src/v3/realtimeStreams/manager.ts(1 hunks)packages/core/src/v3/realtimeStreams/noopManager.ts(1 hunks)packages/core/src/v3/realtimeStreams/streamInstance.ts(1 hunks)packages/core/src/v3/realtimeStreams/streamsWriterV2.ts(1 hunks)packages/core/src/v3/realtimeStreams/types.ts(1 hunks)packages/core/src/v3/runMetadata/manager.ts(4 hunks)packages/core/src/v3/schemas/api.ts(2 hunks)packages/react-hooks/src/hooks/useRealtime.ts(5 hunks)packages/trigger-sdk/src/v3/metadata.ts(2 hunks)packages/trigger-sdk/src/v3/streams.ts(1 hunks)
✅ Files skipped from review due to trivial changes (1)
- packages/trigger-sdk/src/v3/streams.ts
🚧 Files skipped from review as they are similar to previous changes (4)
- packages/core/package.json
- apps/webapp/app/services/realtime/types.ts
- packages/core/src/v3/schemas/api.ts
- packages/core/src/v3/realtimeStreams/noopManager.ts
🧰 Additional context used
📓 Path-based instructions (5)
**/*.{ts,tsx}
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
**/*.{ts,tsx}: Always prefer using isomorphic code like fetch, ReadableStream, etc. instead of Node.js specific code
For TypeScript, we usually use types over interfaces
Avoid enums
No default exports, use function declarations
Files:
packages/trigger-sdk/src/v3/metadata.tspackages/core/src/v3/realtimeStreams/streamsWriterV2.tsapps/webapp/app/routes/realtime.v1.streams.$runId.$target.$streamId.append.tsapps/webapp/app/routes/realtime.v1.streams.$runId.$target.$streamId.tsapps/webapp/app/services/realtime/s2realtimeStreams.server.tspackages/core/src/v3/apiClient/index.tspackages/core/src/v3/runMetadata/manager.tspackages/core/src/v3/realtimeStreams/streamInstance.tsapps/webapp/app/services/realtime/redisRealtimeStreams.server.tspackages/core/src/v3/realtimeStreams/manager.tspackages/react-hooks/src/hooks/useRealtime.tspackages/core/src/v3/realtimeStreams/types.tspackages/core/src/v3/realtimeStreams/index.ts
{packages/core,apps/webapp}/**/*.{ts,tsx}
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
We use zod a lot in packages/core and in the webapp
Files:
packages/core/src/v3/realtimeStreams/streamsWriterV2.tsapps/webapp/app/routes/realtime.v1.streams.$runId.$target.$streamId.append.tsapps/webapp/app/routes/realtime.v1.streams.$runId.$target.$streamId.tsapps/webapp/app/services/realtime/s2realtimeStreams.server.tspackages/core/src/v3/apiClient/index.tspackages/core/src/v3/runMetadata/manager.tspackages/core/src/v3/realtimeStreams/streamInstance.tsapps/webapp/app/services/realtime/redisRealtimeStreams.server.tspackages/core/src/v3/realtimeStreams/manager.tspackages/core/src/v3/realtimeStreams/types.tspackages/core/src/v3/realtimeStreams/index.ts
apps/webapp/**/*.{ts,tsx}
📄 CodeRabbit inference engine (.cursor/rules/webapp.mdc)
When importing from @trigger.dev/core in the webapp, never import the root package path; always use one of the documented subpath exports from @trigger.dev/core’s package.json
Files:
apps/webapp/app/routes/realtime.v1.streams.$runId.$target.$streamId.append.tsapps/webapp/app/routes/realtime.v1.streams.$runId.$target.$streamId.tsapps/webapp/app/services/realtime/s2realtimeStreams.server.tsapps/webapp/app/services/realtime/redisRealtimeStreams.server.ts
{apps/webapp/app/**/*.server.{ts,tsx},apps/webapp/app/routes/**/*.ts}
📄 CodeRabbit inference engine (.cursor/rules/webapp.mdc)
Access environment variables only via the env export from app/env.server.ts; do not reference process.env directly
Files:
apps/webapp/app/routes/realtime.v1.streams.$runId.$target.$streamId.append.tsapps/webapp/app/routes/realtime.v1.streams.$runId.$target.$streamId.tsapps/webapp/app/services/realtime/s2realtimeStreams.server.tsapps/webapp/app/services/realtime/redisRealtimeStreams.server.ts
apps/webapp/app/**/*.ts
📄 CodeRabbit inference engine (.cursor/rules/webapp.mdc)
Modules intended for test consumption under apps/webapp/app/**/*.ts must not read environment variables; accept configuration via options instead
Files:
apps/webapp/app/routes/realtime.v1.streams.$runId.$target.$streamId.append.tsapps/webapp/app/routes/realtime.v1.streams.$runId.$target.$streamId.tsapps/webapp/app/services/realtime/s2realtimeStreams.server.tsapps/webapp/app/services/realtime/redisRealtimeStreams.server.ts
🧠 Learnings (6)
📚 Learning: 2025-08-18T10:07:17.368Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-08-18T10:07:17.368Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Use metadata API (metadata.current/get/set/append/stream, etc.) only inside run functions or lifecycle hooks
Applied to files:
packages/trigger-sdk/src/v3/metadata.tspackages/react-hooks/src/hooks/useRealtime.ts
📚 Learning: 2025-08-18T10:07:17.368Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-08-18T10:07:17.368Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Import Trigger.dev APIs from "trigger.dev/sdk/v3" when writing tasks or related utilities
Applied to files:
packages/trigger-sdk/src/v3/metadata.tspackages/react-hooks/src/hooks/useRealtime.ts
📚 Learning: 2025-10-08T11:48:12.327Z
Learnt from: nicktrn
Repo: triggerdotdev/trigger.dev PR: 2593
File: packages/core/src/v3/workers/warmStartClient.ts:168-170
Timestamp: 2025-10-08T11:48:12.327Z
Learning: The trigger.dev runners execute only in Node 21 and 22 environments, so modern Node.js APIs like AbortSignal.any (introduced in v20.3.0) are supported.
Applied to files:
packages/trigger-sdk/src/v3/metadata.tspackages/core/src/v3/realtimeStreams/manager.ts
📚 Learning: 2024-10-18T15:41:52.352Z
Learnt from: nicktrn
Repo: triggerdotdev/trigger.dev PR: 1418
File: packages/core/src/v3/errors.ts:364-371
Timestamp: 2024-10-18T15:41:52.352Z
Learning: In `packages/core/src/v3/errors.ts`, within the `taskRunErrorEnhancer` function, `error.message` is always defined, so it's safe to directly call `error.message.includes("SIGTERM")` without additional checks.
Applied to files:
packages/core/src/v3/runMetadata/manager.ts
📚 Learning: 2025-07-18T17:49:24.468Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .github/copilot-instructions.md:0-0
Timestamp: 2025-07-18T17:49:24.468Z
Learning: Applies to **/*.{ts,tsx} : Always prefer using isomorphic code like fetch, ReadableStream, etc. instead of Node.js specific code
Applied to files:
packages/core/src/v3/realtimeStreams/manager.tspackages/core/src/v3/realtimeStreams/types.ts
📚 Learning: 2025-08-18T10:07:17.368Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-08-18T10:07:17.368Z
Learning: For Realtime subscriptions or React hooks, provide a Public Access Token and scope it appropriately (e.g., via TriggerAuthContext)
Applied to files:
packages/react-hooks/src/hooks/useRealtime.ts
🧬 Code graph analysis (13)
packages/trigger-sdk/src/v3/metadata.ts (5)
packages/core/src/v3/realtimeStreams/streamInstance.ts (1)
stream(69-94)packages/core/src/v3/runMetadata/manager.ts (1)
stream(297-303)packages/core/src/v3/apiClient/stream.ts (1)
stream(180-182)packages/core/src/v3/runMetadata/index.ts (1)
stream(80-86)packages/trigger-sdk/src/v3/streams.ts (1)
streams(405-411)
packages/core/src/v3/realtimeStreams/streamsWriterV2.ts (2)
packages/core/src/v3/realtimeStreams/types.ts (1)
StreamsWriter(29-31)packages/core/src/v3/realtimeStreams/streamInstance.ts (1)
stream(69-94)
apps/webapp/app/routes/realtime.v1.streams.$runId.$target.$streamId.append.ts (2)
apps/webapp/app/db.server.ts (1)
$replica(103-106)apps/webapp/app/services/realtime/v1StreamsGlobal.server.ts (1)
getRealtimeStreamInstance(25-52)
apps/webapp/app/routes/realtime.v1.streams.$runId.$target.$streamId.ts (3)
apps/webapp/app/services/realtime/v1StreamsGlobal.server.ts (1)
getRealtimeStreamInstance(25-52)apps/webapp/app/services/routeBuilders/apiBuilder.server.ts (1)
createLoaderApiRoute(97-287)apps/webapp/app/db.server.ts (1)
$replica(103-106)
apps/webapp/app/services/realtime/s2realtimeStreams.server.ts (1)
apps/webapp/app/services/realtime/types.ts (3)
StreamResponder(27-35)StreamIngestor(2-19)StreamResponseOptions(21-24)
packages/core/src/v3/apiClient/index.ts (7)
packages/core/src/v3/apiClientManager/index.ts (2)
ApiClientConfiguration(100-102)accessToken(37-45)packages/core/src/v3/apiClientManager/types.ts (1)
ApiClientConfiguration(3-19)packages/core/src/v3/apiClient/core.ts (3)
ZodFetchOptions(31-39)ApiRequestOptions(43-43)zodfetch(71-78)packages/core/src/v3/streams/asyncIterableStream.ts (1)
AsyncIterableStream(1-1)packages/core/src/v3/realtimeStreams/streamInstance.ts (1)
stream(69-94)packages/core/src/v3/schemas/api.ts (4)
CreateStreamResponseBody(1310-1312)CreateStreamResponseBody(1313-1313)AppendToStreamResponseBody(1315-1318)AppendToStreamResponseBody(1319-1319)packages/core/src/v3/utils/getEnv.ts (1)
getEnvVar(11-13)
packages/core/src/v3/runMetadata/manager.ts (2)
packages/core/src/v3/apiClient/index.ts (1)
ApiClient(156-1241)packages/core/src/v3/realtime-streams-api.ts (1)
realtimeStreams(5-5)
packages/core/src/v3/realtimeStreams/streamInstance.ts (7)
packages/core/src/v3/apiClient/index.ts (1)
headers(1229-1240)packages/core/src/v3/apiClient/core.ts (1)
AnyZodFetchOptions(41-41)packages/core/src/v3/realtimeStreams/streamsWriterV1.ts (1)
StreamsWriterV1(26-466)packages/core/src/v3/realtimeStreams/streamsWriterV2.ts (1)
StreamsWriterV2(45-221)packages/core/src/v3/realtimeStreams/types.ts (1)
StreamsWriter(29-31)packages/core/src/v3/streams/asyncIterableStream.ts (1)
AsyncIterableStream(1-1)packages/core/src/v3/runMetadata/manager.ts (1)
stream(297-303)
apps/webapp/app/services/realtime/redisRealtimeStreams.server.ts (3)
packages/core/src/logger.ts (3)
Logger(19-139)LogLevel(15-15)error(69-79)apps/webapp/app/services/realtime/types.ts (3)
StreamIngestor(2-19)StreamResponder(27-35)StreamResponseOptions(21-24)apps/webapp/app/env.server.ts (1)
env(1219-1219)
packages/core/src/v3/realtimeStreams/manager.ts (5)
packages/core/src/v3/realtimeStreams/types.ts (3)
RealtimeStreamsManager(10-22)RealtimeStreamOperationOptions(4-8)RealtimeStreamInstance(24-27)packages/core/src/v3/apiClient/index.ts (1)
headers(1229-1240)packages/core/src/v3/streams/asyncIterableStream.ts (1)
ensureAsyncIterable(107-135)packages/core/src/v3/realtimeStreams/streamInstance.ts (2)
StreamInstance(22-95)stream(69-94)packages/core/src/v3/task-context-api.ts (1)
taskContext(5-5)
packages/react-hooks/src/hooks/useRealtime.ts (3)
packages/react-hooks/src/hooks/useApiClient.ts (2)
UseApiClientOptions(9-25)useApiClient(47-67)packages/core/src/v3/realtimeStreams/types.ts (2)
RealtimeDefinedStream(33-42)InferStreamType(44-44)packages/react-hooks/src/utils/throttle.ts (1)
createThrottledQueue(8-58)
packages/core/src/v3/realtimeStreams/types.ts (6)
packages/core/src/v3/apiClient/core.ts (1)
AnyZodFetchOptions(41-41)packages/core/src/v3/realtimeStreams/index.ts (1)
RealtimeStreamsManager(30-32)packages/core/src/v3/streams/asyncIterableStream.ts (1)
AsyncIterableStream(1-1)packages/core/src/v3/apiClient/index.ts (1)
ApiRequestOptions(142-142)packages/core/src/v3/realtimeStreams/streamInstance.ts (1)
stream(69-94)packages/core/src/v3/runMetadata/manager.ts (1)
stream(297-303)
packages/core/src/v3/realtimeStreams/index.ts (3)
packages/core/src/v3/realtimeStreams/noopManager.ts (1)
NoopRealtimeStreamsManager(11-30)packages/core/src/v3/realtimeStreams/types.ts (3)
RealtimeStreamsManager(10-22)RealtimeStreamOperationOptions(4-8)RealtimeStreamInstance(24-27)packages/core/src/v3/utils/globals.ts (2)
registerGlobal(22-38)getGlobal(40-44)
🪛 Biome (2.1.2)
packages/react-hooks/src/hooks/useRealtime.ts
[error] 747-747: This hook is being called conditionally, but all hooks must be called in the exact same order in every component render.
For React to preserve state between calls, hooks needs to be called unconditionally and always in the same order.
See https://reactjs.org/docs/hooks-rules.html#only-call-hooks-at-the-top-level
(lint/correctness/useHookAtTopLevel)
[error] 753-753: This hook is being called conditionally, but all hooks must be called in the exact same order in every component render.
For React to preserve state between calls, hooks needs to be called unconditionally and always in the same order.
See https://reactjs.org/docs/hooks-rules.html#only-call-hooks-at-the-top-level
(lint/correctness/useHookAtTopLevel)
[error] 761-761: This hook is being called conditionally, but all hooks must be called in the exact same order in every component render.
For React to preserve state between calls, hooks needs to be called unconditionally and always in the same order.
See https://reactjs.org/docs/hooks-rules.html#only-call-hooks-at-the-top-level
(lint/correctness/useHookAtTopLevel)
[error] 779-779: This hook is being called indirectly and conditionally, but all hooks must be called in the exact same order in every component render.
This is the call path until the hook.
For React to preserve state between calls, hooks needs to be called unconditionally and always in the same order.
See https://reactjs.org/docs/hooks-rules.html#only-call-hooks-at-the-top-level
(lint/correctness/useHookAtTopLevel)
[error] 782-782: This hook is being called indirectly and conditionally, but all hooks must be called in the exact same order in every component render.
This is the call path until the hook.
For React to preserve state between calls, hooks needs to be called unconditionally and always in the same order.
See https://reactjs.org/docs/hooks-rules.html#only-call-hooks-at-the-top-level
(lint/correctness/useHookAtTopLevel)
[error] 785-785: This hook is being called indirectly and conditionally, but all hooks must be called in the exact same order in every component render.
This is the call path until the hook.
For React to preserve state between calls, hooks needs to be called unconditionally and always in the same order.
See https://reactjs.org/docs/hooks-rules.html#only-call-hooks-at-the-top-level
(lint/correctness/useHookAtTopLevel)
[error] 794-794: This hook is being called indirectly and conditionally, but all hooks must be called in the exact same order in every component render.
This is the call path until the hook.
For React to preserve state between calls, hooks needs to be called unconditionally and always in the same order.
See https://reactjs.org/docs/hooks-rules.html#only-call-hooks-at-the-top-level
(lint/correctness/useHookAtTopLevel)
[error] 795-795: This hook is being called indirectly and conditionally, but all hooks must be called in the exact same order in every component render.
This is the call path until the hook.
For React to preserve state between calls, hooks needs to be called unconditionally and always in the same order.
See https://reactjs.org/docs/hooks-rules.html#only-call-hooks-at-the-top-level
(lint/correctness/useHookAtTopLevel)
[error] 800-800: This hook is being called indirectly and conditionally, but all hooks must be called in the exact same order in every component render.
This is the call path until the hook.
For React to preserve state between calls, hooks needs to be called unconditionally and always in the same order.
See https://reactjs.org/docs/hooks-rules.html#only-call-hooks-at-the-top-level
(lint/correctness/useHookAtTopLevel)
[error] 805-805: This hook is being called indirectly and conditionally, but all hooks must be called in the exact same order in every component render.
This is the call path until the hook.
For React to preserve state between calls, hooks needs to be called unconditionally and always in the same order.
See https://reactjs.org/docs/hooks-rules.html#only-call-hooks-at-the-top-level
(lint/correctness/useHookAtTopLevel)
[error] 811-811: This hook is being called indirectly and conditionally, but all hooks must be called in the exact same order in every component render.
This is the call path until the hook.
For React to preserve state between calls, hooks needs to be called unconditionally and always in the same order.
See https://reactjs.org/docs/hooks-rules.html#only-call-hooks-at-the-top-level
(lint/correctness/useHookAtTopLevel)
[error] 813-813: This hook is being called indirectly and conditionally, but all hooks must be called in the exact same order in every component render.
This is the call path until the hook.
For React to preserve state between calls, hooks needs to be called unconditionally and always in the same order.
See https://reactjs.org/docs/hooks-rules.html#only-call-hooks-at-the-top-level
(lint/correctness/useHookAtTopLevel)
[error] 820-820: This hook is being called indirectly and conditionally, but all hooks must be called in the exact same order in every component render.
This is the call path until the hook.
For React to preserve state between calls, hooks needs to be called unconditionally and always in the same order.
See https://reactjs.org/docs/hooks-rules.html#only-call-hooks-at-the-top-level
(lint/correctness/useHookAtTopLevel)
[error] 829-829: This hook is being called indirectly and conditionally, but all hooks must be called in the exact same order in every component render.
This is the call path until the hook.
For React to preserve state between calls, hooks needs to be called unconditionally and always in the same order.
See https://reactjs.org/docs/hooks-rules.html#only-call-hooks-at-the-top-level
(lint/correctness/useHookAtTopLevel)
[error] 831-831: This hook is being called indirectly and conditionally, but all hooks must be called in the exact same order in every component render.
This is the call path until the hook.
For React to preserve state between calls, hooks needs to be called unconditionally and always in the same order.
See https://reactjs.org/docs/hooks-rules.html#only-call-hooks-at-the-top-level
(lint/correctness/useHookAtTopLevel)
[error] 871-871: This hook is being called indirectly and conditionally, but all hooks must be called in the exact same order in every component render.
This is the call path until the hook.
For React to preserve state between calls, hooks needs to be called unconditionally and always in the same order.
See https://reactjs.org/docs/hooks-rules.html#only-call-hooks-at-the-top-level
(lint/correctness/useHookAtTopLevel)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (5, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (7, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (1, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (4, 8)
- GitHub Check: units / packages / 🧪 Unit Tests: Packages (1, 1)
- GitHub Check: e2e / 🧪 CLI v3 tests (windows-latest - npm)
- GitHub Check: e2e / 🧪 CLI v3 tests (windows-latest - pnpm)
- GitHub Check: Analyze (javascript-typescript)
🔇 Additional comments (11)
packages/trigger-sdk/src/v3/metadata.ts (3)
10-10: LGTM!The import statement correctly brings in the new streams API needed for the implementation update.
232-234: LGTM!The deprecation annotation is clear and provides a straightforward migration path to
streams.pipe().
235-245: No issues found - implementation is backward compatible.The function returns
AsyncIterableStream<T>, which is defined asAsyncIterable<T> & ReadableStream<T>. This type is fully compatible with the declared return typePromise<AsyncIterable<T>>. All callers across the codebase (14+ usages) correctly await the promise and iterate withfor await...of, which all continue to work correctly with the new implementation.packages/core/src/v3/realtimeStreams/manager.ts (4)
1-26: LGTM! Clean initialization and tracking structure.The Set-based activeStreams tracking is a solid improvement that naturally allows multiple concurrent streams without key collisions.
75-103: LGTM! Clean append logic and validation.The error handling and runId validation are appropriate.
105-138: Excellent timeout and error handling.The
TIMEOUT_SENTINELpattern withPromise.all(which propagates rejections) is a clean solution that properly detects timeouts and surfaces stream failures.
141-198: LGTM! Helpers are well-structured.
getRunIdForOptionsproperly resolves all target types, andparseCreateStreamResponsegracefully handles both v1 and v2 responses.packages/react-hooks/src/hooks/useRealtime.ts (4)
17-26: LGTM! Well-documented throttle option.The JSDoc clearly documents the default behavior.
583-622: LGTM! Well-typed stream hook API.The types are comprehensive and consistently documented.
774-888: Implementation looks solid.The hook properly manages state, cleanup, and subscriptions using SWR and React patterns.
1072-1094: LGTM! Proper throttling and error handling.The throttled queue batches updates efficiently, and error handling correctly distinguishes abort from other failures.
There was a problem hiding this comment.
Actionable comments posted: 1
♻️ Duplicate comments (3)
packages/core/src/v3/realtimeStreams/streamInstance.ts (2)
30-49: Honor the requested target run.
StreamInstanceOptions.target(line 16) is never used: both thecreateStreamcall (line 32) and theStreamsWriterV1constructor (line 49) are hard-coded to"self". This breaks the documented targeting feature—streams.append(..., { target: "parent" | "root" | custom })silently writes to the wrong run.Apply this diff to respect the target option:
+ const target = this.options.target ?? "self"; const { version, headers } = await this.options.apiClient.createStream( this.options.runId, - "self", + target, this.options.key, this.options?.requestOptions ); @@ headers: this.options.apiClient.getHeaders(), signal: this.options.signal, version, - target: "self", + target, })
79-80: Close the controller when aborted.When the abort signal fires, the loop exits via
breakwithout closing the controller. Consumers awaitingfor await (...)will hang indefinitely. Close the stream and optionally calliterator.return()for proper cleanup.Apply this diff:
while (true) { if (self.options.signal?.aborted) { + controller.close(); + if (typeof iterator.return === "function") { + await iterator.return(); + } - break; + return; }packages/core/src/v3/realtimeStreams/streamsWriterV1.ts (1)
66-85: Surface buffer errors to the sender to prevent unhandled rejections.The buffer task (lines 66-85) rethrows errors but is never awaited, risking
unhandledRejectionwarnings. The send loop (lines 220-254) remains unaware of buffer failures, potentially causing silent data loss. Capture the error in an instance field, release the reader lock in a finally block, and have the send loop check and propagate it.Apply this diff:
+ private bufferError: unknown; @@ private startBuffering(): void { this.streamReader = this.serverStream.getReader(); this.bufferReaderTask = (async () => { try { let chunkIndex = 0; while (true) { + if (this.options.signal?.aborted) { + this.streamComplete = true; + break; + } const { done, value } = await this.streamReader!.read(); if (done) { this.streamComplete = true; break; } // Add to ring buffer this.addToRingBuffer(chunkIndex, value); this.highestBufferedIndex = chunkIndex; chunkIndex++; } } catch (error) { - throw error; + this.bufferError = error; + this.streamComplete = true; + } finally { + try { + this.streamReader?.releaseLock(); + } catch {} + this.streamReader = null; } })(); + this.bufferReaderTask.catch(() => {}); // prevent unhandled rejection }Then in
processStream(around line 224):const processStream = async () => { try { let lastSentIndex = startFromChunk - 1; while (true) { + if (this.bufferError) { + throw this.bufferError instanceof Error + ? this.bufferError + : new Error(String(this.bufferError)); + } // Send all chunks that are available in buffer
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (3)
pnpm-lock.yamlis excluded by!**/pnpm-lock.yamlreferences/realtime-streams/package.jsonis excluded by!references/**references/realtime-streams/src/trigger/ai-chat.tsis excluded by!references/**
📒 Files selected for processing (5)
packages/core/src/v3/realtimeStreams/manager.ts(1 hunks)packages/core/src/v3/realtimeStreams/streamInstance.ts(1 hunks)packages/core/src/v3/realtimeStreams/streamsWriterV1.ts(1 hunks)packages/core/src/v3/realtimeStreams/streamsWriterV2.ts(1 hunks)packages/core/src/v3/streams/asyncIterableStream.ts(1 hunks)
🧰 Additional context used
📓 Path-based instructions (2)
**/*.{ts,tsx}
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
**/*.{ts,tsx}: Always prefer using isomorphic code like fetch, ReadableStream, etc. instead of Node.js specific code
For TypeScript, we usually use types over interfaces
Avoid enums
No default exports, use function declarations
Files:
packages/core/src/v3/realtimeStreams/streamsWriterV1.tspackages/core/src/v3/streams/asyncIterableStream.tspackages/core/src/v3/realtimeStreams/streamInstance.tspackages/core/src/v3/realtimeStreams/streamsWriterV2.tspackages/core/src/v3/realtimeStreams/manager.ts
{packages/core,apps/webapp}/**/*.{ts,tsx}
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
We use zod a lot in packages/core and in the webapp
Files:
packages/core/src/v3/realtimeStreams/streamsWriterV1.tspackages/core/src/v3/streams/asyncIterableStream.tspackages/core/src/v3/realtimeStreams/streamInstance.tspackages/core/src/v3/realtimeStreams/streamsWriterV2.tspackages/core/src/v3/realtimeStreams/manager.ts
🧠 Learnings (2)
📚 Learning: 2025-07-18T17:49:24.468Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .github/copilot-instructions.md:0-0
Timestamp: 2025-07-18T17:49:24.468Z
Learning: Applies to **/*.{ts,tsx} : Always prefer using isomorphic code like fetch, ReadableStream, etc. instead of Node.js specific code
Applied to files:
packages/core/src/v3/realtimeStreams/manager.ts
📚 Learning: 2025-10-08T11:48:12.327Z
Learnt from: nicktrn
Repo: triggerdotdev/trigger.dev PR: 2593
File: packages/core/src/v3/workers/warmStartClient.ts:168-170
Timestamp: 2025-10-08T11:48:12.327Z
Learning: The trigger.dev runners execute only in Node 21 and 22 environments, so modern Node.js APIs like AbortSignal.any (introduced in v20.3.0) are supported.
Applied to files:
packages/core/src/v3/realtimeStreams/manager.ts
🧬 Code graph analysis (4)
packages/core/src/v3/realtimeStreams/streamsWriterV1.ts (2)
packages/core/src/v3/realtimeStreams/types.ts (1)
StreamsWriter(29-31)packages/core/src/v3/realtimeStreams/streamInstance.ts (1)
stream(69-94)
packages/core/src/v3/realtimeStreams/streamInstance.ts (7)
packages/core/src/v3/apiClient/index.ts (1)
headers(1229-1240)packages/core/src/v3/apiClient/core.ts (1)
AnyZodFetchOptions(41-41)packages/core/src/v3/realtimeStreams/streamsWriterV1.ts (1)
StreamsWriterV1(26-449)packages/core/src/v3/realtimeStreams/streamsWriterV2.ts (1)
StreamsWriterV2(45-196)packages/core/src/v3/realtimeStreams/types.ts (1)
StreamsWriter(29-31)packages/core/src/v3/streams/asyncIterableStream.ts (1)
AsyncIterableStream(1-1)packages/core/src/v3/runMetadata/manager.ts (1)
stream(297-303)
packages/core/src/v3/realtimeStreams/streamsWriterV2.ts (2)
packages/core/src/v3/realtimeStreams/types.ts (1)
StreamsWriter(29-31)packages/core/src/v3/realtimeStreams/streamInstance.ts (1)
stream(69-94)
packages/core/src/v3/realtimeStreams/manager.ts (5)
packages/core/src/v3/realtimeStreams/types.ts (3)
RealtimeStreamsManager(10-22)RealtimeStreamOperationOptions(4-8)RealtimeStreamInstance(24-27)packages/core/src/v3/apiClient/index.ts (1)
headers(1229-1240)packages/core/src/v3/streams/asyncIterableStream.ts (1)
ensureReadableStream(137-158)packages/core/src/v3/realtimeStreams/streamInstance.ts (2)
StreamInstance(22-95)stream(69-94)packages/core/src/v3/task-context-api.ts (1)
taskContext(5-5)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (23)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (2, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (6, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (6, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (8, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (5, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (7, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (3, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (1, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (4, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (2, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (4, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (7, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (5, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (1, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (8, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (3, 8)
- GitHub Check: units / packages / 🧪 Unit Tests: Packages (1, 1)
- GitHub Check: e2e / 🧪 CLI v3 tests (windows-latest - npm)
- GitHub Check: e2e / 🧪 CLI v3 tests (ubuntu-latest - npm)
- GitHub Check: e2e / 🧪 CLI v3 tests (windows-latest - pnpm)
- GitHub Check: e2e / 🧪 CLI v3 tests (ubuntu-latest - pnpm)
- GitHub Check: typecheck / typecheck
- GitHub Check: Analyze (javascript-typescript)
🔇 Additional comments (2)
packages/core/src/v3/streams/asyncIterableStream.ts (1)
107-158: LGTM! Clean bidirectional stream/iterable converters.Both
ensureAsyncIterableandensureReadableStreamcorrectly handle conversion between AsyncIterable and ReadableStream types. The implementations properly manage reader locks, guard against undefined values, and close controllers as expected.packages/core/src/v3/realtimeStreams/streamsWriterV2.ts (1)
45-196: LGTM! Well-structured S2 streaming implementation.The StreamsWriterV2 class properly:
- Tees the source for independent server/consumer processing
- Handles abort signals throughout the pipeline (constructor, transform, error paths)
- Safely manages the S2 appendSession lifecycle
- Releases reader locks in finally blocks
The implementation aligns well with the v2 streaming architecture.
There was a problem hiding this comment.
Actionable comments posted: 1
♻️ Duplicate comments (2)
packages/trigger-sdk/src/v3/streams.ts (1)
334-346: GuardfetchStreamso the span always ends.If
apiClient.fetchStreamthrows before invoking its callbacks, the span never records an error or ends, producing leaked spans. Wrap the call in a try/catch so we always record the failure, set the status, end the span, and rethrow. (tanstack.com)- return await apiClient.fetchStream(runId, key, { - signal: options?.signal, - timeoutInSeconds: options?.timeoutInSeconds ?? 60, - lastEventId: options?.startIndex ? (options.startIndex - 1).toString() : undefined, - onComplete: () => { - span.end(); - }, - onError: (error) => { - span.recordException(error); - span.setStatus({ code: SpanStatusCode.ERROR }); - span.end(); - }, - }); + try { + return await apiClient.fetchStream(runId, key, { + signal: options?.signal, + timeoutInSeconds: options?.timeoutInSeconds ?? 60, + lastEventId: options?.startIndex ? (options.startIndex - 1).toString() : undefined, + onComplete: () => { + span.end(); + }, + onError: (error) => { + span.recordException(error); + span.setStatus({ code: SpanStatusCode.ERROR }); + span.end(); + }, + }); + } catch (error) { + if (error instanceof Error && error.name === "AbortError") { + span.end(); + throw error; + } + + if (error instanceof Error || typeof error === "string") { + span.recordException(error); + } else { + span.recordException(String(error)); + } + + span.setStatus({ code: SpanStatusCode.ERROR }); + span.end(); + throw error; + }packages/core/test/streamsWriterV1.test.ts (1)
793-805: Fix empty generator to satisfy lint.Biome flags generators without a
yield, so this test will fail CI. Please keep the generator shape but add an unreachableyieldso the rule is satisfied. (next.biomejs.dev)- async function* generateChunks() { - // Yields nothing - return; - } + async function* generateChunks() { + // Yields nothing; keep generator shape for lint compliance + if (false) { + yield undefined as never; + } + }
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (3)
references/realtime-streams/src/components/ai-chat.tsxis excluded by!references/**references/realtime-streams/src/trigger/ai-chat.tsis excluded by!references/**references/realtime-streams/src/trigger/streams.tsis excluded by!references/**
📒 Files selected for processing (3)
packages/core/src/v3/realtimeStreams/types.ts(1 hunks)packages/core/test/streamsWriterV1.test.ts(1 hunks)packages/trigger-sdk/src/v3/streams.ts(1 hunks)
🧰 Additional context used
📓 Path-based instructions (4)
**/*.{ts,tsx}
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
**/*.{ts,tsx}: Always prefer using isomorphic code like fetch, ReadableStream, etc. instead of Node.js specific code
For TypeScript, we usually use types over interfaces
Avoid enums
No default exports, use function declarations
Files:
packages/core/test/streamsWriterV1.test.tspackages/trigger-sdk/src/v3/streams.tspackages/core/src/v3/realtimeStreams/types.ts
**/*.test.{ts,tsx}
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
Our tests are all vitest
Files:
packages/core/test/streamsWriterV1.test.ts
{packages/core,apps/webapp}/**/*.{ts,tsx}
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
We use zod a lot in packages/core and in the webapp
Files:
packages/core/test/streamsWriterV1.test.tspackages/core/src/v3/realtimeStreams/types.ts
**/*.{test,spec}.{ts,tsx,js,jsx}
📄 CodeRabbit inference engine (AGENTS.md)
**/*.{test,spec}.{ts,tsx,js,jsx}: Unit tests must use Vitest
Tests should avoid mocks or stubs and use helpers from @internal/testcontainers when Redis or Postgres are needed
Test files live beside the files under test and should use descriptive describe and it blocks
Files:
packages/core/test/streamsWriterV1.test.ts
🧠 Learnings (3)
📚 Learning: 2025-07-18T17:49:24.468Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .github/copilot-instructions.md:0-0
Timestamp: 2025-07-18T17:49:24.468Z
Learning: Applies to **/*.test.{ts,tsx} : Our tests are all vitest
Applied to files:
packages/core/test/streamsWriterV1.test.ts
📚 Learning: 2025-08-18T10:07:17.368Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-08-18T10:07:17.368Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Import Trigger.dev APIs from "trigger.dev/sdk/v3" when writing tasks or related utilities
Applied to files:
packages/trigger-sdk/src/v3/streams.ts
📚 Learning: 2025-07-18T17:49:24.468Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .github/copilot-instructions.md:0-0
Timestamp: 2025-07-18T17:49:24.468Z
Learning: Applies to **/*.{ts,tsx} : Always prefer using isomorphic code like fetch, ReadableStream, etc. instead of Node.js specific code
Applied to files:
packages/trigger-sdk/src/v3/streams.tspackages/core/src/v3/realtimeStreams/types.ts
🧬 Code graph analysis (3)
packages/core/test/streamsWriterV1.test.ts (2)
packages/core/src/v3/realtimeStreams/streamsWriterV1.ts (1)
StreamsWriterV1(26-449)packages/core/src/v3/streams/asyncIterableStream.ts (1)
ensureReadableStream(137-158)
packages/trigger-sdk/src/v3/streams.ts (11)
packages/core/src/v3/realtimeStreams/manager.ts (2)
pipe(27-73)append(75-99)packages/core/src/v3/realtimeStreams/index.ts (2)
pipe(34-40)append(42-48)packages/core/src/v3/realtimeStreams/noopManager.ts (2)
pipe(12-23)append(25-29)packages/core/src/v3/realtimeStreams/types.ts (7)
PipeStreamOptions(50-70)PipeStreamResult(77-89)ReadStreamOptions(94-116)AppendStreamOptions(121-136)WriterStreamOptions(138-145)RealtimeDefinedStream(34-43)RealtimeStreamOperationOptions(5-9)packages/core/src/v3/semanticInternalAttributes.ts (1)
SemanticInternalAttributes(1-68)packages/core/src/v3/apiClient/index.ts (1)
mergeRequestOptions(1410-1426)packages/core/src/v3/realtime-streams-api.ts (1)
realtimeStreams(5-5)packages/core/src/v3/streams/asyncIterableStream.ts (1)
AsyncIterableStream(1-1)packages/core/src/v3/apiClientManager-api.ts (1)
apiClientManager(5-5)packages/core/src/v3/realtimeStreams/streamInstance.ts (1)
stream(69-94)packages/core/src/v3/task-context-api.ts (1)
taskContext(5-5)
packages/core/src/v3/realtimeStreams/types.ts (7)
packages/core/src/v3/apiClient/core.ts (1)
AnyZodFetchOptions(41-41)packages/core/src/v3/realtimeStreams/index.ts (1)
RealtimeStreamsManager(30-32)packages/core/src/v3/streams/asyncIterableStream.ts (1)
AsyncIterableStream(1-1)packages/core/src/v3/apiClient/index.ts (1)
ApiRequestOptions(142-142)apps/webapp/app/lib.es5.d.ts (1)
Prettify(13-15)packages/core/src/v3/realtimeStreams/streamInstance.ts (1)
stream(69-94)packages/core/src/v3/runMetadata/manager.ts (1)
stream(297-303)
🪛 Biome (2.1.2)
packages/core/test/streamsWriterV1.test.ts
[error] 793-796: This generator function doesn't contain yield.
(lint/correctness/useYield)
packages/trigger-sdk/src/v3/streams.ts
[error] 444-449: Promise executor functions should not be async.
(lint/suspicious/noAsyncPromiseExecutor)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (23)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (7, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (8, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (5, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (4, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (6, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (3, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (5, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (2, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (1, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (4, 8)
- GitHub Check: units / packages / 🧪 Unit Tests: Packages (1, 1)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (7, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (8, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (2, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (3, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (1, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (6, 8)
- GitHub Check: e2e / 🧪 CLI v3 tests (ubuntu-latest - pnpm)
- GitHub Check: e2e / 🧪 CLI v3 tests (windows-latest - pnpm)
- GitHub Check: e2e / 🧪 CLI v3 tests (windows-latest - npm)
- GitHub Check: e2e / 🧪 CLI v3 tests (ubuntu-latest - npm)
- GitHub Check: typecheck / typecheck
- GitHub Check: Analyze (javascript-typescript)
There was a problem hiding this comment.
Actionable comments posted: 1
♻️ Duplicate comments (3)
apps/webapp/app/routes/resources.orgs.$organizationSlug.projects.$projectParam.env.$envParam.runs.$runParam.streams.$streamKey/route.tsx (3)
60-95: Authorize the requested streamKey against the run before streaming.The loader does not verify that the requested
streamKeybelongs to this run. A user could potentially access arbitrary stream keys by manipulating the URL.After loading the run and verifying the environment (line 82), add a check to ensure the streamKey exists for this run:
if (run.runtimeEnvironment.slug !== envParam) { throw new Response("Not Found", { status: 404 }); } + + // Enforce that the requested stream exists for this run + if (!Array.isArray(run.realtimeStreams) || !run.realtimeStreams.includes(streamKey)) { + throw new Response("Not Found", { status: 404 }); + } // Get Last-Event-ID header for resuming from a specific position const lastEventId = request.headers.get("Last-Event-ID") || undefined;
203-207: Fix invalid scrollIntoView ScrollBehavior value.The value
"instant"is not a validScrollBehaviortype. Valid values are"auto"and"smooth". This will cause TypeScript errors and potentially fail at runtime.Apply this diff:
useEffect(() => { if (isAtBottom && bottomRef.current) { - bottomRef.current.scrollIntoView({ behavior: "instant", block: "end" }); + bottomRef.current.scrollIntoView({ behavior: "auto", block: "end" }); } }, [chunks, isAtBottom]);
484-544: Fix resume bug: startIndex=0 is treated as falsy and drops Last-Event-ID.When
startIndexis0, the condition treats it as falsy and omits theLast-Event-IDheader, preventing correct resume from the beginning of the stream.Apply this diff:
const sseSubscription = new SSEStreamSubscription(resourcePath, { signal: abortController.signal, - lastEventId: startIndex ? (startIndex - 1).toString() : undefined, + lastEventId: startIndex !== undefined ? (startIndex - 1).toString() : undefined, timeoutInSeconds: 30, });
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (1)
references/realtime-streams/src/trigger/streams.tsis excluded by!references/**
📒 Files selected for processing (1)
apps/webapp/app/routes/resources.orgs.$organizationSlug.projects.$projectParam.env.$envParam.runs.$runParam.streams.$streamKey/route.tsx(1 hunks)
🧰 Additional context used
📓 Path-based instructions (3)
**/*.{ts,tsx}
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
**/*.{ts,tsx}: Always prefer using isomorphic code like fetch, ReadableStream, etc. instead of Node.js specific code
For TypeScript, we usually use types over interfaces
Avoid enums
No default exports, use function declarations
Files:
apps/webapp/app/routes/resources.orgs.$organizationSlug.projects.$projectParam.env.$envParam.runs.$runParam.streams.$streamKey/route.tsx
{packages/core,apps/webapp}/**/*.{ts,tsx}
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
We use zod a lot in packages/core and in the webapp
Files:
apps/webapp/app/routes/resources.orgs.$organizationSlug.projects.$projectParam.env.$envParam.runs.$runParam.streams.$streamKey/route.tsx
apps/webapp/**/*.{ts,tsx}
📄 CodeRabbit inference engine (.cursor/rules/webapp.mdc)
When importing from @trigger.dev/core in the webapp, never import the root package path; always use one of the documented subpath exports from @trigger.dev/core’s package.json
Files:
apps/webapp/app/routes/resources.orgs.$organizationSlug.projects.$projectParam.env.$envParam.runs.$runParam.streams.$streamKey/route.tsx
🧬 Code graph analysis (1)
apps/webapp/app/routes/resources.orgs.$organizationSlug.projects.$projectParam.env.$envParam.runs.$runParam.streams.$streamKey/route.tsx (7)
apps/webapp/app/services/session.server.ts (1)
requireUserId(25-35)apps/webapp/app/db.server.ts (1)
$replica(103-106)apps/webapp/app/services/realtime/v1StreamsGlobal.server.ts (1)
getRealtimeStreamInstance(25-52)apps/webapp/app/hooks/useOrganizations.ts (1)
useOrganization(39-43)apps/webapp/app/hooks/useProject.tsx (1)
useProject(20-24)apps/webapp/app/hooks/useEnvironment.tsx (1)
useEnvironment(19-23)packages/core/src/v3/apiClient/runStream.ts (2)
SSEStreamPart(175-179)SSEStreamSubscription(182-384)
...izationSlug.projects.$projectParam.env.$envParam.runs.$runParam.streams.$streamKey/route.tsx
Outdated
Show resolved
Hide resolved
ericallam
force-pushed
the
feat/realtime-streams-2
branch
from
e458818 to
186846b
Compare
There was a problem hiding this comment.
Actionable comments posted: 5
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (4)
apps/webapp/app/v3/eventRepository/eventRepository.server.ts (1)
780-787: Extract metadata from properties to match ClickHouse implementation.The entity extraction retrieves
typeandidusingrehydrateAttributefrom properties, butmetadatais hardcoded toundefined. This creates an inconsistency with the ClickHouse implementation (clickhouseEventRepository.server.ts, lines 1149-1154), which does extract and populate entity metadata.Since
SemanticInternalAttributes.ENTITY_METADATAexists and the ClickHouse implementation extracts it, this implementation should do the same.Apply this diff to extract metadata consistently:
const entity = { type: rehydrateAttribute<string>( spanEvent.properties, SemanticInternalAttributes.ENTITY_TYPE ), id: rehydrateAttribute<string>(spanEvent.properties, SemanticInternalAttributes.ENTITY_ID), - metadata: undefined, + metadata: rehydrateAttribute<string>( + spanEvent.properties, + SemanticInternalAttributes.ENTITY_METADATA + ), };packages/trigger-sdk/src/v3/shared.ts (2)
434-456: Correct the type ofrequestOptionsparameter toTriggerApiRequestOptions.Both
triggerAndWaitandbatchTriggerAndWaitdeclarerequestOptionsasApiRequestOptions, but pass it to internal functions that expectTriggerApiRequestOptionsand accessrequestOptions?.clientConfig. SinceApiRequestOptionsdoesn't include theclientConfigfield, this creates a type mismatch that could prevent proper API client configuration.Apply this diff to correct the types:
export function triggerAndWait<TTask extends AnyTask>( id: TaskIdentifier<TTask>, payload: TaskPayload<TTask>, options?: TriggerAndWaitOptions, - requestOptions?: ApiRequestOptions + requestOptions?: TriggerApiRequestOptions ): TaskRunPromise<TaskIdentifier<TTask>, TaskOutput<TTask>> {export async function batchTriggerAndWait<TTask extends AnyTask>( id: TaskIdentifier<TTask>, items: Array<BatchItem<TaskPayload<TTask>>>, options?: BatchTriggerAndWaitOptions, - requestOptions?: ApiRequestOptions + requestOptions?: TriggerApiRequestOptions ): Promise<BatchResult<TaskIdentifier<TTask>, TaskOutput<TTask>>> {Also applies to: 482-493
320-338: Add missingrequestOptionsparameter totriggerAndWaitmethod.The
triggerAndWaitmethod increateSchemaTaskdoesn't accept arequestOptionsparameter, unlike the same method increateTask(line 188). This creates an inconsistency where schema-based tasks cannot pass custom API client configuration, limiting functionality for users ofcreateSchemaTask.Apply this diff to add the missing parameter:
- triggerAndWait: (payload, options) => { + triggerAndWait: (payload, options, requestOptions) => { return new TaskRunPromise<TIdentifier, TOutput>((resolve, reject) => { triggerAndWait_internal<TIdentifier, inferSchemaIn<TSchema>, TOutput>( "triggerAndWait()", params.id, payload, parsePayload, { queue: params.queue?.name, ...options, - } + }, + requestOptions ) .then((result) => { resolve(result); }) .catch((error) => { reject(error); }); }, params.id); },apps/webapp/app/models/organization.server.ts (1)
55-55: Remove unusedfeaturesvariable.The
featuresvariable is no longer used after line 69 was changed to hardcodev3Enabled: true. This is dead code that should be removed.Apply this diff to remove the unused variable:
- const features = featuresForUrl(new URL(env.APP_ORIGIN)); - const organization = await prisma.organization.create({
♻️ Duplicate comments (19)
packages/core/test/streamsWriterV1.test.ts (1)
793-803: Unblock Biome by yielding at least once.Biome still fails this generator because it never yields, so the suite won’t pass. Add a dead-code conditional yield to satisfy the rule without changing behavior. citestatic_analysis_hints
async function* generateChunks() { - // Yields nothing - return; + // Yields nothing; keep generator shape for API compatibility + if (false) { + yield undefined as never; + } }apps/webapp/app/routes/realtime.v1.streams.$runId.$streamId.ts (1)
46-69: Fix Timeout-Seconds parsing to reject NaN/0.The truthy guards still let
"0"or"NaN"through, so we may pass invalid values downstream. Parse explicitly, validate withNumber.isFinite, and keep the 1–600 range enforcement. citepast_review_comments- const timeoutInSecondsRaw = request.headers.get("Timeout-Seconds") ?? undefined; - const timeoutInSeconds = timeoutInSecondsRaw ? parseInt(timeoutInSecondsRaw) : undefined; - - if (timeoutInSeconds && isNaN(timeoutInSeconds)) { - return new Response("Invalid timeout seconds", { status: 400 }); - } - - if (timeoutInSeconds && timeoutInSeconds < 1) { - return new Response("Timeout seconds must be greater than 0", { status: 400 }); - } - - if (timeoutInSeconds && timeoutInSeconds > 600) { - return new Response("Timeout seconds must be less than 600", { status: 400 }); - } + const timeoutInSecondsHeader = request.headers.get("Timeout-Seconds"); + let timeoutInSeconds: number | undefined; + if (timeoutInSecondsHeader !== null) { + const parsed = Number(timeoutInSecondsHeader.trim()); + if (!Number.isFinite(parsed)) { + return new Response("Invalid timeout seconds", { status: 400 }); + } + if (parsed < 1) { + return new Response("Timeout seconds must be greater than 0", { status: 400 }); + } + if (parsed > 600) { + return new Response("Timeout seconds must be less than 600", { status: 400 }); + } + timeoutInSeconds = parsed; + }apps/webapp/app/routes/realtime.v1.streams.$runId.$target.$streamId.ts (2)
21-123: Do not trustX-Stream-Versionfor ingestThe non-
PUTbranch still routes by the client-supplied header, so a v2 client can drive this path and you’ll instantiateS2RealtimeStreams. That instance immediately throws oningestData, turning valid writes into 500s and bypassing the resume guarantees we just built. It also re-opens the version-spoofing risk we closed earlier: a hostile client can pick any backend regardless of the run’s configured version. Use the TaskRun’s authoritativerealtimeStreamsVersioninstead (make sure toselectit in the replica query), enforce a mismatch check, and hard-reject anything other thanv1for server-side ingest.Example fix:
const run = await $replica.taskRun.findFirst({ where: { friendlyId: params.runId, runtimeEnvironmentId: authentication.environment.id, }, select: { id: true, friendlyId: true, + realtimeStreamsVersion: true, parentTaskRun: { select: { friendlyId: true, }, }, @@ - // Extract client ID from header, default to "default" if not provided const clientId = request.headers.get("X-Client-Id") || "default"; - const streamVersion = request.headers.get("X-Stream-Version") || "v1"; + const configuredVersion = (run.realtimeStreamsVersion ?? "v1").toLowerCase(); + const headerVersion = request.headers.get("X-Stream-Version"); + if (headerVersion && headerVersion.toLowerCase() !== configuredVersion) { + return new Response("Stream version mismatch with run configuration", { status: 400 }); + } + if (configuredVersion !== "v1") { + return new Response("Server-side ingest is only supported for v1 streams", { status: 400 }); + } @@ - const realtimeStream = getRealtimeStreamInstance(authentication.environment, streamVersion); + const realtimeStream = getRealtimeStreamInstance(authentication.environment, "v1");
128-194: Loader must userun.realtimeStreamsVersionThe HEAD handler repeats the same header-trusting pattern. For runs configured on S2 this now instantiates
S2RealtimeStreamsandgetLastChunkIndexthrows, so the loader fails. PullrealtimeStreamsVersionout of the run you already loaded (add it toselectinsidefindResource), ignore/mismatch-check the header, and short-circuit when the version isn’tv1so we never call the S2 path here.Suggested change:
return $replica.taskRun.findFirst({ where: { friendlyId: params.runId, runtimeEnvironmentId: authentication.environment.id, }, select: { id: true, friendlyId: true, + realtimeStreamsVersion: true, @@ - const streamVersion = request.headers.get("X-Stream-Version") || "v1"; - - const realtimeStream = getRealtimeStreamInstance(authentication.environment, streamVersion); + const configuredVersion = (run.realtimeStreamsVersion ?? "v1").toLowerCase(); + const headerVersion = request.headers.get("X-Stream-Version"); + if (headerVersion && headerVersion.toLowerCase() !== configuredVersion) { + return new Response("Stream version mismatch with run configuration", { status: 400 }); + } + if (configuredVersion !== "v1") { + return new Response("Last chunk lookup is only supported for v1 realtime streams", { + status: 400, + }); + } + + const realtimeStream = getRealtimeStreamInstance(authentication.environment, "v1");apps/webapp/test/redisRealtimeStreams.test.ts (1)
1360-1366: ** Fix streamResponse call signature: pass options object.**This issue was already identified in previous reviews. The 5th argument must be an options object
{ lastEventId }, not the raw stringlastEventId.apps/webapp/app/routes/resources.orgs.$organizationSlug.projects.$projectParam.env.$envParam.runs.$runParam.streams.$streamKey/route.tsx (3)
60-82: ** Authorize the requested streamKey against the run.**This was already flagged: you must verify that
streamKeyexists inrun.realtimeStreamsbefore streaming. Currently any streamKey can be accessed if the user has access to the run.
201-204: ** Fix invalid scrollIntoView behavior value.**
"instant"is not a validScrollBehavior. Use"auto"for immediate scrolling.
495-495: ** Resume bug: startIndex=0 is treated as falsy.**When
startIndexis0, the conditionstartIndex ? ...evaluates to false and omits the Last-Event-ID header. UsestartIndex !== undefinedortypeof startIndex === 'number'.packages/core/src/v3/realtimeStreams/types.ts (1)
25-28: ** Replace interface accessor with readonly property.**TypeScript interfaces cannot declare
getaccessors. Change toreadonly stream: AsyncIterableStream<T>. Additionally, consider converting to a type alias per repository coding guidelines that prefer types over interfaces.packages/trigger-sdk/src/v3/streams.ts (2)
334-346: ** Wrap fetchStream in try-catch to prevent span leak.**If
apiClient.fetchStreamthrows synchronously (e.g., validation error before creating the stream), neitheronCompletenoronErrorcallbacks will be invoked, leaking the span. Wrap the call in try-catch as suggested in past comments.
613-618: ** Avoid async Promise executor.**Using
new Promise(async (resolve) => ...)violates thenoAsyncPromiseExecutorrule and hides errors. Refactor to use an IIFE:const waitForStreams: Promise<void> = new Promise((resolve, reject) => { (async () => { while (ongoingStreamPromises.length > 0) { const next = ongoingStreamPromises.shift(); if (next) { await next; } } resolve(); })().catch(reject); });packages/core/src/v3/realtimeStreams/noopManager.ts (1)
11-23: Fix type mismatch: ReadableStream must be converted to AsyncIterable, and propagate abort signal.The
sourceparameter is a union typeAsyncIterable<T> | ReadableStream<T>, butcreateAsyncIterableStreamFromAsyncIterableexpects anAsyncIterable<T>. When aReadableStreamis passed, this will fail at runtime. Additionally, thesignalfromoptionsis not being forwarded, preventing cancellation.Apply this diff to normalize the input and propagate the signal:
public pipe<T>( key: string, source: AsyncIterable<T> | ReadableStream<T>, options?: RealtimeStreamOperationOptions ): RealtimeStreamInstance<T> { return { wait: () => Promise.resolve(), get stream(): AsyncIterableStream<T> { - return createAsyncIterableStreamFromAsyncIterable(source); + const asyncIterable = (source as any)[Symbol.asyncIterator] + ? (source as AsyncIterable<T>) + : (async function* () { + const reader = (source as ReadableStream<T>).getReader(); + try { + while (true) { + const { done, value } = await reader.read(); + if (done) break; + yield value as T; + } + } finally { + reader.releaseLock(); + } + })(); + return createAsyncIterableStreamFromAsyncIterable( + asyncIterable, + undefined, + options?.signal + ); }, }; }packages/core/src/v3/realtimeStreams/manager.ts (1)
46-48: AbortSignal.any fallback silently ignores user-provided signal in non-supporting environments.Line 47 uses optional chaining
AbortSignal.any?.([...]), which means ifAbortSignal.anyis undefined, the fallback?? abortController.signalcompletely ignoresoptions.signal, preventing user-initiated cancellation. While trigger.dev runs on Node 21/22 (which supportAbortSignal.any), the coding guidelines prefer isomorphic code for portability.Consider this portable fallback (though may be deferred given Node 21/22 requirement):
// Create an AbortController for this stream const abortController = new AbortController(); // Chain with user-provided signal if present - const combinedSignal = options?.signal - ? AbortSignal.any?.([options.signal, abortController.signal]) ?? abortController.signal - : abortController.signal; + let combinedSignal = abortController.signal; + if (options?.signal) { + if (typeof AbortSignal.any === "function") { + combinedSignal = AbortSignal.any([options.signal, abortController.signal]); + } else { + // Manual combination for older environments + const onAbort = () => abortController.abort(); + options.signal.addEventListener("abort", onAbort, { once: true }); + } + }Based on learnings
packages/core/src/v3/apiClient/runStream.ts (2)
285-287: Validate batch data structure before casting.The type assertion is unsafe and could cause runtime errors if the response format doesn't match expectations.
Apply this diff to add safe validation using Zod:
+import { z } from "zod"; + +const BatchDataSchema = z.object({ + records: z.array( + z.object({ + body: z.string(), + seq_num: z.number(), + timestamp: z.number(), + }) + ), +});Then update the transform:
} else { if (chunk.event === "batch") { - const data = safeParseJSON(chunk.data) as { - records: Array<{ body: string; seq_num: number; timestamp: number }>; - }; + const parsed = safeParseJSON(chunk.data); + const result = BatchDataSchema.safeParse(parsed); + + if (!result.success) { + console.error("Invalid batch data format:", result.error); + return; + } + + const data = result.data; for (const record of data.records) {As per coding guidelines.
509-529: Add error handling for stream subscription and processing.The promise chain lacks error handling. If
subscription.subscribe()rejects or the stream processing fails, it will cause an unhandled promise rejection.Apply this diff to add error handling:
// Start stream processing in the background subscription.subscribe().then((stream) => { stream .pipeThrough( new TransformStream({ transform(chunk, controller) { controller.enqueue({ type: streamKey, chunk: chunk.chunk as TStreams[typeof streamKey], run, }); }, }) ) .pipeTo( new WritableStream({ write(chunk) { controller.enqueue(chunk); }, }) - ); - }); + ) + .catch((error) => { + console.error(`Error processing stream ${streamKey}:`, error); + }); + }).catch((error) => { + console.error(`Error subscribing to stream ${streamKey}:`, error); + activeStreams.delete(streamKey); + });packages/react-hooks/src/hooks/useRealtime.ts (4)
740-772: Critical: Hooks called conditionally, violating Rules of Hooks.The routing logic throws an error without calling any hooks in one branch, while other branches call
useRealtimeStreamImplementation(which contains hooks). This violates React's fundamental rule that hooks must be called unconditionally in the same order every render.Restructure to always call hooks before validation:
export function useRealtimeStream<TPart>( runIdOrDefinedStream: string | RealtimeDefinedStream<TPart>, streamKeyOrOptionsOrRunId?: string | UseRealtimeStreamOptions<TPart>, options?: UseRealtimeStreamOptions<TPart> ): UseRealtimeStreamInstance<TPart> { + // Determine parameters before calling any hooks + let runId: string; + let streamKey: string; + let opts: UseRealtimeStreamOptions<TPart> | undefined; + if (typeof runIdOrDefinedStream === "string") { if (typeof streamKeyOrOptionsOrRunId === "string") { - return useRealtimeStreamImplementation( - runIdOrDefinedStream, - streamKeyOrOptionsOrRunId, - options - ); + runId = runIdOrDefinedStream; + streamKey = streamKeyOrOptionsOrRunId; + opts = options; } else { - return useRealtimeStreamImplementation( - runIdOrDefinedStream, - "default", - streamKeyOrOptionsOrRunId - ); + runId = runIdOrDefinedStream; + streamKey = "default"; + opts = streamKeyOrOptionsOrRunId; } } else { if (typeof streamKeyOrOptionsOrRunId === "string") { - return useRealtimeStreamImplementation( - streamKeyOrOptionsOrRunId, - runIdOrDefinedStream.id, - options - ); + runId = streamKeyOrOptionsOrRunId; + streamKey = runIdOrDefinedStream.id; + opts = options; } else { throw new Error( "Invalid second argument to useRealtimeStream. When using a defined stream instance, the second argument to useRealtimeStream must be a run ID." ); } } + + return useRealtimeStreamImplementation(runId, streamKey, opts); }
1067-1070: Correct startIndex handling (0 should be valid).
startIndex ? …treats 0 as falsy, breaking the "start from beginning" semantics when using explicit 0.Apply this diff:
- lastEventId: startIndex ? (startIndex - 1).toString() : undefined, + lastEventId: startIndex != null ? String(startIndex - 1) : undefined,
871-885: Resubscribe when streamKey or options change.The effect only re-runs for
runId/enabled, so changingstreamKeyor option values won't start a new subscription.Apply this diff:
useEffect(() => { if (typeof options?.enabled === "boolean" && !options.enabled) { return; } if (!runId) { return; } triggerRequest().finally(() => {}); return () => { stop(); }; - }, [runId, stop, options?.enabled]); + }, [runId, streamKey, triggerRequest, stop, options?.enabled]);
831-869: Add option-derived values to triggerRequest dependencies.
triggerRequestcloses overoptions?.timeoutInSeconds,options?.startIndex,options?.throttleInMs, andonData, but these are not in the dependency array. Changes to these values will leave the subscription using stale values.Apply this diff:
- }, [runId, streamKey, mutateParts, partsRef, abortControllerRef, apiClient, setError]); + }, [ + runId, + streamKey, + mutateParts, + partsRef, + abortControllerRef, + apiClient, + setError, + onData, + options?.timeoutInSeconds, + options?.startIndex, + options?.throttleInMs, + ]);
🧹 Nitpick comments (10)
internal-packages/run-engine/src/engine/types.ts (1)
151-151: LGTM! Consider adding documentation.The addition of
realtimeStreamsVersionto track the realtime streams backend version is appropriate and follows the existing pattern of optional fields in the interface.Optional improvements to consider:
- Add JSDoc documentation to clarify the field's purpose and valid values:
+ /** Version of the realtime streams backend to use (e.g., "v1" for Redis, "v2" for S2). */ realtimeStreamsVersion?: string;
- Stronger typing if the valid versions are known and limited:
realtimeStreamsVersion?: "v1" | "v2";However, keeping it as
stringprovides flexibility for future versions, which may be preferable given this is an evolving feature.apps/webapp/seed.mts (2)
56-70: Consider making externalRef values configurable.The hardcoded
externalRefvalues work for local development seeding, but consider making them configurable via environment variables to support different environments or developer preferences.For example:
const referenceProjects = [ { name: "hello-world", externalRef: process.env.HELLO_WORLD_PROJECT_REF ?? "proj_rrkpdguyagvsoktglnod", }, { name: "d3-chat", externalRef: process.env.D3_CHAT_PROJECT_REF ?? "proj_cdmymsrobxmcgjqzhdkq", }, { name: "realtime-streams", externalRef: process.env.REALTIME_STREAMS_PROJECT_REF ?? "proj_klxlzjnzxmbgiwuuwhvb", }, ];
119-121: Consider automating .env file updates.The manual note to update
.envfiles is helpful, but this step could be automated by the seed script itself to improve developer experience and reduce setup friction.For example, append to the seed function:
import { writeFile } from "fs/promises"; import { resolve } from "path"; // After project creation loop const d3ChatEnvPath = resolve(__dirname, "../../d3-chat/.env"); const realtimeStreamsEnvPath = resolve(__dirname, "../../realtime-streams/.env"); const d3ChatProject = referenceProjects.find(p => p.name === "d3-chat"); const realtimeStreamsProject = referenceProjects.find(p => p.name === "realtime-streams"); if (d3ChatProject) { try { await writeFile( d3ChatEnvPath, `TRIGGER_PROJECT_REF=${d3ChatProject.externalRef}\n`, { flag: "a" } ); console.log(`✅ Updated ${d3ChatEnvPath}`); } catch (e) { console.log(`⚠️ Could not update ${d3ChatEnvPath}, please update manually`); } } // Similar for realtime-streamsapps/webapp/app/routes/resources.orgs.$organizationSlug.projects.$projectParam.env.$envParam.runs.$runParam.streams.$streamKey/route.tsx (1)
482-541: Consider error recovery in useRealtimeStream.The hook sets
isConnectedtofalseon errors but doesn't implement automatic reconnection. For transient network failures, consider adding retry logic similar to the SSEStreamSubscription's built-in retry mechanism, or document that consumers should remount the component to retry.packages/core/src/v3/realtimeStreams/types.ts (2)
11-23: Consider converting RealtimeStreamsManager interface to type.The coding guidelines state "For TypeScript, we usually use types over interfaces". Unless this interface is extended elsewhere, convert to a type alias for consistency:
export type RealtimeStreamsManager = { pipe<T>(...): RealtimeStreamInstance<T>; append<TPart extends BodyInit>(...): Promise<void>; };As per coding guidelines
30-32: Consider converting StreamsWriter interface to type.For consistency with the repository's preference for types over interfaces:
export type StreamsWriter = { wait(): Promise<void>; };As per coding guidelines
packages/core/src/v3/realtimeStreams/streamsWriterV2.ts (1)
134-153: Pipeline abort check in transform could race.The transform's abort check at line 139 (
if (this.aborted)) may race with data already enqueued. If the signal is aborted between the check and thecontroller.enqueuecall, you'll attempt to enqueue after error. Consider wrapping the enqueue in a try-catch:transform: (chunk, controller) => { if (this.aborted) { controller.error(new Error("Stream aborted")); return; } try { controller.enqueue(AppendRecord.make(JSON.stringify({ data: chunk, id: nanoid(7) }))); } catch (error) { // Stream may have been aborted/closed between check and enqueue } },packages/trigger-sdk/src/v3/streams.ts (1)
360-381: Append overload resolution is complex; consider simplifying.The nested conditionals in the
appendfunction to distinguish between overload variants are difficult to follow. Consider explicitly checking argument types first:function append<TPart extends BodyInit>( keyOrValue: string | TPart, valueOrOptions?: TPart | AppendStreamOptions, options?: AppendStreamOptions ): Promise<void> { // Three-argument form: append(key, value, options?) if (typeof keyOrValue === "string" && valueOrOptions !== undefined && !isAppendStreamOptions(valueOrOptions)) { return appendInternal(keyOrValue, valueOrOptions, options); } // Two-argument form: append(key, options) or append(value, options) if (typeof keyOrValue === "string" && isAppendStreamOptions(valueOrOptions)) { return appendInternal(DEFAULT_STREAM_KEY, keyOrValue, valueOrOptions); } // One or two-argument form: append(value, options?) return appendInternal( DEFAULT_STREAM_KEY, keyOrValue, isAppendStreamOptions(valueOrOptions) ? valueOrOptions : options ); }packages/cli-v3/src/entryPoints/dev-run-worker.ts (1)
155-160: Simplify the debug flag expression.The boolean evaluation on lines 158-159 is unnecessarily complex. The expression evaluates to
boolean | undefinedand then uses?? false, but the result of the comparison is already boolean or undefined.Apply this diff to simplify:
const standardRealtimeStreamsManager = new StandardRealtimeStreamsManager( apiClientManager.clientOrThrow(), getEnvVar("TRIGGER_STREAM_URL", getEnvVar("TRIGGER_API_URL")) ?? "https://api.trigger.dev", - (getEnvVar("TRIGGER_STREAMS_DEBUG") === "1" || getEnvVar("TRIGGER_STREAMS_DEBUG") === "true") ?? - false + getEnvVar("TRIGGER_STREAMS_DEBUG") === "1" || getEnvVar("TRIGGER_STREAMS_DEBUG") === "true" );packages/cli-v3/src/entryPoints/managed-run-worker.ts (1)
135-140: Simplify the debug flag expression.The boolean evaluation on lines 138-139 is unnecessarily complex, identical to the pattern in dev-run-worker.ts.
Apply this diff to simplify:
const standardRealtimeStreamsManager = new StandardRealtimeStreamsManager( apiClientManager.clientOrThrow(), getEnvVar("TRIGGER_STREAM_URL", getEnvVar("TRIGGER_API_URL")) ?? "https://api.trigger.dev", - (getEnvVar("TRIGGER_STREAMS_DEBUG") === "1" || getEnvVar("TRIGGER_STREAMS_DEBUG") === "true") ?? - false + getEnvVar("TRIGGER_STREAMS_DEBUG") === "1" || getEnvVar("TRIGGER_STREAMS_DEBUG") === "true" );
There was a problem hiding this comment.
Actionable comments posted: 4
♻️ Duplicate comments (4)
apps/webapp/app/services/realtime/s2realtimeStreams.server.ts (1)
95-105: Avoid logging customer data inpart.Despite the level being changed to
debug, the fullpartis still logged at line 98. This arbitrary customer data could be large or contain PII. Log only metadata such as stream name, partId, and part length.Apply this diff:
- this.logger.debug(`S2 appending to stream`, { part, stream: s2Stream }); + this.logger.debug(`S2 appending to stream`, { + stream: s2Stream, + partId, + size: part.length, + });packages/core/src/v3/realtimeStreams/streamInstance.ts (2)
30-35: Respect the requested target run.The
targetoption fromStreamInstanceOptionsis ignored: both thecreateStreamcall (line 32) and the v1 writer construction (line 49) are hard-coded to"self". This breaks the documented targeting feature—calls likestreams.append(..., { target: "parent" })will silently write to the wrong run.Apply this diff:
+ const target = this.options.target ?? "self"; const { version, headers } = await this.options.apiClient.createStream( this.options.runId, - "self", + target, this.options.key, this.options?.requestOptions );parsedResponse.version === "v1" ? new StreamsWriterV1({ key: this.options.key, runId: this.options.runId, source: this.options.source, baseUrl: this.options.baseUrl, headers: this.options.apiClient.getHeaders(), signal: this.options.signal, version, - target: "self", + target, })Also applies to: 49-49
79-82: Clean up the iterator when aborted.When the abort signal fires, the controller is closed but
iterator.return()is not called. This may leave the underlying async iterator in an unclosed state, potentially leaking resources.Apply this diff:
if (self.options.signal?.aborted) { controller.close(); + if (typeof iterator.return === "function") { + await iterator.return(); + } break; }packages/trigger-sdk/src/v3/streams.ts (1)
305-347: WrapfetchStreamin try-catch to prevent span leak.If
apiClient.fetchStreamthrows synchronously (e.g., validation error, initial request failure), the span will never be ended because theonCompleteandonErrorcallbacks won't be invoked.Apply this diff:
const span = tracer.startSpan("streams.read()", { attributes: { key, runId, [SemanticInternalAttributes.ENTITY_TYPE]: "realtime-stream", [SemanticInternalAttributes.ENTITY_ID]: `${runId}:${key}`, [SemanticInternalAttributes.ENTITY_METADATA]: JSON.stringify({ startIndex: options?.startIndex, }), [SemanticInternalAttributes.STYLE_ICON]: "streams", ...accessoryAttributes({ items: [ { text: key, variant: "normal", }, ], style: "codepath", }), }, }); - return await apiClient.fetchStream(runId, key, { - signal: options?.signal, - timeoutInSeconds: options?.timeoutInSeconds ?? 60, - lastEventId: options?.startIndex ? (options.startIndex - 1).toString() : undefined, - onComplete: () => { - span.end(); - }, - onError: (error) => { - span.recordException(error); - span.setStatus({ code: SpanStatusCode.ERROR }); - span.end(); - }, - }); + try { + return await apiClient.fetchStream(runId, key, { + signal: options?.signal, + timeoutInSeconds: options?.timeoutInSeconds ?? 60, + lastEventId: options?.startIndex ? (options.startIndex - 1).toString() : undefined, + onComplete: () => { + span.end(); + }, + onError: (error) => { + span.recordException(error); + span.setStatus({ code: SpanStatusCode.ERROR }); + span.end(); + }, + }); + } catch (error) { + if (error instanceof Error && error.name === "AbortError") { + span.end(); + throw error; + } + + if (error instanceof Error || typeof error === "string") { + span.recordException(error); + } else { + span.recordException(String(error)); + } + + span.setStatus({ code: SpanStatusCode.ERROR }); + span.end(); + throw error; + } }
🧹 Nitpick comments (4)
apps/webapp/app/services/realtime/s2realtimeStreams.server.ts (1)
218-220: Consider whether wildcard CORS header exposure is necessary.Setting
Access-Control-Expose-Headers: *exposes all response headers to cross-origin requests. Verify this level of permissiveness aligns with security requirements, or specify only the headers that clients need.packages/core/src/v3/realtimeStreams/streamInstance.ts (2)
137-154: Remove unused helper functions.
streamToAsyncIteratorandsafeReleaseLockare defined but never used in this file and are not exported. Remove them to reduce clutter.Apply this diff:
-async function* streamToAsyncIterator<T>(stream: ReadableStream<T>): AsyncIterableIterator<T> { - const reader = stream.getReader(); - try { - while (true) { - const { done, value } = await reader.read(); - if (done) return; - yield value; - } - } finally { - safeReleaseLock(reader); - } -} - -function safeReleaseLock(reader: ReadableStreamDefaultReader<any>) { - try { - reader.releaseLock(); - } catch (error) {} -}
132-133: Specify radix in parseInt.While
parseIntwithout a radix works for decimal strings, explicitly passing10is a best practice to avoid unexpected behavior with leading zeros or hexadecimal strings.Apply this diff:
version: "v2", accessToken, basin, - flushIntervalMs: flushIntervalMs ? parseInt(flushIntervalMs) : undefined, - maxRetries: maxRetries ? parseInt(maxRetries) : undefined, + flushIntervalMs: flushIntervalMs ? parseInt(flushIntervalMs, 10) : undefined, + maxRetries: maxRetries ? parseInt(maxRetries, 10) : undefined, };packages/trigger-sdk/src/v3/streams.ts (1)
573-579: Log suppressed enqueue errors.
safeEnqueuesilently suppresses all errors. While this prevents crashes when the stream is closed, it also hides legitimate bugs (e.g., memory issues, unexpected controller states). Log the error to aid debugging.Apply this diff:
function safeEnqueue(data: TPart) { try { controller.enqueue(data); } catch (error) { - // suppress errors when the stream has been closed + // Suppress errors when the stream has been closed, but log for debugging + if (error instanceof Error && !error.message.includes("stream is closed")) { + console.error("Unexpected error in safeEnqueue:", error); + } } }
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (2)
pnpm-lock.yamlis excluded by!**/pnpm-lock.yamlreferences/realtime-streams/src/trigger/streams.tsis excluded by!references/**
📒 Files selected for processing (4)
apps/webapp/app/services/realtime/s2realtimeStreams.server.ts(1 hunks)packages/core/package.json(1 hunks)packages/core/src/v3/realtimeStreams/streamInstance.ts(1 hunks)packages/trigger-sdk/src/v3/streams.ts(1 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
- packages/core/package.json
🧰 Additional context used
📓 Path-based instructions (5)
**/*.{ts,tsx}
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
**/*.{ts,tsx}: Always prefer using isomorphic code like fetch, ReadableStream, etc. instead of Node.js specific code
For TypeScript, we usually use types over interfaces
Avoid enums
No default exports, use function declarations
Files:
apps/webapp/app/services/realtime/s2realtimeStreams.server.tspackages/trigger-sdk/src/v3/streams.tspackages/core/src/v3/realtimeStreams/streamInstance.ts
{packages/core,apps/webapp}/**/*.{ts,tsx}
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
We use zod a lot in packages/core and in the webapp
Files:
apps/webapp/app/services/realtime/s2realtimeStreams.server.tspackages/core/src/v3/realtimeStreams/streamInstance.ts
apps/webapp/**/*.{ts,tsx}
📄 CodeRabbit inference engine (.cursor/rules/webapp.mdc)
When importing from @trigger.dev/core in the webapp, never import the root package path; always use one of the documented subpath exports from @trigger.dev/core’s package.json
Files:
apps/webapp/app/services/realtime/s2realtimeStreams.server.ts
{apps/webapp/app/**/*.server.{ts,tsx},apps/webapp/app/routes/**/*.ts}
📄 CodeRabbit inference engine (.cursor/rules/webapp.mdc)
Access environment variables only via the env export from app/env.server.ts; do not reference process.env directly
Files:
apps/webapp/app/services/realtime/s2realtimeStreams.server.ts
apps/webapp/app/**/*.ts
📄 CodeRabbit inference engine (.cursor/rules/webapp.mdc)
Modules intended for test consumption under apps/webapp/app/**/*.ts must not read environment variables; accept configuration via options instead
Files:
apps/webapp/app/services/realtime/s2realtimeStreams.server.ts
🧠 Learnings (3)
📚 Learning: 2025-08-18T10:07:17.368Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-08-18T10:07:17.368Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Import Trigger.dev APIs from "trigger.dev/sdk/v3" when writing tasks or related utilities
Applied to files:
packages/trigger-sdk/src/v3/streams.ts
📚 Learning: 2025-07-18T17:49:24.468Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .github/copilot-instructions.md:0-0
Timestamp: 2025-07-18T17:49:24.468Z
Learning: Applies to **/*.{ts,tsx} : Always prefer using isomorphic code like fetch, ReadableStream, etc. instead of Node.js specific code
Applied to files:
packages/trigger-sdk/src/v3/streams.ts
📚 Learning: 2025-08-18T10:07:17.368Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-08-18T10:07:17.368Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Use triggerAndWait() only from within a task context (not from generic app code) and handle result.ok or use unwrap() with error handling
Applied to files:
packages/trigger-sdk/src/v3/streams.ts
🧬 Code graph analysis (3)
apps/webapp/app/services/realtime/s2realtimeStreams.server.ts (1)
apps/webapp/app/services/realtime/types.ts (3)
StreamResponder(27-35)StreamIngestor(2-19)StreamResponseOptions(21-24)
packages/trigger-sdk/src/v3/streams.ts (10)
packages/core/src/v3/realtimeStreams/manager.ts (2)
pipe(27-73)append(75-99)packages/core/src/v3/realtimeStreams/noopManager.ts (2)
pipe(12-23)append(25-29)packages/core/src/v3/realtimeStreams/index.ts (2)
pipe(34-40)append(42-48)packages/core/src/v3/realtimeStreams/types.ts (7)
PipeStreamOptions(50-70)PipeStreamResult(77-89)ReadStreamOptions(94-116)AppendStreamOptions(121-136)WriterStreamOptions(138-145)RealtimeDefinedStream(34-43)RealtimeStreamOperationOptions(5-9)packages/core/src/v3/semanticInternalAttributes.ts (1)
SemanticInternalAttributes(1-68)packages/core/src/v3/apiClient/index.ts (1)
mergeRequestOptions(1410-1426)packages/core/src/v3/realtime-streams-api.ts (1)
realtimeStreams(5-5)packages/core/src/v3/apiClientManager-api.ts (1)
apiClientManager(5-5)packages/core/src/v3/realtimeStreams/streamInstance.ts (1)
stream(69-95)packages/core/src/v3/task-context-api.ts (1)
taskContext(5-5)
packages/core/src/v3/realtimeStreams/streamInstance.ts (7)
packages/core/src/v3/apiClient/index.ts (1)
headers(1229-1240)packages/core/src/v3/apiClient/core.ts (1)
AnyZodFetchOptions(41-41)packages/core/src/v3/realtimeStreams/streamsWriterV1.ts (1)
StreamsWriterV1(26-449)packages/core/src/v3/realtimeStreams/streamsWriterV2.ts (1)
StreamsWriterV2(46-197)packages/core/src/v3/realtimeStreams/types.ts (1)
StreamsWriter(30-32)packages/core/src/v3/streams/asyncIterableStream.ts (1)
AsyncIterableStream(1-1)packages/core/src/v3/runMetadata/manager.ts (1)
stream(297-303)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (23)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (8, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (7, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (1, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (4, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (5, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (6, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (3, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (7, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (2, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (5, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (8, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (3, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (1, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (6, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (4, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (2, 8)
- GitHub Check: units / packages / 🧪 Unit Tests: Packages (1, 1)
- GitHub Check: e2e / 🧪 CLI v3 tests (ubuntu-latest - pnpm)
- GitHub Check: e2e / 🧪 CLI v3 tests (windows-latest - pnpm)
- GitHub Check: e2e / 🧪 CLI v3 tests (ubuntu-latest - npm)
- GitHub Check: e2e / 🧪 CLI v3 tests (windows-latest - npm)
- GitHub Check: typecheck / typecheck
- GitHub Check: Analyze (javascript-typescript)
🔇 Additional comments (1)
apps/webapp/app/services/realtime/s2realtimeStreams.server.ts (1)
4-4: Use isomorphic crypto.randomUUID() instead of Node.js import.The coding guidelines require isomorphic code. Replace the Node.js-specific import with the Web Crypto API.
As per coding guidelines.
Apply this diff:
-import { randomUUID } from "node:crypto"; +import { randomUUID } from "crypto";⛔ Skipped due to learnings
Learnt from: CR Repo: triggerdotdev/trigger.dev PR: 0 File: .github/copilot-instructions.md:0-0 Timestamp: 2025-07-18T17:49:24.468Z Learning: Applies to **/*.{ts,tsx} : Always prefer using isomorphic code like fetch, ReadableStream, etc. instead of Node.js specific codeLearnt from: CR Repo: triggerdotdev/trigger.dev PR: 0 File: .cursor/rules/webapp.mdc:0-0 Timestamp: 2025-08-29T10:06:49.293Z Learning: Maintain service/configuration separation (e.g., create testable service modules and separate configuration modules like realtimeClient.server.ts vs realtimeClientGlobal.server.ts)
… stream data from tasks to the server
…ND_SENTINEL system
…ing resumable streams. We also will now send invisible "ping" packets to keep connected clients alive when there are no real data packets to send, which will be especially helpful to older clients
… the stream keys. Also added a new db:seed script to seed a fresh database for local development with reference projects setup
ericallam
force-pushed
the
feat/realtime-streams-2
branch
from
4ba912d to
211d804
Compare
There was a problem hiding this comment.
Actionable comments posted: 2
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
packages/trigger-sdk/src/v3/shared.ts (1)
1270-1281: Fix runCount attribute type check.Array.isArray(body.runCount) is incorrect; runCount is a number elsewhere. This branch never executes.
- if ("runCount" in body && Array.isArray(body.runCount)) { + if ("runCount" in body && typeof body.runCount === "number") { span.setAttribute("runCount", body.runCount); }
♻️ Duplicate comments (28)
apps/webapp/seed.mts (1)
78-91: Non-atomic project creation can leave orphaned records.The create-then-update pattern is not atomic. If the script fails between the
createProjectcall and theupdatecall, an orphaned project with an incorrectexternalRefwill remain. On the next run, the query on line 74 won't find this orphaned project (because it searches for the finalexternalRef), leading to duplicate project creation attempts.Consider passing
externalRefdirectly tocreateProjectif supported, or wrapping both operations in aprisma.$transactionto ensure atomicity.packages/trigger-sdk/src/v3/streams.ts (3)
602-611: Propagate execute errors to the span.Errors from
options.executeare logged but not re-thrown or recorded in the span. This can result in silent failures where the writer appears to succeed but the execute logic failed.Similar to the merge callback, collect these errors and propagate them after
waitForStreamscompletes:if (result) { ongoingStreamPromises.push( result.catch((error) => { - console.error(error); + errors.push(error instanceof Error ? error : new Error(String(error))); }) ); } } catch (error) { - console.error(error); + errors.push(error instanceof Error ? error : new Error(String(error))); }
334-346: WrapfetchStreamin try-catch to prevent span leak.If
apiClient.fetchStreamthrows before creating the stream (e.g., validation error, initial request failure), the span will never be ended since neitheronCompletenoronErrorcallbacks will be invoked.Apply this diff:
const span = tracer.startSpan("streams.read()", { // ... attributes }); + try { return await apiClient.fetchStream(runId, key, { signal: options?.signal, timeoutInSeconds: options?.timeoutInSeconds ?? 60, lastEventId: options?.startIndex ? (options.startIndex - 1).toString() : undefined, onComplete: () => { span.end(); }, onError: (error) => { span.recordException(error); span.setStatus({ code: SpanStatusCode.ERROR }); span.end(); }, }); + } catch (error) { + if (error instanceof Error && error.name === "AbortError") { + span.end(); + throw error; + } + + if (error instanceof Error || typeof error === "string") { + span.recordException(error); + } else { + span.recordException(String(error)); + } + + span.setStatus({ code: SpanStatusCode.ERROR }); + span.end(); + throw error; + } }
586-598: Propagate merge errors to the span.The
mergecallback logs errors to the console but does not propagate them to the wrapping span or the caller. This can hide stream processing failures and result in incomplete data without any indication in the observability trace.Consider collecting errors and propagating them after
waitForStreamscompletes. For example, accumulate errors in an array and afterwaitForStreamsresolves, record them as span events or exceptions:+ const errors: Error[] = []; + function writerInternal<TPart>(key: string, options: WriterStreamOptions<TPart>) { // ... existing code ... merge(streamArg) { ongoingStreamPromises.push( (async () => { const reader = streamArg.getReader(); while (true) { const { done, value } = await reader.read(); if (done) break; safeEnqueue(value); } })().catch((error) => { - console.error(error); + errors.push(error instanceof Error ? error : new Error(String(error))); }) ); }, + + // After waitForStreams completes, record errors + waitForStreams.then(() => { + if (errors.length > 0) { + // Errors will be recorded by pipeInternal's span + throw new AggregateError(errors, "Stream merge/execute errors occurred"); + } + });apps/webapp/app/services/realtime/s2realtimeStreams.server.ts (2)
143-146: Clean up the broken citation placeholder.
Remove the:contentReference[...]fragment so the comment reads clearly.- "S2-Format": "raw", // UTF-8 JSON encoding (no base64 overhead) when your data is text. :contentReference[oaicite:8]{index=8} + "S2-Format": "raw", // UTF-8 JSON encoding (no base64 overhead) when your data is text.
95-105: Remove sensitive payload from logs.
Loggingpartemits arbitrary customer content (potentially PII) to logs; capture only metadata like stream name, partId, and payload size instead.- this.logger.debug(`S2 appending to stream`, { part, stream: s2Stream }); + this.logger.debug(`S2 appending to stream`, { + stream: s2Stream, + partId, + size: part.length, + }); ... - this.logger.debug(`S2 append result`, { result }); + this.logger.debug(`S2 append result`, { + startSeq: result.start.seq_num, + endSeq: result.end.seq_num, + tailSeq: result.tail.seq_num, + });packages/core/src/v3/apiClient/runStream.ts (3)
767-779: LGTM: explicit radix in parseInt prevents locale/radix ambiguity.The parseInt(..., 10) change is correct and resolves prior robustness concerns.
284-304: Validate v2 “batch” payloads with Zod; remove unsafe type assertions.chunk.data and record.body are parsed with safeParseJSON then blindly asserted. Malformed data will throw at runtime inside transform(). Replace with schema-backed validation and skip/emit errors on failure.
As per coding guidelines / Based on learnings.
Apply within this block:
- if (chunk.event === "batch") { - const data = safeParseJSON(chunk.data) as { - records: Array<{ body: string; seq_num: number; timestamp: number }>; - }; - - for (const record of data.records) { + if (chunk.event === "batch") { + const parsedData = safeParseJSON(chunk.data); + const batchResult = BatchDataSchema.safeParse(parsedData); + if (!batchResult.success) { + this.options.onError?.(new Error(`Invalid batch data: ${batchResult.error.message}`)); + return; + } + for (const record of batchResult.data.records) { this.lastEventId = record.seq_num.toString(); - - const parsedBody = safeParseJSON(record.body) as { data: unknown; id: string }; - if (seenIds.has(parsedBody.id)) { + const parsedBodyUnknown = safeParseJSON(record.body); + const bodyResult = ParsedBodySchema.safeParse(parsedBodyUnknown); + if (!bodyResult.success) { + this.options.onError?.(new Error(`Invalid record body: ${bodyResult.error.message}`)); + continue; + } + if (seenIds.has(bodyResult.data.id)) { continue; } - seenIds.add(parsedBody.id); + seenIds.add(bodyResult.data.id); chunkController.enqueue({ id: record.seq_num.toString(), - chunk: parsedBody.data, + chunk: bodyResult.data.data, timestamp: record.timestamp, }); } }Add once near the top of the file (outside this hunk):
import { z } from "zod"; const BatchDataSchema = z.object({ records: z.array( z.object({ body: z.string(), seq_num: z.number(), timestamp: z.number(), }) ), }); const ParsedBodySchema = z.object({ id: z.string(), data: z.unknown(), });Also applies to: 292-297
509-529: Handle subscribe/stream errors to avoid unhandled promise rejections and stuck subscriptions.subscription.subscribe() and stream.pipeTo(...) lack .catch handlers; failures won’t be surfaced and activeStreams never cleans up. Add error handling and controller.error(...) call.
- subscription.subscribe().then((stream) => { + subscription.subscribe().then((stream) => { stream .pipeThrough( new TransformStream({ transform(chunk, controller) { controller.enqueue({ type: streamKey, chunk: chunk.chunk as TStreams[typeof streamKey], run, }); }, }) ) .pipeTo( new WritableStream({ write(chunk) { controller.enqueue(chunk); }, }) - ); - }); + ) + .catch((error) => { + controller.error(error); + activeStreams.delete(streamKey); + }); + }).catch((error) => { + controller.error(error); + activeStreams.delete(streamKey); + });packages/core/src/v3/schemas/common.ts (2)
336-346: Constrain realtimeStreamsVersion to known values instead of free string.Use z.enum(["v1","v2"]) for validation/typing (keeps forward-compat with optional).
run: TaskRun.and( z.object({ traceContext: z.record(z.unknown()).optional(), - realtimeStreamsVersion: z.string().optional(), + realtimeStreamsVersion: z.enum(["v1", "v2"]).optional(), }) ),
390-398: Add missing realtimeStreamsVersion to V3TaskRunExecution.run.V3 variant omits the field; add it to keep schemas aligned with runtime usage.
run: V3TaskRun.and( z.object({ traceContext: z.record(z.unknown()).optional(), + realtimeStreamsVersion: z.enum(["v1", "v2"]).optional(), }) ),packages/react-hooks/src/hooks/useRealtime.ts (4)
741-772: Critical: hooks are called conditionally; normalize args then call one hook unconditionally.Current overload returns early in branches that call useRealtimeStreamImplementation; violates Rules of Hooks and triggers Biome errors. Normalize (runId, streamKey, opts) first, then call the hook once.
export function useRealtimeStream<TPart>( runIdOrDefinedStream: string | RealtimeDefinedStream<TPart>, streamKeyOrOptionsOrRunId?: string | UseRealtimeStreamOptions<TPart>, options?: UseRealtimeStreamOptions<TPart> ): UseRealtimeStreamInstance<TPart> { - if (typeof runIdOrDefinedStream === "string") { - if (typeof streamKeyOrOptionsOrRunId === "string") { - return useRealtimeStreamImplementation( - runIdOrDefinedStream, - streamKeyOrOptionsOrRunId, - options - ); - } else { - return useRealtimeStreamImplementation( - runIdOrDefinedStream, - "default", - streamKeyOrOptionsOrRunId - ); - } - } else { - if (typeof streamKeyOrOptionsOrRunId === "string") { - return useRealtimeStreamImplementation( - streamKeyOrOptionsOrRunId, - runIdOrDefinedStream.id, - options - ); - } else { - throw new Error( - "Invalid second argument to useRealtimeStream. When using a defined stream instance, the second argument to useRealtimeStream must be a run ID." - ); - } - } + let runId: string; + let streamKey: string; + let opts: UseRealtimeStreamOptions<TPart> | undefined; + + if (typeof runIdOrDefinedStream === "string") { + if (typeof streamKeyOrOptionsOrRunId === "string") { + runId = runIdOrDefinedStream; + streamKey = streamKeyOrOptionsOrRunId; + opts = options; + } else { + runId = runIdOrDefinedStream; + streamKey = "default"; + opts = streamKeyOrOptionsOrRunId; + } + } else { + if (typeof streamKeyOrOptionsOrRunId !== "string") { + throw new Error( + "Invalid second argument to useRealtimeStream. When using a defined stream instance, the second argument to useRealtimeStream must be a run ID." + ); + } + runId = streamKeyOrOptionsOrRunId; + streamKey = runIdOrDefinedStream.id; + opts = options; + } + + return useRealtimeStreamImplementation(runId, streamKey, opts); }
871-885: Resubscribe when streamKey or triggerRequest change.Effect depends only on runId/enabled; switching streamKey or updated triggerRequest won’t restart the stream.
- }, [runId, stop, options?.enabled]); + }, [runId, streamKey, stop, options?.enabled, triggerRequest]);
1065-1071: startIndex: treat 0 as valid; don’t use truthy check.0 should start from the beginning; current ternary drops it.
- lastEventId: startIndex ? (startIndex - 1).toString() : undefined, + lastEventId: startIndex != null ? String(startIndex - 1) : undefined,
831-888: Fix stale options in triggerRequest’s memoization.triggerRequest captures onData/timeoutInSeconds/startIndex/throttleInMs but doesn’t list them as deps, leading to stale subscriptions after prop changes. Include them.
- }, [runId, streamKey, mutateParts, partsRef, abortControllerRef, apiClient, setError]); + }, [ + runId, + streamKey, + mutateParts, + partsRef, + abortControllerRef, + apiClient, + setError, + onData, + options?.timeoutInSeconds, + options?.startIndex, + options?.throttleInMs, + ]);packages/core/src/v3/types/tasks.ts (1)
620-623: AddrequestOptionstobatchTriggerAndWaitsignature.The SDK implementation already accepts
requestOptions, but the type surface still omits it. This mismatch blocks callers from passing per-request settings. Please align the signature with the other trigger methods.- batchTriggerAndWait: ( - items: Array<BatchTriggerAndWaitItem<TInput>>, - options?: BatchTriggerAndWaitOptions - ) => Promise<BatchResult<TIdentifier, TOutput>>; + batchTriggerAndWait: ( + items: Array<BatchTriggerAndWaitItem<TInput>>, + options?: BatchTriggerAndWaitOptions, + requestOptions?: TriggerApiRequestOptions + ) => Promise<BatchResult<TIdentifier, TOutput>>;apps/webapp/app/routes/realtime.v1.streams.$runId.$streamId.ts (1)
46-59: ParseTimeout-Secondswithout truthy guards.Reintroducing the truthy checks lets
"0"or"NaN"skip validation, so invalid inputs slip through. Parse explicitly and enforce range on all provided values.- const timeoutInSecondsRaw = request.headers.get("Timeout-Seconds") ?? undefined; - const timeoutInSeconds = timeoutInSecondsRaw ? parseInt(timeoutInSecondsRaw) : undefined; - - if (timeoutInSeconds && isNaN(timeoutInSeconds)) { - return new Response("Invalid timeout seconds", { status: 400 }); - } - - if (timeoutInSeconds && timeoutInSeconds < 1) { - return new Response("Timeout seconds must be greater than 0", { status: 400 }); - } - - if (timeoutInSeconds && timeoutInSeconds > 600) { - return new Response("Timeout seconds must be less than 600", { status: 400 }); - } + const timeoutInSecondsRaw = request.headers.get("Timeout-Seconds"); + let timeoutInSeconds: number | undefined; + if (timeoutInSecondsRaw !== null) { + const value = Number(timeoutInSecondsRaw.trim()); + if (!Number.isFinite(value)) { + return new Response("Invalid timeout seconds", { status: 400 }); + } + if (value < 1) { + return new Response("Timeout seconds must be greater than 0", { status: 400 }); + } + if (value > 600) { + return new Response("Timeout seconds must be less than 600", { status: 400 }); + } + timeoutInSeconds = value; + }packages/core/test/streamsWriterV1.test.ts (1)
793-796: Fix generator-without-yield lint failure.Biome flags this generator because it never yields, causing CI to fail. Add a dead-code yield to retain behavior while satisfying the rule.
async function* generateChunks() { - // Yields nothing - return; + // Yields nothing; keep generator shape for lint compliance + if (false) { + yield undefined as never; + } }apps/webapp/app/routes/realtime.v1.streams.$runId.$target.$streamId.ts (4)
21-40: Query realtimeStreamsVersion for version-safe ingestion.The action handler needs to select
realtimeStreamsVersionfrom the database so it can use the authoritative version instead of the client-providedX-Stream-Versionheader (line 97). Without this field, the handler will trust the client to specify the correct version, creating a version mismatch risk.Apply this diff to the query:
const run = await $replica.taskRun.findFirst({ where: { friendlyId: params.runId, runtimeEnvironmentId: authentication.environment.id, }, select: { id: true, friendlyId: true, + realtimeStreamsVersion: true, parentTaskRun: { select: { friendlyId: true,
94-124: Use TaskRun's authoritative version instead of client header.The non-PUT handler currently trusts the client-provided
X-Stream-Versionheader (line 97) to select the stream backend, but each TaskRun has an authoritativerealtimeStreamsVersionfield that should be used instead (see the PUT handler at line 70 for the correct pattern). Allowing clients to specify the version creates a mismatch risk where v2 data could be routed to a v1 run's stream or vice versa.Apply this diff after adding
realtimeStreamsVersionto the query:- const clientId = request.headers.get("X-Client-Id") || "default"; - const streamVersion = request.headers.get("X-Stream-Version") || "v1"; + const clientId = request.headers.get("X-Client-Id") || "default"; if (!request.body) { return new Response("No body provided", { status: 400 }); } @@ - const realtimeStream = getRealtimeStreamInstance(authentication.environment, streamVersion); + const realtimeStream = getRealtimeStreamInstance( + authentication.environment, + run.realtimeStreamsVersion + );
133-154: Loader query missing realtimeStreamsVersion field.The loader's TaskRun query doesn't select
realtimeStreamsVersion, forcing the handler to fall back to the client-providedX-Stream-Versionheader (line 179). This creates the same version mismatch risk as in the action handler.Apply this diff:
select: { id: true, friendlyId: true, + realtimeStreamsVersion: true, parentTaskRun: { select: { friendlyId: true,
156-195: Use TaskRun version in loader instead of client header.Similar to the action handler, the loader should use
run.realtimeStreamsVersionfrom the database rather than the client-providedX-Stream-Versionheader (line 179).Apply this diff after adding
realtimeStreamsVersionto the query:// Extract client ID from header, default to "default" if not provided const clientId = request.headers.get("X-Client-Id") || "default"; - const streamVersion = request.headers.get("X-Stream-Version") || "v1"; - const realtimeStream = getRealtimeStreamInstance(authentication.environment, streamVersion); + const realtimeStream = getRealtimeStreamInstance( + authentication.environment, + run.realtimeStreamsVersion + );packages/core/src/v3/realtimeStreams/noopManager.ts (1)
11-23: Fix ReadableStream handling: source may not have an async iterator.
createAsyncIterableStreamFromAsyncIterableexpects anAsyncIterable<T>, but thesourceparameter acceptsAsyncIterable<T> | ReadableStream<T>. When aReadableStreamis passed, it will fail because ReadableStream doesn't implementSymbol.asyncIterator. You must detect the type and convert ReadableStream to AsyncIterable before calling the helper.Apply this diff to handle both types:
export class NoopRealtimeStreamsManager implements RealtimeStreamsManager { public pipe<T>( key: string, source: AsyncIterable<T> | ReadableStream<T>, options?: RealtimeStreamOperationOptions ): RealtimeStreamInstance<T> { + // Convert ReadableStream to AsyncIterable if needed + const asyncIterableSource = (source as any)[Symbol.asyncIterator] + ? (source as AsyncIterable<T>) + : (async function* () { + const reader = (source as ReadableStream<T>).getReader(); + try { + while (true) { + const { done, value } = await reader.read(); + if (done) return; + yield value; + } + } finally { + try { + reader.releaseLock(); + } catch {} + } + })(); + return { wait: () => Promise.resolve(), get stream(): AsyncIterableStream<T> { - return createAsyncIterableStreamFromAsyncIterable(source); + return createAsyncIterableStreamFromAsyncIterable( + asyncIterableSource, + undefined, + options?.signal + ); }, }; }packages/core/src/v3/realtimeStreams/streamInstance.ts (2)
29-63: Honor the requested target run.
StreamInstanceOptions.targetis accepted but never used: both thecreateStreamcall (line 32) and the v1 writer construction (line 49) are hard-coded to"self". As a result,streams.pipe(..., { target: "parent" | "root" })will silently write to the wrong run, breaking the documented targeting feature.Apply this diff:
private async initializeWriter(): Promise<StreamsWriterInstance<T>> { + const target = this.options.target ?? "self"; const { version, headers } = await this.options.apiClient.createStream( this.options.runId, - "self", + target, this.options.key, this.options?.requestOptions ); @@ ? new StreamsWriterV1({ key: this.options.key, runId: this.options.runId, source: this.options.source, baseUrl: this.options.baseUrl, headers: this.options.apiClient.getHeaders(), signal: this.options.signal, version, - target: "self", + target, })
69-95: Close the consumer stream when aborted.When the abort signal fires (line 79), the loop exits without calling
controller.close(). Consumers awaitingfor awaitwill hang indefinitely. You must close the controller and optionally calliterator.return()before exiting.Apply this diff:
while (true) { if (self.options.signal?.aborted) { + controller.close(); + if (typeof iterator.return === "function") { + await iterator.return(); + } - break; + return; }packages/core/src/v3/realtimeStreams/streamsWriterV1.ts (1)
63-86: Prevent unhandled rejection from buffer task.When the source stream errors or is aborted,
startBufferingrethrows (line 83) and the promise inbufferReaderTaskrejects with nobody awaiting it, producing an unhandled rejection. Additionally, the request loop (lines 220-254) has no way to detect the failure. Capture the error, release the reader lock, and make the sender fail fast.Apply this diff:
+ private bufferError: unknown; private bufferReaderTask: Promise<void> | null = null; private streamComplete = false; @@ private startBuffering(): void { this.streamReader = this.serverStream.getReader(); this.bufferReaderTask = (async () => { try { let chunkIndex = 0; while (true) { const { done, value } = await this.streamReader!.read(); if (done) { this.streamComplete = true; break; } // Add to ring buffer this.addToRingBuffer(chunkIndex, value); this.highestBufferedIndex = chunkIndex; chunkIndex++; } } catch (error) { - throw error; + this.bufferError = error; + this.streamComplete = true; + } finally { + try { + this.streamReader?.releaseLock(); + } catch {} + this.streamReader = null; } })(); + this.bufferReaderTask.catch(() => {}); // Prevent unhandled rejection }Then in
processStream(around line 224):try { let lastSentIndex = startFromChunk - 1; while (true) { + if (this.bufferError) { + reject( + this.bufferError instanceof Error + ? this.bufferError + : new Error(String(this.bufferError)) + ); + return; + } // Send all chunks that are available in bufferpackages/core/src/v3/realtimeStreams/types.ts (1)
25-28: Invalid interface accessor syntax; replace with readonly property.TypeScript interfaces cannot declare accessors using the
getkeyword. This will cause a compilation error.Apply this diff:
export interface RealtimeStreamInstance<T> { wait(): Promise<void>; - get stream(): AsyncIterableStream<T>; + readonly stream: AsyncIterableStream<T>; }As per coding guidelines, you may also consider converting the interface to a type alias to match the repo's preference for types over interfaces.
packages/core/src/v3/realtimeStreams/manager.ts (1)
43-48: User-provided abort signal is silently ignored whenAbortSignal.anyis unavailable.Line 47 uses optional chaining on
AbortSignal.any, which means if it's unavailable (older browsers or environments), the fallback?? abortController.signalcompletely ignores the user'soptions.signal, preventing stream cancellation.While Node 21/22 support
AbortSignal.any, this core SDK code may run in browser contexts where support varies.Apply this diff to manually combine signals when
anyis unavailable:// Create an AbortController for this stream const abortController = new AbortController(); // Chain with user-provided signal if present - const combinedSignal = options?.signal - ? AbortSignal.any?.([options.signal, abortController.signal]) ?? abortController.signal - : abortController.signal; + let combinedSignal: AbortSignal; + if (options?.signal) { + if (AbortSignal.any) { + combinedSignal = AbortSignal.any([options.signal, abortController.signal]); + } else { + // Manual combination: forward abort from user signal + combinedSignal = abortController.signal; + options.signal.addEventListener("abort", () => abortController.abort(), { once: true }); + } + } else { + combinedSignal = abortController.signal; + }
🧹 Nitpick comments (6)
apps/webapp/app/components/runs/v3/RunIcon.tsx (1)
111-112: Fix indentation to match other case statements.The return statement on line 112 has extra indentation compared to other cases in the switch statement. Please align it with the indentation pattern used in the rest of the file (e.g., lines 61, 65, 67).
Apply this diff to fix the indentation:
case "streams": - return <StreamsIcon className={cn(className, "text-text-dimmed")} />; + return <StreamsIcon className={cn(className, "text-text-dimmed")} />;apps/webapp/app/models/organization.server.ts (1)
55-55: Consider removing unusedfeaturesvariable.The
featuresvariable is fetched but no longer appears to be used after hardcodingv3Enabledtotrueon line 69. If it's not needed elsewhere in this function, consider removing it to eliminate dead code.Apply this diff if the variable is truly unused:
- const features = featuresForUrl(new URL(env.APP_ORIGIN)); - const organization = await prisma.organization.create({packages/trigger-sdk/src/v3/shared.ts (3)
434-456: Expose clientConfig to callers: widen requestOptions type.triggerAndWait currently accepts ApiRequestOptions, preventing callers from passing clientConfig. Align with other surfaces by using TriggerApiRequestOptions.
-export function triggerAndWait<TTask extends AnyTask>( +export function triggerAndWait<TTask extends AnyTask>( id: TaskIdentifier<TTask>, payload: TaskPayload<TTask>, options?: TriggerAndWaitOptions, - requestOptions?: ApiRequestOptions + requestOptions?: TriggerApiRequestOptions ): TaskRunPromise<TaskIdentifier<TTask>, TaskOutput<TTask>> {
482-493: Same as above for batchTriggerAndWait.Allow clientConfig via TriggerApiRequestOptions.
export async function batchTriggerAndWait<TTask extends AnyTask>( id: TaskIdentifier<TTask>, items: Array<BatchItem<TaskPayload<TTask>>>, options?: BatchTriggerAndWaitOptions, - requestOptions?: ApiRequestOptions + requestOptions?: TriggerApiRequestOptions ): Promise<BatchResult<TaskIdentifier<TTask>, TaskOutput<TTask>>> {
320-339: Parity: allow requestOptions in createSchemaTask.triggerAndWait.createTask.triggerAndWait gained requestOptions, but createSchemaTask.triggerAndWait didn’t. Add the parameter and forward it to triggerAndWait_internal for consistency.
- triggerAndWait: (payload, options) => { + triggerAndWait: (payload, options, requestOptions) => { return new TaskRunPromise<TIdentifier, TOutput>((resolve, reject) => { triggerAndWait_internal<TIdentifier, inferSchemaIn<TSchema>, TOutput>( "triggerAndWait()", params.id, payload, parsePayload, { queue: params.queue?.name, ...options, - } + }, + requestOptions )apps/webapp/app/v3/environmentVariables/environmentVariablesRepository.server.ts (1)
191-199: Optional: persist non-secret default explicitly.When creating EnvironmentVariableValue, isSecret is passed through possibly undefined. Explicitly default to false to match intended behavior during sync and queries.
- isSecret: options.isSecret, + isSecret: options.isSecret ?? false,
| case "realtime-stream": { | ||
| if (!span.entity.id) { | ||
| logger.error(`SpanPresenter: No realtime stream id`, { | ||
| spanId, | ||
| realtimeStreamId: span.entity.id, | ||
| }); | ||
| return { ...data, entity: null }; | ||
| } | ||
|
|
||
| const [runId, streamKey] = span.entity.id.split(":"); | ||
|
|
||
| if (!runId || !streamKey) { | ||
| logger.error(`SpanPresenter: Invalid realtime stream id`, { | ||
| spanId, | ||
| realtimeStreamId: span.entity.id, | ||
| }); | ||
| return { ...data, entity: null }; | ||
| } | ||
|
|
||
| const metadata = span.entity.metadata | ||
| ? (safeJsonParse(span.entity.metadata) as Record<string, unknown> | undefined) | ||
| : undefined; | ||
|
|
||
| return { | ||
| ...data, | ||
| entity: { | ||
| type: "realtime-stream" as const, | ||
| object: { | ||
| runId, | ||
| streamKey, | ||
| metadata, | ||
| }, | ||
| }, | ||
| }; | ||
| } |
There was a problem hiding this comment.
Make realtime-stream id parsing robust; normalize metadata null-to-undefined.
Split at the first colon to preserve streamKey with colons; ensure metadata is undefined (not null) on parse failure.
- const [runId, streamKey] = span.entity.id.split(":");
+ const idx = span.entity.id.indexOf(":");
+ const runId = idx > -1 ? span.entity.id.slice(0, idx) : "";
+ const streamKey = idx > -1 ? span.entity.id.slice(idx + 1) : "";
@@
- const metadata = span.entity.metadata
- ? (safeJsonParse(span.entity.metadata) as Record<string, unknown> | undefined)
- : undefined;
+ const metadata = span.entity.metadata
+ ? ((safeJsonParse(span.entity.metadata) ?? undefined) as
+ | Record<string, unknown>
+ | undefined)
+ : undefined;🤖 Prompt for AI Agents
In apps/webapp/app/presenters/v3/SpanPresenter.server.ts around lines 555 to
589, the realtime-stream id handling should split only at the first colon and
normalize metadata null to undefined: instead of span.entity.id.split(":"), find
the first colon (indexOf) and derive runId = id.slice(0, idx) and streamKey =
id.slice(idx + 1) so streamKey may contain additional colons; update the
invalid-id check accordingly. For metadata, call safeJsonParse and if it returns
null explicitly convert it to undefined before assigning metadata so the
returned entity.metadata is undefined on parse failure or absence.
| function define<TPart>(opts: RealtimeDefineStreamOptions): RealtimeDefinedStream<TPart> { | ||
| return { | ||
| id: opts.id, | ||
| pipe(value, options) { | ||
| return pipe(opts.id, value, options); | ||
| }, | ||
| read(runId, options) { | ||
| return read(runId, opts.id, options); | ||
| }, | ||
| append(value, options) { | ||
| return append(opts.id, value as BodyInit, options); | ||
| }, | ||
| writer(options) { | ||
| return writer(opts.id, options); | ||
| }, | ||
| }; | ||
| } |
There was a problem hiding this comment.
Add type constraint for append compatibility.
The define function doesn't constrain TPart to extend BodyInit, but the append method requires BodyInit. The cast on line 647 is unsafe and can cause runtime errors if TPart isn't compatible with BodyInit.
Consider one of these solutions:
Solution 1: Constrain TPart (recommended)
-function define<TPart>(opts: RealtimeDefineStreamOptions): RealtimeDefinedStream<TPart> {
+function define<TPart extends BodyInit>(opts: RealtimeDefineStreamOptions): RealtimeDefinedStream<TPart> {
return {
id: opts.id,
pipe(value, options) {
return pipe(opts.id, value, options);
},
read(runId, options) {
return read(runId, opts.id, options);
},
append(value, options) {
- return append(opts.id, value as BodyInit, options);
+ return append(opts.id, value, options);
},
writer(options) {
return writer(opts.id, options);
},
};
}Solution 2: Make append optional in RealtimeDefinedStream
If streams should support non-BodyInit types, make append optional in the RealtimeDefinedStream type definition (in core package) and document that append is only available for BodyInit-compatible types.
🤖 Prompt for AI Agents
In packages/trigger-sdk/src/v3/streams.ts around lines 637 to 653, the generic
TPart is not constrained but append calls require BodyInit and currently uses an
unsafe cast; change the function signature to constrain TPart to BodyInit (e.g.,
define<TPart extends BodyInit>(...)) and remove the cast on the append call so
the compiler ensures compatibility; if necessary, also update
RealtimeDefineStreamOptions / RealtimeDefinedStream types in the core package to
reflect the BodyInit constraint (or alternatively make append optional in the
stream type if non-BodyInit streams must be supported).
This PR introduces an upgraded Realtime streams backend and SDK that makes streams more reliable (and resumable) with increased or the removal of limits. We've also improved the visibily of streams via the run dashboard.
New limits
View the below limits table for more details:
Additionally, previously only a single client stream could be sent to a Realtime stream. Now, you can send multiple client streams to a single Realtime stream.
These new limits only apply when using the new streams backend powered by S2, and must be enabled explicitly when triggering a run by setting the new
v2RealtimeStreamsfuture flag when configuring your trigger client (or alternatively you can set theTRIGGER_V2_REALTIME_STREAMS=1environment variable in your backend code):Reliability improvements
When appending to a stream, the backend will now reliably resume appending from the last chunk index if there is a lost connection. Additionally, we've improved the reliability of reading from a stream by automatically resuming failed reads from the last chunk index if there is a lost connection.
This means that both sides of the stream will be much more reliable and will not lose data even when faced with network issues or other disruptions.
SDK improvements
We've moved the stream logic into their own dedicated namespace in the SDK instead of being mixed in with the other metadata methods:
You can now pipe to a stream using the
streams.pipemethod, which returns a result that can be used to wait until the stream is complete:When calling
streams.pipefrom inside a task, the stream is automatically associated with the current run. You can also optionally specify a target run ID to pipe to a stream on a different run:This means that, if you specify a target run ID, you can pipe to a stream outside of a task:
We've also added a new
streams.readmethod to read from a stream:You can also specify a timeout and start index to read from:
Default stream
Runs also now have a "default" stream which means you can optionally skip specifying a stream key:
streams.append/writer
You can append a single chunk to a stream using the
streams.appendmethod:Or you can use the
streams.writermethod to write multiple chunks to a stream or merge a stream into another stream:Both of these methods accept a target run ID to append/write to a stream on a different run:
streams.define
You can now define a stream in one place with the chunk type and the stream ID and use it in multiple places, DRYing up your code:
New useRealtimeStream hook
We've added a new
useRealtimeStreamhook to subscribe to a stream by its run ID and optional stream key:Just like the previous new functions, you can skip specifying the stream key when using
useRealtimeStream:You can also pass in the stream instance directly:
Dashboard improvements
We're now surfacing streams in the runs dashboard that will allow you to view the stream data in real-time:
CleanShot.2025-10-24.at.17.19.22.mp4