Skip to content

Commit 3c1cf65

Browse files
carderneTrigger.dev RepoOps
authored andcommitted
feat(convex): add experimental transactional task component
## Goal Validate whether a first-party Convex component makes Trigger.dev a useful, low-friction compute backend for Convex applications, without building new platform infrastructure before demand is established. The user-facing proposition is: **request heavy or long-running work transactionally from Convex, then consume its status and results through normal Convex reactivity.** This is for workloads that do not fit comfortably in Convex's runtime: video/audio processing with ffmpeg, browser automation with Playwright, Python workloads, memory-heavy imports, and long-running agent loops. If a job fits comfortably inside Convex, it should stay there. This complements Convex Workflow rather than competing with it as an orchestration framework. ## Scope and constraints This draft adds the experimental `@trigger.dev/convex` package under `packages/convex`. - **No new Trigger.dev server code or routes.** - **No Convex HTTP callback routes.** - No outbound webhook infrastructure or Trigger lifecycle-hook setup. - Use existing public Trigger APIs and short scheduled Convex actions. - Keep the initial feature set focused on validating the integration rather than wrapping the entire SDK. ## What Convex users can do - Trigger a task from a Convex mutation, atomically with their own application writes. - Pass typed payloads using type-only imports of Trigger task definitions, without bundling task code into Convex. - Supply supported trigger options: tags, delay, queue name, concurrency key, machine, metadata, max attempts, max duration, and TTL. - Deduplicate application requests with an optional component-level idempotency key. - Read run status, metadata, output, and errors using reactive queries, including a bounded paginated run list. - Run an optional application completion mutation in the same transaction that records completion. - Inspect paused synchronization and explicitly resume it after fixing the cause. Tasks are still authored and deployed normally with Trigger.dev, and retain normal task-side capabilities such as retries, queues, concurrency, child tasks, and waitpoints. The component does not need to expose every feature for tasks to use them. ## How it works ```text Convex mutation -> component record + scheduled submission, in the same transaction -> short action calls the existing Trigger task API -> Trigger.dev executes the actual workload -> short scheduled actions retrieve status with backoff -> component stores observations and terminal result -> Convex reactive queries update, and an optional app mutation applies completion ``` There is **no continuously running polling process in Convex**. Each check makes an HTTP request and exits; nothing runs there between checks. The heavy workload runs entirely on Trigger.dev. Polling backs off from approximately 15 to 60 seconds. This trades completion latency and API/function execution cost for a much smaller installation surface. It also observes terminal states such as cancellation without depending on task hooks executing successfully. ## Reliability and data handling - Application transaction rollback leaves neither a queued record nor a scheduled submission. - Remote submission uses a stable idempotency key derived from the component record ID, with a 24-hour TTL. Submission retries stop within the original one-hour window. This is bounded retry safety, **not an unlimited exactly-once guarantee**. - A watchdog is scheduled transactionally before each network action, so an interrupted action does not silently stop synchronization. Generation checks reject obsolete responses. - Permanent API errors, or eight consecutive transient/callback failures, pause synchronization and expose a synchronization error. Recovery uses an explicit `resume` operation. - Completion and application callback writes commit together. A throwing callback rolls them back together and is retried; a broken callback can delay the recorded completion state. - Credentials and request payloads are stored separately from queryable run data, and deleted after successful completion. Paused requests retain them for recovery. - Application functions remain responsible for user/tenant authorization. The README includes an ownership-checked query example. - Inputs and inline result fields have explicit size budgets. Large output can be omitted with an indicator; supplied output presigned URLs are preserved rather than downloaded. - Run lists have bounded page counts and read budgets. ## Intentionally deferred Batch triggering, cancellation commands, waitpoint completion, schedules, streamed progress, preview-environment mapping, automatic retention cleanup, CLI scaffolding, and platform outbound status webhooks. Observing that a run was canceled is supported; issuing cancellation through this component is not yet supported. This MVP is not intended for high-volume fan-out or subsecond progress. Polling costs, retained records, expiring presigned URLs, credential rotation, and recovery limits are documented rather than hidden. ## Validation - Package build passes. - Fourteen integration tests pass using Convex's in-memory runtime and real HTTP requests to a local server, without mocking fetch. - Coverage includes transaction rollback, request deduplication, retrying a lost submission response with the same remote key, transient polling failures, cancellation detection, active status/progress, completion callback rollback/recovery, and input/output limits. - Regression tests also cover EXPIRED/TIMED_OUT responses with isCompleted=false, retryable 404 responses, ordinary permanent 404 responses, and explicit x-should-retry=false overrides. - Test source typechecking, scoped lint, scoped Knip, whitespace checks, and npm package dry-run checks pass. ## Why this is a draft / remaining work - [x] Validate installation and generated component references in a live Convex application using a local unpublished tarball. - [x] Execute a real Trigger.dev task and verify reactive progress/status/results, failure, cancellation detection, completion callbacks, rollback, deduplication, and credential recovery end to end. - [ ] Measure polling latency, Convex function execution cost, and Trigger API traffic with representative workloads. - [ ] Get feedback from Convex users on whether this removes meaningful integration work. - [ ] Decide whether to publish the experimental package and pursue a Convex component directory listing. Live validation now passes against a hosted Convex development deployment and Trigger.dev test cloud. The repeatable test suite completed in 81.6 seconds with 20 subscription updates. A 45-second synthetic Node crypto task completed successfully; its result appeared in Convex about 14.4 seconds after remote completion. Failure, cancellation detection, progress, atomic callbacks, rollback, deduplication, and recovery from an invalid credential also passed. Completed runs left zero retained request/credential records. The local example uses an unpublished tarball and Trigger CLI local bundling, so neither registry publication nor new server infrastructure is required. This validates the integration, not high-volume economics or workloads beyond Convex's duration limit. Representative compute workloads and demand validation remain open. The first live app check also exposed circular TypeScript inference when registering a callback through the generated API. The README now explicitly types the Trigger instance to avoid it. Mono-RevId: e550894024d815463d26a8188ae57ce094f7ce77
1 parent b470f8a commit 3c1cf65

16 files changed

Lines changed: 1538 additions & 0 deletions

File tree

‎.changeset/convex-component.md‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
---
2+
"@trigger.dev/convex": patch
3+
---
4+
5+
Add an experimental Convex component for triggering tasks transactionally and reading their status and results through reactive queries, without configuring webhooks.

‎packages/convex/README.md‎

Lines changed: 172 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,172 @@
1+
# @trigger.dev/convex
2+
3+
Run Trigger.dev tasks from Convex mutations and read their status and results through Convex reactive queries.
4+
5+
Use Convex for application state and Trigger.dev for work that needs longer execution, more memory, ffmpeg, Playwright, Python, or other system dependencies. Tasks are still written and deployed normally with Trigger.dev.
6+
7+
**Experimental MVP.** No HTTP routes, webhooks, Trigger lifecycle hooks, or changes to your Trigger server are required.
8+
9+
## Install
10+
11+
```sh
12+
npm install @trigger.dev/convex convex
13+
npx convex env set TRIGGER_SECRET_KEY tr_dev_...
14+
```
15+
16+
Choose the Trigger environment's secret key. Never expose it to browser code.
17+
18+
```ts
19+
// convex/convex.config.ts
20+
import { defineApp } from "convex/server";
21+
import trigger from "@trigger.dev/convex/convex.config.js";
22+
23+
const app = defineApp();
24+
app.use(trigger);
25+
export default app;
26+
```
27+
28+
Run `npx convex dev` to generate your application's component references.
29+
30+
```ts
31+
// convex/trigger.ts
32+
import { Trigger } from "@trigger.dev/convex";
33+
import { components, internal } from "./_generated/api";
34+
35+
export const trigger: Trigger = new Trigger(components.trigger, {
36+
onComplete: internal.jobs.onComplete, // Optional internal mutation
37+
});
38+
```
39+
40+
The explicit `Trigger` annotation avoids circular type inference through Convex's generated API when configuring a completion callback.
41+
42+
## Example: request work and read the result
43+
44+
The example assumes a Trigger task named `render-video` exists. Type-only task imports do not bundle task code into Convex.
45+
46+
```ts
47+
// convex/schema.ts
48+
import { defineSchema, defineTable } from "convex/server";
49+
import { v } from "convex/values";
50+
51+
export default defineSchema({
52+
jobs: defineTable({
53+
owner: v.string(),
54+
runId: v.optional(v.string()),
55+
status: v.string(),
56+
output: v.optional(v.any()),
57+
}).index("by_runId", ["runId"]),
58+
});
59+
```
60+
61+
```ts
62+
// convex/jobs.ts
63+
import { vRun } from "@trigger.dev/convex";
64+
import { v } from "convex/values";
65+
import { mutation, query, internalMutation } from "./_generated/server";
66+
import { trigger } from "./trigger";
67+
import type { renderVideo } from "../trigger/renderVideo";
68+
69+
export const request = mutation({
70+
args: { videoUrl: v.string() },
71+
handler: async (ctx, { videoUrl }) => {
72+
const user = await ctx.auth.getUserIdentity();
73+
if (!user) throw new Error("Unauthorized");
74+
75+
const jobId = await ctx.db.insert("jobs", {
76+
owner: user.subject,
77+
status: "PENDING",
78+
});
79+
const runId = await trigger.trigger<typeof renderVideo>(
80+
ctx,
81+
"render-video",
82+
{ videoUrl },
83+
{ tags: [`job:${jobId}`] },
84+
);
85+
await ctx.db.patch(jobId, { runId });
86+
return jobId;
87+
},
88+
});
89+
90+
export const status = query({
91+
args: { jobId: v.id("jobs") },
92+
handler: async (ctx, { jobId }) => {
93+
const user = await ctx.auth.getUserIdentity();
94+
const job = await ctx.db.get(jobId);
95+
if (!user || !job || job.owner !== user.subject) return null;
96+
return job.runId ? trigger.getRun(ctx, job.runId) : null;
97+
},
98+
});
99+
100+
export const onComplete = internalMutation({
101+
args: { run: vRun },
102+
handler: async (ctx, { run }) => {
103+
const job = await ctx.db.query("jobs")
104+
.withIndex("by_runId", q => q.eq("runId", run.id))
105+
.unique();
106+
if (job) await ctx.db.patch(job._id, {
107+
status: run.status,
108+
output: run.output,
109+
});
110+
},
111+
});
112+
```
113+
114+
```tsx
115+
// React
116+
const run = useQuery(api.jobs.status, { jobId });
117+
// run.status, run.output, run.error, run.metadata, run.syncStatus
118+
```
119+
120+
The ID returned by `trigger.trigger` identifies the **component record**, not the Trigger.dev run. The latter becomes available as `run.runId` after submission.
121+
122+
## API
123+
124+
- `trigger(ctx, taskId, payload, options?)`: enqueue work. Calling from a mutation makes the request atomic with your application's writes.
125+
- `getRun(ctx, id)`: read one record reactively.
126+
- `listRuns(ctx, paginationOpts)`: paginate records, newest first, up to 100 per page and a bounded read budget.
127+
- `resume(ctx, id)`: resume paused synchronization using the current credentials.
128+
129+
Trigger options include tags, delay, queue name, concurrency key, machine, metadata, max attempts, max duration, TTL, and a component-level idempotency key.
130+
131+
An optional `idempotencyKey` deduplicates requests across this component instance, regardless of task. Reusing a key returns the original record, even if its payload or task differs. Keys remain reserved while records exist.
132+
133+
The constructor accepts `secretKey` (defaults to `TRIGGER_SECRET_KEY`), `baseUrl` (defaults to `https://api.trigger.dev`), and `onComplete`. HTTP base URLs are permitted only for localhost development.
134+
135+
**Authorization belongs in your application's functions.** The component has no tenant or user model. Do not expose arbitrary record IDs or `listRuns` to clients without enforcing ownership. Queries never return stored credentials or request payloads.
136+
137+
## Execution and reliability
138+
139+
1. The calling mutation writes a component record and schedules submission. If the transaction rolls back, neither survives.
140+
2. A short action submits to the existing Trigger API with a stable key derived from the record ID.
141+
3. Scheduled actions retrieve status with backoff from approximately 15 to 60 seconds between checks. Nothing runs in Convex between checks.
142+
4. A terminal state, including cancellation, is persisted. The optional completion mutation runs in the same transaction, then stored request data and credentials are deleted.
143+
144+
A scheduled watchdog is committed before each network action. An interrupted action is retried on the next check; obsolete responses cannot overwrite newer observations.
145+
146+
Remote submission uses a 24-hour idempotency TTL. Automatic submission retries stop within one hour of creating the record, well before that TTL. This is **not an unlimited exactly-once guarantee**.
147+
148+
Permanent API errors pause synchronization immediately. Eight consecutive transient or callback failures also pause it. Check `syncStatus === "paused"` and `syncError`. Fix the cause, then call `resume` from an authorized mutation. If submission has not been confirmed, it can only resume within the original one-hour window. After that, reconcile with the Trigger dashboard before creating new work.
149+
150+
Keep the same Trigger environment when resuming. Changing credentials does not migrate remote runs between environments. Existing requests retain their original credentials until resumed.
151+
152+
Completion callbacks should be short mutations. If a callback throws, completion and its application writes roll back together and are retried. A broken callback can therefore delay the component's completion state; `syncError` reports synchronization trouble. No callback is needed to use reactive results.
153+
154+
## Limits
155+
156+
- Polling costs Convex executions and Trigger API requests. This MVP is for validating the integration, not high-volume fan-out or subsecond progress.
157+
- Payload plus trigger options must fit within 256 KiB. Pass storage references for larger inputs.
158+
- Output, error, and metadata each have a 256 KiB inline budget. Oversized output is omitted with `outputOmitted: true`. Oversized error or metadata is omitted.
159+
- Trigger's `outputPresignedUrl`, when supplied, is preserved rather than downloaded. These URLs expire. Retrieve the run again through the Trigger API when you need a fresh URL.
160+
- Run records are retained. Paused requests retain their payload and credentials for recovery. Automatic retention cleanup is not included.
161+
- No batch API, cancellation command, waitpoint completion, schedules, or streaming yet. Trigger tasks themselves can use those features normally.
162+
163+
## Development
164+
165+
From the repository root:
166+
167+
```sh
168+
pnpm --filter @trigger.dev/convex build
169+
pnpm --filter @trigger.dev/convex test
170+
```
171+
172+
Tests use Convex's in-memory test runtime and real HTTP requests to a local server.

‎packages/convex/package.json‎

Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,49 @@
1+
{
2+
"name": "@trigger.dev/convex",
3+
"version": "0.0.1",
4+
"description": "Run Trigger.dev tasks from Convex mutations and query their results reactively",
5+
"license": "Apache-2.0",
6+
"type": "module",
7+
"publishConfig": {
8+
"access": "public"
9+
},
10+
"repository": {
11+
"type": "git",
12+
"url": "https://github.com/triggerdotdev/trigger.dev",
13+
"directory": "packages/convex"
14+
},
15+
"files": [
16+
"dist",
17+
"src/component",
18+
"!src/component/**/*.test.ts"
19+
],
20+
"exports": {
21+
".": {
22+
"types": "./dist/client/index.d.ts",
23+
"default": "./dist/client/index.js"
24+
},
25+
"./convex.config.js": {
26+
"types": "./dist/component/convex.config.d.ts",
27+
"default": "./dist/component/convex.config.js"
28+
},
29+
"./_generated/component.js": {
30+
"types": "./dist/component/_generated/component.d.ts"
31+
},
32+
"./package.json": "./package.json"
33+
},
34+
"scripts": {
35+
"build": "rm -rf dist && tsc -p tsconfig.json",
36+
"test": "vitest run",
37+
"typecheck": "tsc -p tsconfig.json --noEmit"
38+
},
39+
"peerDependencies": {
40+
"convex": "^1.48.0"
41+
},
42+
"devDependencies": {
43+
"@types/node": "24.13.3",
44+
"convex": "^1.48.0",
45+
"convex-test": "^0.0.65",
46+
"typescript": "catalog:",
47+
"vitest": "4.1.11"
48+
}
49+
}
Lines changed: 81 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,81 @@
1+
import {
2+
createFunctionHandle,
3+
type FunctionReference,
4+
type GenericDataModel,
5+
type GenericMutationCtx,
6+
type GenericQueryCtx,
7+
type PaginationOptions,
8+
} from "convex/server";
9+
import type { ComponentApi } from "../component/_generated/component.js";
10+
import type { Config, Run, TriggerOptions } from "../component/shared.js";
11+
12+
export { vRun } from "../component/shared.js";
13+
export type { Run, TriggerOptions, ComponentApi };
14+
type MutationCtx = Pick<GenericMutationCtx<GenericDataModel>, "runMutation">;
15+
type QueryCtx = Pick<GenericQueryCtx<GenericDataModel>, "runQuery">;
16+
type TaskLike = { id: string; trigger: (...args: any[]) => any };
17+
18+
export type TriggerConfig = {
19+
secretKey?: string;
20+
baseUrl?: string;
21+
onComplete?: FunctionReference<"mutation", "internal", { run: Run }, any>;
22+
};
23+
24+
export class Trigger {
25+
constructor(
26+
private component: ComponentApi,
27+
private config: TriggerConfig = {}
28+
) {}
29+
30+
private runtimeConfig(): Config {
31+
const secretKey = this.config.secretKey ?? process.env.TRIGGER_SECRET_KEY;
32+
if (!secretKey) throw new Error("Set TRIGGER_SECRET_KEY or pass secretKey");
33+
const url = new URL(this.config.baseUrl ?? "https://api.trigger.dev");
34+
if (
35+
url.protocol !== "https:" &&
36+
!(url.protocol === "http:" && ["localhost", "127.0.0.1", "[::1]"].includes(url.hostname))
37+
) {
38+
throw new Error("baseUrl must use HTTPS, except for local development");
39+
}
40+
if (url.username || url.password || url.search || url.hash) throw new Error("Invalid baseUrl");
41+
return { secretKey, baseUrl: url.toString().replace(/\/$/, "") };
42+
}
43+
44+
async trigger<TTask extends TaskLike = TaskLike>(
45+
ctx: MutationCtx,
46+
taskId: TTask["id"],
47+
payload: Parameters<TTask["trigger"]>[0],
48+
options: TriggerOptions & { idempotencyKey?: string } = {}
49+
): Promise<string> {
50+
const { idempotencyKey, ...triggerOptions } = options;
51+
return ctx.runMutation(this.component.runs.trigger, {
52+
taskId,
53+
payload,
54+
options: triggerOptions,
55+
idempotencyKey,
56+
config: this.runtimeConfig(),
57+
callback: this.config.onComplete
58+
? await createFunctionHandle(this.config.onComplete)
59+
: undefined,
60+
});
61+
}
62+
63+
async getRun(ctx: QueryCtx, id: string): Promise<Run | null> {
64+
return ctx.runQuery(this.component.runs.get, { id });
65+
}
66+
67+
async listRuns(
68+
ctx: QueryCtx,
69+
paginationOpts: PaginationOptions
70+
): Promise<{
71+
page: Run[];
72+
isDone: boolean;
73+
continueCursor: string;
74+
}> {
75+
return ctx.runQuery(this.component.runs.list, { paginationOpts });
76+
}
77+
78+
async resume(ctx: MutationCtx, id: string): Promise<null> {
79+
return ctx.runMutation(this.component.runs.resume, { id, config: this.runtimeConfig() });
80+
}
81+
}
Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
import type { ApiFromModules, FilterApi, FunctionReference } from "convex/server";
2+
import type * as runs from "../runs.js";
3+
4+
type PublicApi = FilterApi<ApiFromModules<{ runs: typeof runs }>, FunctionReference<any, "public">>;
5+
6+
// Derive the component boundary from its functions, so it cannot drift from their validators.
7+
export type ComponentApi<Name extends string | undefined = string | undefined> = {
8+
runs: {
9+
[Key in keyof PublicApi["runs"]]: PublicApi["runs"][Key] extends FunctionReference<
10+
infer Type,
11+
any,
12+
infer Args,
13+
infer Result
14+
>
15+
? FunctionReference<Type, "internal", Args, Result, Name>
16+
: never;
17+
};
18+
};
Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
import { anyApi, type ApiFromModules, type FilterApi, type FunctionReference } from "convex/server";
2+
import type * as runs from "./runs.js";
3+
import type * as worker from "./worker.js";
4+
5+
type Api = ApiFromModules<{ runs: typeof runs; worker: typeof worker }>;
6+
export const internal = anyApi as unknown as FilterApi<Api, FunctionReference<any, "internal">>;
Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
import { MAX_INLINE_BYTES, REQUEST_TIMEOUT_MS, type Config } from "./shared.js";
2+
3+
export class ApiError extends Error {
4+
constructor(
5+
message: string,
6+
public retryable: boolean
7+
) {
8+
super(message);
9+
}
10+
}
11+
12+
export async function request(
13+
config: Config,
14+
path: string,
15+
body?: unknown
16+
): Promise<Record<string, unknown>> {
17+
const controller = new AbortController();
18+
const timeout = setTimeout(() => controller.abort(), REQUEST_TIMEOUT_MS);
19+
try {
20+
const response = await fetch(config.baseUrl + path, {
21+
method: body === undefined ? "GET" : "POST",
22+
headers: { Authorization: `Bearer ${config.secretKey}`, "Content-Type": "application/json" },
23+
body: body === undefined ? undefined : JSON.stringify(body),
24+
signal: controller.signal,
25+
redirect: "error",
26+
});
27+
if (!response.ok) {
28+
await response.body?.cancel();
29+
const retryHeader = response.headers.get("x-should-retry");
30+
// Never persist response bodies, which could contain credentials or payloads.
31+
throw new ApiError(
32+
`Trigger.dev API returned HTTP ${response.status}`,
33+
retryHeader === "true" ||
34+
(retryHeader !== "false" &&
35+
(response.status === 408 ||
36+
response.status === 409 ||
37+
response.status === 429 ||
38+
response.status >= 500))
39+
);
40+
}
41+
const data: unknown = await response.json();
42+
if (!data || typeof data !== "object" || Array.isArray(data)) {
43+
throw new ApiError("Invalid Trigger.dev response", true);
44+
}
45+
return data as Record<string, unknown>;
46+
} finally {
47+
clearTimeout(timeout);
48+
}
49+
}
50+
51+
export function inlineValue(value: unknown): { value?: any; omitted: boolean } {
52+
if (value === undefined) return { omitted: false };
53+
if (new TextEncoder().encode(JSON.stringify(value)).byteLength > MAX_INLINE_BYTES) {
54+
return { omitted: true };
55+
}
56+
return { value, omitted: false };
57+
}
Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
import { defineComponent } from "convex/server";
2+
3+
export default defineComponent("trigger");

0 commit comments

Comments
 (0)