Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .craft.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,9 @@ targets:
- name: npm
id: "@sentry/junior"
includeNames: /^sentry-junior-\d.*\.tgz$/
- name: npm
id: "@sentry/junior-migrations"
includeNames: /^sentry-junior-migrations-\d.*\.tgz$/
- name: npm
id: "@sentry/junior-plugin-api"
includeNames: /^sentry-junior-plugin-api-\d.*\.tgz$/
Expand Down
3 changes: 3 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,7 @@ jobs:
- .github/actions/**
core:
- packages/junior/**
- packages/junior-migrations/**
- packages/junior-plugin-api/**
- packages/junior-scheduler/**
- packages/junior-testing/**
Expand Down Expand Up @@ -216,6 +217,7 @@ jobs:
- run: pnpm --filter @sentry/junior-memory build
- run: pnpm --filter @sentry/junior-github build
- run: pnpm --filter @sentry/junior-vercel build
- run: pnpm --filter @sentry/junior-migrations test
- run: pnpm --filter @sentry/junior-memory test
- run: pnpm --filter @sentry/junior-github test
- run: pnpm --filter @sentry/junior-vercel test
Expand Down Expand Up @@ -343,6 +345,7 @@ jobs:
run: |
mkdir -p artifacts
pnpm --filter @sentry/junior pack --pack-destination artifacts
pnpm --filter @sentry/junior-migrations pack --pack-destination artifacts
pnpm --filter @sentry/junior-plugin-api pack --pack-destination artifacts
pnpm --filter @sentry/junior-agent-browser pack --pack-destination artifacts
pnpm --filter @sentry/junior-amplitude pack --pack-destination artifacts
Expand Down
3 changes: 3 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,8 @@ pnpm release:check
- `policies/` contains durable repo-wide engineering rules.
- Package and module `README.md` files explain implemented architecture and
non-obvious invariants near the code they describe.
- The [database migration guide](https://junior.sentry.dev/contribute/database-migrations/)
explains when and how to generate schema and data migrations.
- Code, schemas, exported types, and tests are the implementation authority.
- `openspec/changes/` contains temporary implementation plans. Move durable
explanation beside the code and delete a plan when its work is complete.
Expand Down Expand Up @@ -119,6 +121,7 @@ pnpm build:pkg
This repo uses Craft for manual lockstep npm releases of:

- `@sentry/junior`
- `@sentry/junior-migrations`
- `@sentry/junior-plugin-api`
- `@sentry/junior-agent-browser`
- `@sentry/junior-amplitude`
Expand Down
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ Start here:
| Package | Purpose |
| ------------------------------ | ---------------------------------------------------------------------------- |
| `@sentry/junior` | Core Slack bot runtime |
| `@sentry/junior-migrations` | Mixed Drizzle SQL and TypeScript migration format and runner |
| `@sentry/junior-plugin-api` | Lightweight plugin API types and helpers |
| `@sentry/junior-agent-browser` | Agent Browser plugin package for browser automation |
| `@sentry/junior-amplitude` | Read-only Amplitude product analytics through Amplitude's hosted MCP server |
Expand Down
2 changes: 1 addition & 1 deletion ast-grep/rules/no-core-plugin-dynamic-imports.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,4 +12,4 @@ constraints:
all:
- regex: '^["@'']@sentry/junior-.+["@'']$'
- not:
regex: '^["@'']@sentry/junior-plugin-api["@'']$'
regex: '^["@'']@sentry/junior-(?:plugin-api|migrations)["@'']$'
2 changes: 1 addition & 1 deletion ast-grep/rules/no-core-plugin-reexports.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,4 +14,4 @@ rule:
- not:
has:
kind: string
regex: '^["@'']@sentry/junior-plugin-api["@'']$'
regex: '^["@'']@sentry/junior-(?:plugin-api|migrations)["@'']$'
2 changes: 1 addition & 1 deletion ast-grep/rules/no-core-plugin-static-imports.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,4 +14,4 @@ rule:
- not:
has:
kind: string
regex: '^["@'']@sentry/junior-plugin-api["@'']$'
regex: '^["@'']@sentry/junior-(?:plugin-api|migrations)["@'']$'
8 changes: 4 additions & 4 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -12,19 +12,19 @@
"worktree:setup": "node scripts/worktree.mjs setup",
"cloudflare:token": "node scripts/refresh-cloudflare-tunnel-token.mjs",
"prepare": "simple-git-hooks",
"lint": "pnpm --filter @sentry/junior lint && pnpm --filter @sentry/junior-memory lint && pnpm --filter @sentry/junior-github lint && pnpm --filter @sentry/junior-vercel lint && pnpm ast-grep:lint && pnpm package:lint",
"lint": "pnpm --filter @sentry/junior-migrations lint && pnpm --filter @sentry/junior lint && pnpm --filter @sentry/junior-memory lint && pnpm --filter @sentry/junior-github lint && pnpm --filter @sentry/junior-vercel lint && pnpm ast-grep:lint && pnpm package:lint",
"lint:fix": "pnpm --filter @sentry/junior lint:fix",
"ast-grep:lint": "ast-grep scan",
"lint-staged": "lint-staged",
"build": "pnpm --filter @sentry/junior build && pnpm --filter @sentry/junior-memory build && pnpm --filter @sentry/junior-github build && pnpm --filter @sentry/junior-vercel build && pnpm --filter @sentry/junior-dashboard build",
"build": "pnpm --filter @sentry/junior-migrations build && pnpm --filter @sentry/junior build && pnpm --filter @sentry/junior-memory build && pnpm --filter @sentry/junior-github build && pnpm --filter @sentry/junior-vercel build && pnpm --filter @sentry/junior-dashboard build",
"build:example": "pnpm --filter @sentry/junior-example build",
"docs:dev": "pnpm --filter @sentry/junior-docs dev",
"docs:build": "pnpm --filter @sentry/junior-docs build",
"docs:check": "pnpm --filter @sentry/junior-docs check",
"package:lint": "for pkg in packages/junior packages/junior-plugin-api packages/junior-scheduler packages/junior-memory packages/junior-dashboard packages/junior-github packages/junior-agent-browser packages/junior-amplitude packages/junior-cloudflare packages/junior-datadog packages/junior-hex packages/junior-linear packages/junior-maintenance packages/junior-notion packages/junior-sentry packages/junior-vercel; do pnpm exec publint \"$pkg\" || exit $?; done",
"package:lint": "for pkg in packages/junior packages/junior-migrations packages/junior-plugin-api packages/junior-scheduler packages/junior-memory packages/junior-dashboard packages/junior-github packages/junior-agent-browser packages/junior-amplitude packages/junior-cloudflare packages/junior-datadog packages/junior-hex packages/junior-linear packages/junior-maintenance packages/junior-notion packages/junior-sentry packages/junior-vercel; do pnpm exec publint \"$pkg\" || exit $?; done",
"release:check": "node scripts/check-release-config.mjs",
"start": "pnpm --filter @sentry/junior-example dev",
"test": "pnpm --filter @sentry/junior build && pnpm --filter @sentry/junior-memory build && pnpm --filter @sentry/junior-github build && pnpm --filter @sentry/junior-vercel build && pnpm --filter @sentry/junior-dashboard build && pnpm --filter @sentry/junior test && pnpm --filter @sentry/junior-memory test && pnpm --filter @sentry/junior-github test && pnpm --filter @sentry/junior-vercel test && pnpm --filter @sentry/junior-dashboard test",
"test": "pnpm --filter @sentry/junior-migrations test && pnpm --filter @sentry/junior build && pnpm --filter @sentry/junior-memory build && pnpm --filter @sentry/junior-github build && pnpm --filter @sentry/junior-vercel build && pnpm --filter @sentry/junior-dashboard build && pnpm --filter @sentry/junior test && pnpm --filter @sentry/junior-memory test && pnpm --filter @sentry/junior-github test && pnpm --filter @sentry/junior-vercel test && pnpm --filter @sentry/junior-dashboard test",
"test:e2e:dashboard": "pnpm --filter @sentry/junior-dashboard build && playwright test -c packages/junior-dashboard/playwright.config.ts",
"test:watch": "pnpm --filter @sentry/junior test:watch",
"evals": "pnpm --filter @sentry/junior-evals evals",
Expand Down
9 changes: 5 additions & 4 deletions packages/docs/astro.config.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -178,6 +178,10 @@ export default defineConfig({
label: "Contribute",
items: [
{ label: "Development", link: "/contribute/development/" },
{
label: "Database Migrations",
link: "/contribute/database-migrations/",
},
{ label: "Testing", link: "/contribute/testing/" },
{
label: "Local Agent Validation",
Expand All @@ -191,10 +195,7 @@ export default defineConfig({
],
},
],
plugins: [
sentryStarlightTheme(),
sentryAgentMarkdown(),
],
plugins: [sentryStarlightTheme(), sentryAgentMarkdown()],
}),
],
markdown: {
Expand Down
10 changes: 5 additions & 5 deletions packages/docs/src/content/docs/cli/upgrade.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: "junior upgrade"
description: "Apply Junior and plugin SQL schema migrations."
description: "Apply Junior and plugin schema or data migrations."
type: reference
summary: Bring the configured Junior SQL database up to the installed schema.
prerequisites:
Expand All @@ -12,7 +12,8 @@ related:
---

Use `junior upgrade` after installing a new Junior release. The command applies
pending Drizzle migrations to the database configured by `DATABASE_URL`.
pending SQL schema and TypeScript data entries from the ordered Drizzle journals
owned by Junior and its enabled plugins.

## Usage

Expand All @@ -22,9 +23,8 @@ Run the command from your Junior app:
pnpm exec junior upgrade
```

The command takes no arguments. It migrates the core Junior schema first, then
the schemas owned by enabled plugins. Already-applied migrations are recognized
as up to date and are not rerun.
The command takes no arguments. It migrates Junior first, then enabled plugins.
Already-applied entries are recognized as up to date and are not rerun.

## Upgrade bridge for older databases

Expand Down
113 changes: 113 additions & 0 deletions packages/docs/src/content/docs/contribute/database-migrations.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,113 @@
---
title: Database Migrations
description: Generate and review Junior schema and data migrations.
type: tutorial
summary: Choose the right migration type, generate it in the owning package, and verify the result.
prerequisites:
- /contribute/development/
related:
- /cli/upgrade/
- /contribute/testing/
- /contribute/releasing/
---

Junior keeps schema and data migrations in the same ordered Drizzle journal.
Each journal entry has exactly one source file:

| Change | Generate | Result |
| ----------------------------------------------------------------------------- | ---------------- | ----------------------------------- |
| Tables, columns, indexes, or a simple SQL backfill | Schema migration | `<tag>.sql` plus a Drizzle snapshot |
| Data that needs application code, state storage, Redis, or resumable progress | Data migration | `<tag>.ts` with no new snapshot |

Generate the migration in the package that owns the schema. Core Junior uses
`@sentry/junior`; plugin tables use their plugin package.

## Generate a schema migration

First change the owning Drizzle schema, such as
`packages/junior/src/db/schema.ts`. Then run:

```bash
pnpm --filter @sentry/junior db:generate --name add_delivery_status
```

For a plugin, replace the package name:

```bash
pnpm --filter @sentry/junior-memory db:generate --name add_memory_source
```

Drizzle adds a SQL file, updates `meta/_journal.json`, and writes a schema
snapshot. The generated SQL looks like:

```sql title="migrations/0007_add_delivery_status.sql"
ALTER TABLE "junior_conversations"
ADD COLUMN "delivery_status" text;
```

Review the SQL against the schema change. Prefer SQL when the work is entirely
inside the same database and does not need application-level decoding.

## Generate a data migration

Use a data migration when SQL is not enough—for example, when moving data from
state storage, preserving a Redis index, decoding an old application record, or
checkpointing a long backfill.

```bash
pnpm --filter @sentry/junior db:generate:data --name backfill_delivery_status
```

The command adds a TypeScript entry to the same journal:

```ts title="migrations/0008_backfill_delivery_status.ts"
import type { MigrationV1 } from "@sentry/junior-migrations";

const migration = {
apiVersion: 1,
async up(context) {
void context;
},
} satisfies MigrationV1;

export default migration;
```

Implement `up` with the capabilities on `context`:

- `database` for queries, transactions, and database locks
- `state` for Junior or plugin-scoped state
- `redis` when a migration must preserve raw Redis structures
- `progress` for a checkpoint that survives a failed or interrupted run
- `log` for operator-facing progress messages

Keep one-off decoding and transformation logic in the migration file. Do not
import current Junior runtime modules from a data migration.

## Review and verify

Before committing:

1. Confirm the new journal entry has exactly one `.sql` or `.ts` file.
2. Commit the updated `meta/_journal.json`.
3. Commit the new snapshot for a schema migration; a data migration should not
add one.
4. Do not edit, rename, reorder, or delete a migration that has shipped.
5. Make long-running data migrations safe to retry and use `progress` when
partial work must resume.

Run the focused migration checks:

```bash
pnpm --filter @sentry/junior-migrations test
pnpm typecheck
```

`junior upgrade` runs schema and data entries in journal order. Schema-bootstrap
mode is only for constructing empty test databases; it is not an upgrade path
for an existing installation.

## Next step

Add focused coverage using the guidance in [Testing](/contribute/testing/),
then review the release path in [Releasing](/contribute/releasing/).
1 change: 1 addition & 0 deletions packages/docs/src/content/docs/contribute/development.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ type: tutorial
summary: Set up Junior locally, run checks, and use isolated worktrees for parallel agent or contributor tasks.
prerequisites: []
related:
- /contribute/database-migrations/
- /contribute/testing/
- /contribute/releasing/
- /start-here/quickstart/
Expand Down
1 change: 1 addition & 0 deletions packages/docs/src/content/docs/contribute/releasing.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ related:
Junior uses lockstep package releases for:

- `@sentry/junior`
- `@sentry/junior-migrations`
- `@sentry/junior-plugin-api`
- `@sentry/junior-agent-browser`
- `@sentry/junior-amplitude`
Expand Down
6 changes: 3 additions & 3 deletions packages/junior-evals/postgres-global-setup.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ import {
type PostgresHarnessConfig,
} from "@sentry/junior-testing/postgres";
import { migrateSchema } from "@/chat/conversations/sql/migrations";
import { migratePluginSchemas } from "@/chat/plugins/migrations";
import { bootstrapPluginSchemas } from "@/chat/plugins/migrations";
import { createPostgresJuniorSqlExecutor } from "@/db/postgres";

const workspaceRoot = path.resolve(
Expand Down Expand Up @@ -43,8 +43,8 @@ export default async function setup(
migrateTemplate: async (connectionString) => {
const executor = createPostgresJuniorSqlExecutor({ connectionString });
try {
await migrateSchema(executor);
await migratePluginSchemas(executor, [
await migrateSchema(executor, { mode: "schema-bootstrap" });
await bootstrapPluginSchemas(executor, [
{
dir: path.resolve(
workspaceRoot,
Expand Down
2 changes: 2 additions & 0 deletions packages/junior-memory/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,8 @@ exported types, tools, and tests are authoritative.
- `MEMORY_RECALL_MAX_VECTOR_DISTANCE` or
`recallMaxVectorDistance` configures the vector candidate threshold.
- Generate schema changes with `pnpm --filter @sentry/junior-memory db:generate`.
- Generate self-contained data migrations with
`pnpm --filter @sentry/junior-memory db:generate:data --name <name>`.

Follow `../../policies/data-redaction.md`, `../../policies/security.md`, and the
plugin contract in `../junior-plugin-api/README.md`.
2 changes: 2 additions & 0 deletions packages/junior-memory/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@
"scripts": {
"build": "tsup && tsc -p tsconfig.build.json --emitDeclarationOnly",
"db:generate": "pnpm exec drizzle-kit generate --config drizzle.config.ts",
"db:generate:data": "pnpm exec junior-migrations generate --config drizzle.config.ts --out migrations",
"lint": "oxlint --config ../junior/.oxlintrc.json --deny-warnings src tests tsup.config.ts",
"prepare": "pnpm run build",
"prepack": "pnpm run build",
Expand All @@ -39,6 +40,7 @@
"zod": "catalog:"
},
"devDependencies": {
"@sentry/junior-migrations": "workspace:*",
"@sentry/junior-testing": "workspace:*",
"@types/node": "^25.9.1",
"drizzle-kit": "catalog:",
Expand Down
54 changes: 54 additions & 0 deletions packages/junior-migrations/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
# `@sentry/junior-migrations`

Junior's migration format extends a Drizzle Kit journal with TypeScript data
migrations. Every journal entry resolves to exactly one `<tag>.sql` or
`<tag>.ts` file. Drizzle Kit continues to own numbering and schema snapshots;
the Junior migration runner owns execution and exact per-entry tracking.

TypeScript migrations target a versioned capability API and must not import
Junior runtime internals. Migration-specific parsers and transforms belong
permanently in the migration file so application refactors or deletions cannot
break pending upgrades.

## Authoring

Generate schema migrations with the owning package's normal Drizzle command.
Generate data migrations through this package's wrapper:

```bash
junior-migrations generate \
--config drizzle.config.ts \
--out migrations \
--name backfill_actor
```

The wrapper asks Drizzle Kit to create a custom journal entry, replaces the
empty SQL file with a `MigrationV1` TypeScript scaffold, and removes the
unchanged custom snapshot. Schema generation continues from the latest real
schema snapshot while preserving the mixed journal order.

## Compatibility

Drizzle Kit remains the supported authoring tool. Drizzle ORM's stock
`migrate()` function is not a supported executor for mixed journals because it
requires every entry to have a SQL file. Call `runMigrationJournal` instead and
provide the host database adapter and, for TypeScript entries, a context and
loader. The
same database adapter drives the journal ledger and is exposed as
`context.database`, so migration files never own connection or driver setup.

Normal upgrades run with `mode: "all"` and execute every SQL and TypeScript
entry in journal order. `mode: "schema-bootstrap"` is reserved for constructing
an empty database at the latest schema in tests or bootstrap tooling. It skips
TypeScript data migrations while executing SQL entries across the full journal,
so it must not be used to upgrade an existing installation.

The runner rejects runtime imports of application source, relative modules,
and unversioned `@sentry/junior` modules. Migrations may import the append-only
`@sentry/junior/migration-helpers/v1` surface for stable parsing primitives and
other reusable infrastructure. One-off migration decisions and data transforms
must still remain in the journal entry. Add a new helper or capability version
rather than changing an existing contract.

This source validation is an authoring guard for trusted packaged migration
code, not a security sandbox for untrusted scripts.
Loading
Loading