fix(v2): stop telling callers something the server did not do (#6676)

A works-as-advertised sweep of the v2 surface found one defect class in five
places: input is validated for shape, then its meaning is re-derived
independently by each consumer — so a filter compiles differently than it
validated, or a write commits and the response then reports failure.

Knowledge tag filters were validated once and re-parsed three times. The
document list read `Number()`, search read `parseFloat()`; the list matched
booleans case-insensitively, search compared against the literal `'true'`; the
list escaped LIKE metacharacters, search did not; and the date pattern was
tested against the untrimmed string the validator had already trimmed. Two of
those paths dropped the predicate entirely and answered 200 with the whole
knowledge base — on a billed endpoint. Values are now coerced once, where the
resolved field type is known, and both builders consume the result. A builder
that cannot compile an already-validated filter now raises instead of silently
widening the result set.

`PUT /api/v2/secrets/{name}` with `scope: personal` committed the secret and
then answered 500, because a user-global write was reported through a
workspace-scoped mirror lookup, and an org admin's inherited access has no
`permissions` row for the fan-out to find. The personal path no longer decides
success from a per-workspace mirror. The `workspaceId` descriptions said a
personal secret lives in one workspace; it does not, and they now say so.

A custom tool could be stored with a schema the read path cannot serialize —
`POST /workflows/import` and Copilot both wrote through name-only checks — so
one row made the whole workspace list 500, and a title-only PATCH committed,
audited, then reported failure. Every write now passes the same guard the
response schema is derived from.

`POST /api/v2/tables` accepted `workflowGroupId` on an initial column. Nothing
can populate it legitimately, and it made every later column-add and group-add
fail with no way to clear it. The key is refused at the boundary, and
`createTable` now runs the invariant every later mutation already runs, closing
the internal and v1 ingresses too. Those invariants moved to a leaf module:
reaching them through `workflow-columns` pulled the executable tool registry
into the tables page graph, taking it from 1,767 modules to 6,999.

An out-of-range upload part number answered 500 rather than the 400 its
published contract promises, because the throw happened above the route's
try/catch and was not an `HttpError`.

Author: waleedlatif1

apps/docs/openapi-v2-resources.json

@@ -1965,22 +1965,22 @@
             "name": "workspaceId",
             "in": "query",
             "required": true,
-            "description": "Workspace in which the secret is available.",
+            "description": "Workspace the request is authorized against. A workspace secret is deleted from it; a personal secret is deleted for the caller in all of their workspaces.",
             "schema": {
               "type": "string",
               "minLength": 1,
-              "description": "Workspace in which the secret is available."
+              "description": "Workspace the request is authorized against. A workspace secret is deleted from it; a personal secret is deleted for the caller in all of their workspaces."
             }
           },
           {
             "name": "scope",
             "in": "query",
             "required": true,
-            "description": "Whether the secret belongs to the workspace or the caller.",
+            "description": "Whether the secret belongs to the workspace or to the caller. A personal secret belongs to the caller across every workspace, not to one workspace.",
             "schema": {
               "type": "string",
               "enum": ["workspace", "personal"],
-              "description": "Whether the secret belongs to the workspace or the caller."
+              "description": "Whether the secret belongs to the workspace or to the caller. A personal secret belongs to the caller across every workspace, not to one workspace."
             }
           }
         ],
@@ -4088,7 +4088,7 @@
           "scope": {
             "type": "string",
             "enum": ["workspace", "personal"],
-            "description": "Whether the secret belongs to the workspace or the caller."
+            "description": "Whether the secret belongs to the workspace or to the caller. A personal secret belongs to the caller across every workspace, not to one workspace."
           },
           "role": {
             "type": "string",
@@ -4184,12 +4184,12 @@
           "workspaceId": {
             "type": "string",
             "minLength": 1,
-            "description": "Workspace in which the secret is available."
+            "description": "Workspace the request is authorized against. A workspace secret is written to it; a personal secret is written to the caller and is available in all of their workspaces."
           },
           "scope": {
             "type": "string",
             "enum": ["workspace", "personal"],
-            "description": "Whether the secret belongs to the workspace or the caller."
+            "description": "Whether the secret belongs to the workspace or to the caller. A personal secret belongs to the caller across every workspace, not to one workspace."
           },
           "value": {
             "type": "string",
@@ -4224,7 +4224,7 @@
           "scope": {
             "type": "string",
             "enum": ["workspace", "personal"],
-            "description": "Whether the secret belongs to the workspace or the caller."
+            "description": "Whether the secret belongs to the workspace or to the caller. A personal secret belongs to the caller across every workspace, not to one workspace."
           },
           "deleted": {
             "type": "boolean",

apps/docs/openapi-v2-tables.json

@@ -4370,10 +4370,6 @@
                       "description": "ISO 4217 code for currency columns.",
                       "type": "string",
                       "pattern": "^[A-Za-z]{3}$"
-                    },
-                    "workflowGroupId": {
-                      "description": "Workflow group initially associated with the column.",
-                      "type": "string"
                     }
                   },
                   "required": ["name", "type"],

apps/sim/app/api/v2/uploads/[uploadId]/parts/[partNumber]/route.test.ts

@@ -29,6 +29,7 @@ vi.mock('@/lib/uploads/upload-session/service', () => ({
   verifyUploadSessionToken: mockVerifyUploadSessionToken,
 }))
 
+import { OrchestrationError } from '@/lib/core/orchestration/types'
 import { PUT } from '@/app/api/v2/uploads/[uploadId]/parts/[partNumber]/route'
 
 const SESSION = {
@@ -37,6 +38,8 @@ const SESSION = {
   method: 'multipart',
   status: 'uploading',
   expiresAt: new Date('2999-01-01T00:00:00.000Z'),
+  partSize: 3,
+  partCount: 2,
 } as const
 
 describe('PUT /api/v2/uploads/[uploadId]/parts/[partNumber]', () => {
@@ -84,6 +87,41 @@ describe('PUT /api/v2/uploads/[uploadId]/parts/[partNumber]', () => {
     expect(mockWriteLocalMultipartPart).not.toHaveBeenCalled()
   })
 
+  /**
+   * The part number is a path segment of a session-scoped signed URL, so any
+   * holder of a legitimate part URL can address a part the session does not
+   * have. `expectedUploadPartSize` classifies that as a validation failure;
+   * `service.test.ts` pins that classification on the real implementation,
+   * which this suite mocks away.
+   */
+  it('maps an out-of-range part number to the documented 400', async () => {
+    mockExpectedUploadPartSize.mockImplementation(() => {
+      throw new OrchestrationError('validation', 'partNumber must be between 1 and 2')
+    })
+
+    const response = await request({ partNumber: '99' })
+
+    expect(response.status).toBe(400)
+    await expect(response.json()).resolves.toEqual({
+      error: { code: 'BAD_REQUEST', message: 'partNumber must be between 1 and 2' },
+    })
+    expect(mockWriteLocalMultipartPart).not.toHaveBeenCalled()
+  })
+
+  it('still renders an unclassified part-size failure as a generic 500', async () => {
+    mockExpectedUploadPartSize.mockImplementation(() => {
+      throw new Error('unexpected')
+    })
+
+    const response = await request()
+
+    expect(response.status).toBe(500)
+    await expect(response.json()).resolves.toEqual({
+      error: { code: 'INTERNAL_ERROR', message: 'Internal server error' },
+    })
+    expect(mockWriteLocalMultipartPart).not.toHaveBeenCalled()
+  })
+
   it('rejects expired upload sessions before writing the part', async () => {
     mockVerifyUploadSessionToken.mockReturnValue({
       ...SESSION,
@@ -101,17 +139,21 @@ describe('PUT /api/v2/uploads/[uploadId]/parts/[partNumber]', () => {
   })
 })
 
-function request(options?: { contentLength?: string | null }) {
+function request(options?: { contentLength?: string | null; partNumber?: string }) {
   const headers = new Headers({ 'Content-Type': 'application/octet-stream' })
   if (options?.contentLength !== null) {
     headers.set('Content-Length', options?.contentLength ?? '3')
   }
+  const partNumber = options?.partNumber ?? '1'
   return PUT(
-    new NextRequest('http://localhost:3000/api/v2/uploads/upload-1/parts/1?token=signed-token', {
-      method: 'PUT',
-      headers,
-      body: new Uint8Array([1, 2, 3]),
-    }),
-    { params: Promise.resolve({ uploadId: 'upload-1', partNumber: '1' }) }
+    new NextRequest(
+      `http://localhost:3000/api/v2/uploads/upload-1/parts/${partNumber}?token=signed-token`,
+      {
+        method: 'PUT',
+        headers,
+        body: new Uint8Array([1, 2, 3]),
+      }
+    ),
+    { params: Promise.resolve({ uploadId: 'upload-1', partNumber }) }
   )
 }

apps/sim/app/api/v2/uploads/[uploadId]/parts/[partNumber]/route.ts

@@ -12,7 +12,12 @@ import {
   type UploadSessionRecord,
   verifyUploadSessionToken,
 } from '@/lib/uploads/upload-session/service'
-import { v2Error, v2HttpError, v2UploadDataPlaneError } from '@/app/api/v2/lib/response'
+import {
+  v2CaughtOrchestrationError,
+  v2Error,
+  v2HttpError,
+  v2UploadDataPlaneError,
+} from '@/app/api/v2/lib/response'
 
 interface LocalPartRouteParams {
   params: Promise<{ uploadId: string; partNumber: string }>
@@ -60,7 +65,18 @@ export const PUT = withRouteHandler(
     }
 
     const { partNumber } = parsed.data.params
-    const expectedSize = expectedUploadPartSize(session, partNumber)
+    let expectedSize: number
+    try {
+      expectedSize = expectedUploadPartSize(session, partNumber)
+    } catch (error) {
+      // The part number is a path segment of a session-scoped signed URL, so a
+      // caller can address a part this session does not have. That refusal is a
+      // classified domain failure, and the data plane's generic 500 tail would
+      // otherwise render it as an internal error.
+      const classified = v2CaughtOrchestrationError(error)
+      if (classified) return classified
+      throw error
+    }
     const contentLength = request.headers.get('content-length')
     if (contentLength !== null && Number(contentLength) !== expectedSize) {
       return v2Error('BAD_REQUEST', `Part ${partNumber} must contain exactly ${expectedSize} bytes`)

apps/sim/lib/api/contracts/v2/__tests__/tables.test.ts

@@ -59,6 +59,27 @@ describe('v2 table column contracts', () => {
     ).toMatchObject({ success: true, data: { updates: { required: true } } })
   })
 
+  /**
+   * v2 mints workflow group ids server-side and has no way to declare a group
+   * on the create body, so any id a caller supplied would name a group that
+   * does not exist. `createTable` does not check that, but every later schema
+   * mutation does — accepting the field made the created table's columns and
+   * groups permanently unaddable, with nothing on the update body able to clear
+   * it.
+   */
+  it('refuses a workflow group id on an initial column', () => {
+    const result = v2CreateTableBodySchema.safeParse({
+      workspaceId: WORKSPACE_ID,
+      name: 'contacts',
+      schema: {
+        columns: [{ name: 'email', type: 'string', workflowGroupId: 'wfg_does_not_exist' }],
+      },
+    })
+
+    expect(result.success).toBe(false)
+    expect(issueCodes(result.error?.issues ?? [])).toContain('unrecognized_keys')
+  })
+
   it('keeps required in table responses for existing stored schemas', () => {
     expect(
       v2ApiTableSchema.safeParse({

apps/sim/lib/api/contracts/v2/secrets.ts

@@ -15,7 +15,9 @@ const SECRET_NAME_REGEX = /^[A-Za-z0-9_]+$/
 
 export const v2SecretScopeSchema = z
   .enum(['workspace', 'personal'])
-  .describe('Whether the secret belongs to the workspace or the caller.')
+  .describe(
+    'Whether the secret belongs to the workspace or to the caller. A personal secret belongs to the caller across every workspace, not to one workspace.'
+  )
 export type V2SecretScope = z.output<typeof v2SecretScopeSchema>
 
 export const v2SecretNameSchema = z
@@ -76,7 +78,9 @@ export type V2SecretParams = z.output<typeof v2SecretParamsSchema>
 
 export const v2SetSecretBodySchema = z
   .object({
-    workspaceId: workspaceIdSchema.describe('Workspace in which the secret is available.'),
+    workspaceId: workspaceIdSchema.describe(
+      'Workspace the request is authorized against. A workspace secret is written to it; a personal secret is written to the caller and is available in all of their workspaces.'
+    ),
     scope: v2SecretScopeSchema,
     value: z
       .string()
@@ -90,7 +94,9 @@ export type V2SetSecretBody = z.input<typeof v2SetSecretBodySchema>
 
 export const v2DeleteSecretQuerySchema = z
   .object({
-    workspaceId: workspaceIdSchema.describe('Workspace in which the secret is available.'),
+    workspaceId: workspaceIdSchema.describe(
+      'Workspace the request is authorized against. A workspace secret is deleted from it; a personal secret is deleted for the caller in all of their workspaces.'
+    ),
     scope: v2SecretScopeSchema,
   })
   .strict()

apps/sim/lib/api/contracts/v2/tables.ts

@@ -390,24 +390,23 @@ export const v2TableColumnInputSchema = z
   .strict()
   .superRefine(refineColumnOptions)
 
-const v2InitialTableColumnInputSchema = z
-  .object({
-    ...v2TableColumnInputShape,
-    workflowGroupId: z
-      .string()
-      .optional()
-      .describe('Workflow group initially associated with the column.'),
-  })
-  .strict()
-  .superRefine(refineColumnOptions)
-
+/**
+ * Initial columns take the same shape as every other v2 column input.
+ *
+ * They deliberately cannot name a workflow group: v2 has no way to declare one
+ * on this body and mints group ids server-side, so any id a caller supplied
+ * would necessarily dangle. A dangling `workflowGroupId` is a schema invariant
+ * violation, and `createTable` does not check it while every later schema
+ * mutation does — so accepting the field made the table's own columns and
+ * groups permanently unaddable, with no update body field able to clear it.
+ */
 export const v2CreateTableBodySchema = v1CreateTableBodySchema
   .omit({ folderId: true, schema: true })
   .extend({
     schema: z
       .object({
         columns: z
-          .array(v2InitialTableColumnInputSchema)
+          .array(v2TableColumnInputSchema)
           .min(1, 'Table must have at least one column')
           .max(
             TABLE_LIMITS.MAX_COLUMNS_PER_TABLE,

apps/sim/lib/credentials/environment.ts

@@ -3,7 +3,7 @@ import { credential, credentialMember, permissions, workspace } from '@sim/db/sc
 import { permissionSatisfies } from '@sim/platform-authz/workspace'
 import { chunkArray } from '@sim/utils/helpers'
 import { generateId } from '@sim/utils/id'
-import { and, eq, inArray, isNotNull, isNull, notInArray, or, sql } from 'drizzle-orm'
+import { and, asc, eq, inArray, isNotNull, isNull, notInArray, or, sql } from 'drizzle-orm'
 import { acquireUserBillingIdentityLock } from '@/lib/billing/organizations/billing-identity-lock'
 import type { DbOrTx } from '@/lib/db/types'
 import {
@@ -552,7 +552,53 @@ export async function upsertPersonalEnvCredentialForUser(params: {
   await db.transaction(upsert)
 }
 
-/** Deletes one caller-owned personal secret's credential metadata in every workspace. */
+export interface PersonalEnvCredentialMetadata {
+  id: string
+  createdAt: Date
+  updatedAt: Date
+}
+
+/**
+ * Reads one caller-owned personal secret's credential metadata without scoping to
+ * a workspace.
+ *
+ * A personal secret is stored once per user; the `env_personal` credential rows
+ * are per-workspace mirrors, so a reader that needs the secret's own timestamps
+ * must not require a mirror in one particular workspace. The earliest mirror is
+ * the authoritative creation time — later ones are written when the caller joins
+ * another workspace, long after the secret itself was created.
+ */
+export async function getPersonalEnvCredentialMetadata(params: {
+  userId: string
+  envKey: string
+}): Promise<PersonalEnvCredentialMetadata | null> {
+  const [row] = await db
+    .select({
+      id: credential.id,
+      createdAt: credential.createdAt,
+      updatedAt: credential.updatedAt,
+    })
+    .from(credential)
+    .where(
+      and(
+        eq(credential.type, 'env_personal'),
+        eq(credential.envOwnerUserId, params.userId),
+        eq(credential.envKey, params.envKey)
+      )
+    )
+    .orderBy(asc(credential.createdAt))
+    .limit(1)
+
+  return row ?? null
+}
+
+/**
+ * Deletes one caller-owned personal secret's credential metadata in every workspace.
+ *
+ * Deliberately unscoped by workspace: the value being removed alongside it lives
+ * in the user-global `environment` row, so leaving mirrors behind in other
+ * workspaces would advertise a secret that no longer exists.
+ */
 export async function deletePersonalEnvCredentialForUser(params: {
   userId: string
   envKey: string