Skip to content

Commit 72c99ff

Browse files
committed
Merge remote-tracking branch 'origin/staging' into workflow-updates-v2
2 parents 7934df7 + 258a37c commit 72c99ff

335 files changed

Lines changed: 12451 additions & 2520 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.claude/rules/sim-url-state.md

Lines changed: 18 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -34,7 +34,7 @@ Put state in the URL **only** when it is *all* of: shareable, deep-linkable, boo
3434
## Anti-patterns (forbidden)
3535

3636
- Direct `useSearchParams().get(...)` or `new URLSearchParams(window.location.search)` to **read** state.
37-
- Hand-built query strings + `router.replace`/`router.push` to **mutate** state.
37+
- Hand-built query strings + `router.replace`/`router.push` to **mutate** state. **If the target path equals the current path, it is a query mutation, not a navigation** — even when written as a full path template. Re-serializing the path by hand is lossy by construction: it drops every param the template forgets. Use the nuqs setter (`setParams({ key: null }, { history: 'replace', scroll: false })`) — `null` always removes the key, and only the params you name are touched. Both options are already nuqs defaults (see "Conventions"); write them explicitly because a group whose shared options set `history: 'push'` (e.g. `filesUrlKeys`) would otherwise push a back-stack entry for a strip.
3838
- `window.history.replaceState`/`pushState` to mutate a param.
3939
- Duplicating URL state into a store and syncing it with effects / `popstate` listeners.
4040
- High-frequency or large state in the URL (cursor, pan/zoom, un-debounced keystrokes, big JSON blobs).
@@ -44,7 +44,7 @@ These reads/mutations are **not** anti-patterns and stay as-is:
4444

4545
- **Outbound URL builders**`new URLSearchParams({...})` to construct a `href`, a download endpoint, an external WebSocket/API URL, or a `window.open(_, '_blank')` destination.
4646
- **Route navigations**`router.push('/path/[id]?folderId=x')` that changes the route *path*, not just the current query. A nuqs setter only mutates the query on the current path; cross-path navigation stays on `router`.
47-
- **Read-once auth / redirect signals**`token`, `callbackUrl`, `redirect`, `error`, `invite_flow`, `upgraded`, `redirect_workflow`, etc. These are navigation signals consumed once (often read-then-strip), not synced view-state. Leave them on `useSearchParams`.
47+
- **Read-once auth / redirect signals**`token`, `callbackUrl`, `redirect`, `error`, `invite_flow`, `new` (invite signup flow), `upgraded`, `redirect_workflow`, etc. These are navigation signals consumed once (often read-then-strip), not synced view-state. Leave them on `useSearchParams`. Key names are per-surface: files' `new` is a genuine nuqs param (`files/search-params.ts`), while invite's `new` is a one-shot signup signal.
4848

4949
## Per-feature `search-params.ts` — single source of truth
5050

@@ -128,7 +128,22 @@ If a client param must be re-read server-side after a change, set `shallow: fals
128128

129129
## Suspense boundary
130130

131-
`useQueryState`/`useQueryStates` read `useSearchParams` internally, so any client component using them must sit under a `<Suspense>` boundary (Next.js requirement). Wrap the page entry with a real-chrome fallback so a suspend never flashes a blank frame — see `apps/sim/app/workspace/[workspaceId]/files/page.tsx`.
131+
`useQueryState`/`useQueryStates` read `useSearchParams` internally, so any client component using them must sit under a `<Suspense>` boundary (Next.js requirement). Wrap the page entry with a real-chrome fallback so a suspend never flashes a blank frame.
132+
133+
**Never `fallback={null}` on a page entry.** The route's co-located `loading.tsx` default export *is* the correct fallback — one skeleton serves both the route-level navigation transition (which Next renders automatically) and the in-page suspend (which this boundary renders). If the segment has no `loading.tsx`, add one; the route transition needs it anyway. Import it absolutely (`sim-imports.md`):
134+
135+
```typescript
136+
import { KnowledgeBase } from '@/app/workspace/[workspaceId]/knowledge/[id]/base'
137+
import KnowledgeBaseLoading from '@/app/workspace/[workspaceId]/knowledge/[id]/loading'
138+
139+
<Suspense fallback={<KnowledgeBaseLoading />}>
140+
<KnowledgeBase id={id} knowledgeBaseName={kbName || 'Knowledge Base'} />
141+
</Suspense>
142+
```
143+
144+
Reference: `apps/sim/app/workspace/[workspaceId]/knowledge/[id]/page.tsx`.
145+
146+
This applies to **page entries**. An inner `<Suspense>` wrapping a `lazy()` component is the exception: there `fallback={null}` is correct, precisely so the suspend resolves at the nearest boundary instead of flashing the whole route — see `sim-imports.md`, "Code-splitting through barrels".
132147

133148
## Debounced text inputs
134149

.devcontainer/docker-compose.yml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,7 @@ services:
1919
- BETTER_AUTH_SECRET=${BETTER_AUTH_SECRET:-your_auth_secret_here}
2020
- ENCRYPTION_KEY=${ENCRYPTION_KEY:-your_encryption_key_here}
2121
- COPILOT_API_KEY=${COPILOT_API_KEY}
22+
- MSHIP_SYSPROMPT_OVERRIDE=${MSHIP_SYSPROMPT_OVERRIDE:-}
2223
- NEXT_PUBLIC_CHAT_DISABLED=${NEXT_PUBLIC_CHAT_DISABLED:-}
2324
- SIM_AGENT_API_URL=${SIM_AGENT_API_URL}
2425
- OLLAMA_URL=${OLLAMA_URL:-http://localhost:11434}

.github/workflows/ci.yml

Lines changed: 12 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -155,15 +155,20 @@ jobs:
155155
fail-fast: false
156156
matrix:
157157
include:
158-
# Only the app image needs the paid 8-core/32 GB runner: next build
159-
# exhausts the free 16 GB one (exit 137). The others build in <5 min.
160-
# bs_runner mirrors that per-image sizing on Blacksmith — a single
161-
# pinned tier put every image on 8 vCPU, where the non-app builds idle
162-
# at 12-15% CPU and under 10% memory.
158+
# Only the app image needs a large runner: next build exhausts the free
159+
# 16 GB one (exit 137). The others build in <5 min and idle at 12-15%
160+
# CPU on 8 vCPU, so they stay on the smaller tiers.
161+
#
162+
# 16 vCPU on Blacksmith because this build is the critical path to a
163+
# deploy — nothing ships until the image is pushed — and its two
164+
# dominant steps both scale with cores (`bun install` ~300-400s, `next
165+
# build` ~260s). The same `next build` runs on 16 vCPU in the separate
166+
# Build App verification job, which does not gate anything; this one
167+
# was doing comparable work on half the cores.
163168
- dockerfile: ./docker/app.Dockerfile
164169
ecr_repo_secret: ECR_APP
165170
gh_runner: linux-x64-8-core
166-
bs_runner: blacksmith-8vcpu-ubuntu-2404
171+
bs_runner: blacksmith-16vcpu-ubuntu-2404
167172
- dockerfile: ./docker/db.Dockerfile
168173
ecr_repo_secret: ECR_MIGRATIONS
169174
gh_runner: ubuntu-latest
@@ -278,7 +283,7 @@ jobs:
278283
ghcr_image: ghcr.io/simstudioai/simstudio
279284
ecr_repo_secret: ECR_APP
280285
gh_runner: linux-x64-8-core
281-
bs_runner: blacksmith-8vcpu-ubuntu-2404
286+
bs_runner: blacksmith-16vcpu-ubuntu-2404
282287
- dockerfile: ./docker/db.Dockerfile
283288
ghcr_image: ghcr.io/simstudioai/migrations
284289
ecr_repo_secret: ECR_MIGRATIONS

apps/docs/content/docs/en/integrations/dynatrace.mdx

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -123,7 +123,7 @@ Get the full details of a single Dynatrace problem, including root cause, affect
123123
| `environmentUrl` | string | Yes | Dynatrace environment URL \(e.g., https://abc12345.live.dynatrace.com, or https://your-activegate:9999/e/abc12345 for Managed\) |
124124
| `apiToken` | string | Yes | Dynatrace access token \(dt0c01...\) with the problems.read scope |
125125
| `problemId` | string | Yes | ID of the problem \(e.g., -1234567890123456789_1700000000000V2\) |
126-
| `fields` | string | No | Comma-separated optional properties to include: evidenceDetails, impactAnalysis, recentComments |
126+
| `fields` | string | No | Comma-separated optional properties to include. Defaults to all of them: evidenceDetails, impactAnalysis, recentComments |
127127

128128
#### Output
129129

@@ -581,7 +581,7 @@ Get a single vulnerability with its description, remediation guidance, affected
581581
| `environmentUrl` | string | Yes | Dynatrace environment URL \(e.g., https://abc12345.live.dynatrace.com, or https://your-activegate:9999/e/abc12345 for Managed\) |
582582
| `apiToken` | string | Yes | Dynatrace access token \(dt0c01...\) with the securityProblems.read scope |
583583
| `securityProblemId` | string | Yes | ID of the security problem |
584-
| `fields` | string | No | Comma-separated optional properties to include: +riskAssessment, +managementZones, +codeLevelVulnerabilityDetails, +globalCounts |
584+
| `fields` | string | No | Comma-separated optional properties to include, each prefixed with +. Defaults to every detail property: +riskAssessment, +managementZones, +codeLevelVulnerabilityDetails, +globalCounts, +filteredCounts, +description, +remediationDescription, +events, +vulnerableComponents, +affectedEntities, +exposedEntities, +reachableDataAssets, +relatedEntities, +relatedContainerImages, +relatedAttacks, +entryPoints |
585585
| `managementZoneFilter` | string | No | Restrict the counts to management zones, e.g. names\("Production"\) |
586586
| `from` | string | No | Start of the timeframe as UTC milliseconds, ISO 8601, or a relative expression such as now-24h. Defaults to the last 24 hours |
587587

@@ -772,7 +772,7 @@ Get a single attack with its entry point, payload, attacker, and the vulnerabili
772772
| `environmentUrl` | string | Yes | Dynatrace environment URL \(e.g., https://abc12345.live.dynatrace.com, or https://your-activegate:9999/e/abc12345 for Managed\) |
773773
| `apiToken` | string | Yes | Dynatrace access token \(dt0c01...\) with the attacks.read scope |
774774
| `attackId` | string | Yes | ID of the attack |
775-
| `fields` | string | No | Comma-separated optional properties to include: +attackTarget, +request, +entrypoint, +vulnerability, +securityProblem, +attacker, +managementZones |
775+
| `fields` | string | No | Comma-separated optional properties to include, each prefixed with +. Defaults to all of them: +attackTarget, +request, +entrypoint, +vulnerability, +securityProblem, +attacker, +managementZones |
776776

777777
#### Output
778778

apps/docs/content/docs/en/integrations/embeddings.mdx

Lines changed: 25 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -25,7 +25,7 @@ Sim's knowledge bases embed separately, at a fixed vector width and from a small
2525

2626
## Usage Instructions
2727

28-
Turn text into embedding vectors for semantic search, clustering, and similarity. Supports OpenAI, Google Gemini, Cohere, and Mistral embedding models.
28+
Turn text into embedding vectors for semantic search, clustering, and similarity. Supports OpenAI, OpenRouter, Google Gemini, Cohere, and Mistral embedding models.
2929

3030

3131

@@ -55,6 +55,30 @@ Generate embeddings from text using OpenAI's embedding models
5555
| `dimensions` | number | Dimensionality of each vector |
5656
| `usage` | json | Token usage |
5757

58+
### OpenRouter Embeddings
59+
60+
Generate embeddings through OpenRouter
61+
62+
#### Input
63+
64+
| Parameter | Type | Required | Description |
65+
| --------- | ---- | -------- | ----------- |
66+
| `input` | string | Yes | Text to embed, or an array of texts to embed in one call |
67+
| `model` | string | No | Embedding model to use |
68+
| `taskType` | string | No | What the embedding is for, when the model supports task conditioning: document, query, similarity, classification, or clustering |
69+
| `dimensions` | number | No | Output dimensions, when the model supports truncation. Defaults to native. |
70+
| `apiKey` | string | Yes | API key for the selected embedding provider |
71+
72+
#### Output
73+
74+
| Parameter | Type | Description |
75+
| --------- | ---- | ----------- |
76+
| `embeddings` | json | Generated embeddings |
77+
| `model` | string | Model used |
78+
| `provider` | string | Provider used |
79+
| `dimensions` | number | Dimensionality of each vector |
80+
| `usage` | json | Token usage |
81+
5882
### Gemini Embeddings
5983

6084
Generate embeddings from text using Google's Gemini embedding models

apps/docs/content/docs/en/integrations/meta.json

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -224,6 +224,7 @@
224224
"smartlead",
225225
"smtp",
226226
"snowflake",
227+
"snowflake-service-account",
227228
"sportmonks",
228229
"sqs",
229230
"square",
Lines changed: 120 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,120 @@
1+
---
2+
title: Snowflake Programmatic Access Tokens
3+
description: Create a Snowflake programmatic access token and connect it to Sim so workflows can query your account
4+
---
5+
6+
import { Callout } from 'fumadocs-ui/components/callout'
7+
import { Step, Steps } from 'fumadocs-ui/components/steps'
8+
import { FAQ } from '@/components/ui/faq'
9+
10+
A Snowflake programmatic access token (PAT) lets a workflow authenticate to your account over the Snowflake SQL API without a password or a key pair. The token belongs to one Snowflake user. Left unrestricted it can act as any role that user holds; with `ROLE_RESTRICTION` set it is pinned to exactly one.
11+
12+
Sim stores the token alongside your account host as one credential. Once it is added, every Snowflake block picks it from a dropdown — and the block's database, schema, table, warehouse, role, file-format, and procedure fields become pickers that list what the token can actually see.
13+
14+
## Prerequisites
15+
16+
- A Snowflake user you can generate a token for. Generating a token for another user requires the ability to run `ALTER USER` on them.
17+
- Your account host — the `<account_identifier>.snowflakecomputing.com` hostname, for example `myorg-myaccount.snowflakecomputing.com`. Snowsight shows it under **Account details**.
18+
- A network policy covering the user, or an authentication policy that waives the requirement (see below).
19+
20+
<Callout type="warn">
21+
Snowflake's **network policy** requirement varies by user type, and getting it wrong is the most common reason a token is rejected:
22+
23+
- `TYPE = PERSON` — you can generate a token without a network policy, but the user **must** be covered by one to authenticate with it.
24+
- `TYPE = SERVICE` and `TYPE = LEGACY_SERVICE` — a network policy is required to generate **and** to use a token.
25+
- `TYPE = SERVICE_AGENT` — exempt; generate and use freely.
26+
27+
If your account has no network policy, either create one (allowing Sim's egress) or set `NETWORK_POLICY_EVALUATION = ENFORCED_NOT_REQUIRED` on an authentication policy applied to the user.
28+
</Callout>
29+
30+
## Creating the Token
31+
32+
### Option 1 — Snowsight
33+
34+
<Steps>
35+
<Step>
36+
Open **Governance & security****Users & roles** and select the user the workflow should run as
37+
</Step>
38+
<Step>
39+
Under **Programmatic access tokens**, click **Generate new token**
40+
</Step>
41+
<Step>
42+
Give it a name, optionally restrict it to a single role, and set the expiry in days
43+
</Step>
44+
<Step>
45+
Copy the token secret. Snowflake shows it **once**, at creation
46+
</Step>
47+
</Steps>
48+
49+
### Option 2 — SQL
50+
51+
```sql
52+
ALTER USER my_service_user ADD PROGRAMMATIC ACCESS TOKEN sim_workflows
53+
ROLE_RESTRICTION = 'SIM_WORKFLOW_ROLE'
54+
DAYS_TO_EXPIRY = 90;
55+
```
56+
57+
`DAYS_TO_EXPIRY` defaults to 15 days and cannot exceed 365 — an authentication policy can lower that ceiling further via `PROGRAMMATIC_ACCESS_TOKEN_MAX_EXPIRY_IN_DAYS`. **A token can never be non-expiring**, and the value cannot be changed after creation — to extend it, generate a new token and swap the credential in Sim. Plan the rotation when you create it.
58+
59+
Service users (`TYPE = SERVICE`, `LEGACY_SERVICE`, or `SERVICE_AGENT`) **must** set `ROLE_RESTRICTION`, unless an authentication policy exempts them. For person users it is optional but recommended: a restricted token can only ever act as that one role.
60+
61+
<Callout type="info">
62+
If an authentication policy applies to the user, `'PROGRAMMATIC_ACCESS_TOKEN'` must appear in its `AUTHENTICATION_METHODS` list, otherwise the token is refused.
63+
</Callout>
64+
65+
## Adding the Credential to Sim
66+
67+
<Steps>
68+
<Step>
69+
Add a **Snowflake** block to a workflow, open the credential dropdown, and choose to add a programmatic access token
70+
</Step>
71+
<Step>
72+
Enter the **account host** (`myorg-myaccount.snowflakecomputing.com`) and paste the **token**
73+
</Step>
74+
<Step>
75+
Save. Sim verifies the credential by running `SELECT CURRENT_USER(), CURRENT_ACCOUNT(), CURRENT_ROLE()` over the SQL API — a metadata-only statement that needs no warehouse and consumes no credits. A rejected token, an unreachable host, or a blocking network policy each produce a specific error rather than a generic failure.
76+
</Step>
77+
</Steps>
78+
79+
The host and the token are encrypted before being stored, and the token is never returned to the browser — the block sends a credential id and Sim resolves it server-side.
80+
81+
## Using the Credential in Workflows
82+
83+
Select the credential on any Snowflake block. You never enter the host again: every tool derives its endpoint from the host stored on the credential.
84+
85+
With a credential selected, these fields become pickers backed by metadata-only statements:
86+
87+
| Field | Lists | Needs |
88+
| --- | --- | --- |
89+
| Database | `SHOW DATABASES` | credential |
90+
| Schema | `SHOW SCHEMAS IN DATABASE` | database |
91+
| Table | `SHOW TABLES IN SCHEMA` | database, schema |
92+
| Warehouse | `SHOW WAREHOUSES` | credential |
93+
| Execution role | `CURRENT_AVAILABLE_ROLES()` | credential |
94+
| Named file format | `SHOW FILE FORMATS IN SCHEMA` | database, schema |
95+
| Procedure | `SHOW PROCEDURES IN SCHEMA` | database, schema |
96+
97+
Each picker runs as the token's user under its **default** role — not the execution role set on the block — so an empty list is usually a privilege gap rather than an empty account. Switch any field to advanced mode to type a name directly or reference an upstream block's output instead.
98+
99+
<Callout type="info">
100+
**Unload Data exports a table, not a query.** The COPY INTO grammar places the
101+
source immediately before its options, so an inline query would sit one
102+
parenthesis away from being able to rewrite them. To export a query result,
103+
materialize it first — a view, or `CREATE TABLE AS SELECT` via Execute SQL —
104+
then unload that object.
105+
</Callout>
106+
107+
## Rotating and Revoking
108+
109+
A token's expiry is fixed at creation. To rotate, generate a new token on the same user and update the credential in Sim — the old one stays valid until you remove it. `ALTER USER ... REMOVE PROGRAMMATIC ACCESS TOKEN <name>` revokes immediately and cannot be undone.
110+
111+
<FAQ items={[
112+
{ question: "Why a programmatic access token instead of a password?", answer: "The token is scoped to one user, can be restricted to a single role, expires on a schedule you choose, and can be revoked on its own without changing anyone's password or breaking other integrations." },
113+
{ question: "Does the token expire?", answer: "Yes. DAYS_TO_EXPIRY defaults to 15 days and can be set up to 365 at creation. It cannot be changed afterwards, so pick the value you want up front and plan a rotation." },
114+
{ question: "I lost the token — can I see it again?", answer: "No. Snowflake shows the secret only at creation. Generate a new token and update the credential in Sim." },
115+
{ question: "Why does adding the credential fail with an authentication error?", answer: "The three common causes are a token that has expired or been revoked, a user with no network policy (required to authenticate for every type except SERVICE_AGENT, unless an authentication policy waives it), and an authentication policy that omits PROGRAMMATIC_ACCESS_TOKEN from its AUTHENTICATION_METHODS. A wrong account host is reported separately — Snowflake resolves any *.snowflakecomputing.com name, so Sim identifies a mistyped host by the 404 it answers with." },
116+
{ question: "Why is a picker empty?", answer: "The pickers run SHOW statements as the token's user under its default role — the block's execution role is not applied to them. If the objects you expect are visible only to another role, grant the default role usage on them, restrict the token to the role that has access, or type the name in advanced mode." },
117+
{ question: "Does listing objects cost credits?", answer: "No. Every picker and the credential check run metadata-only statements, which Snowflake serves without a running warehouse." },
118+
{ question: "Can one credential reach two Snowflake accounts?", answer: "No. A token is bound to the user in one account, and the credential stores that account's host. Add one credential per account." },
119+
{ question: "How many tokens can a user have?", answer: "Snowflake allows up to 15 active programmatic access tokens per user." },
120+
]} />

0 commit comments

Comments
 (0)