feat: shared backend on Cloud Run with a workshop password by pietroid · Pull Request #1 · VeryGoodOpenSource/copilotkit_flutter_workshop

pietroid · 2026-10-23T20:11:26Z

Shared backend: runtime/ deploys to Cloud Run (deploy_runtime workflow, Workload Identity, no key file); PRs typecheck it, test the password gate and build the image (runtime_ci). The key and the password live in Secret Manager, and a manual warm_runtime workflow keeps the service awake before an event.
Password: the app sends X-Workshop-Code and a random X-Workshop-Session on every run (--dart-define=WORKSHOP_CODE); the runtime checks the code only when WORKSHOP_CODE is set, so local runtimes are unchanged. /info and the CORS preflight stay open. The gate (runtime/gate.ts) logs each run with its session id and caps runs per session and in total (MAX_RUNS_PER_SESSION, MAX_RUNS_TOTAL).
tool/setup.mjs and tool/doctor.mjs check the password up front with an empty run, and only ask for it when the backend needs one; tool/workshop.config.json points at the shared backend.
The model key is ANTHROPIC_API_KEY everywhere.
One-time Google Cloud setup: runtime/deploy/setup_gcp.sh (secrets, deployer role, public access). Run it again after the first deploy.

Before merging, add two repo secrets: GCP_WORKLOAD_IDENTITY_PROVIDER and GCP_SERVICE_ACCOUNT. The old CLAUDE_API_KEY, ANTHROPIC_API_KEY and WORKSHOP_CODE repo secrets are not used.

Tested: the deployed service (401 without the code, streamed model runs, doctor.mjs), the gate tests, and step 02 in Chrome. Secret Manager and the deployer role are set up in the project, but the workflows themselves have not run yet, and neither have the Flutter tests.

- runtime/Dockerfile and ignore files: the image holds only server.ts and env.ts, and a source deploy uploads nothing else (no .env) - deploy_runtime workflow: builds the image, deploys to Cloud Run through Workload Identity and checks that the password gate is on - runtime_ci workflow: typechecks the runtime and builds the image on pull requests - runtime/deploy: one-time Google Cloud setup script and the steps - point tool/workshop.config.json at the deployed backend

- the runtime, the step 03 server, setup and doctor read CLAUDE_API_KEY and hand it to the Anthropic SDK as ANTHROPIC_API_KEY - the deploy workflow passes the CLAUDE_API_KEY repository secret to Cloud Run - update the examples, the README and the deploy steps

Statyk7

Thanks for this Pietro, the design is solid: Workload Identity with no key file and a repo + main condition, a runtime service account with no roles, .env kept out of the image, a smoke test that asserts the 401, a constant-time comparison, and local runtimes unchanged.

Tested on this branch: runtime typecheck passes; Flutter analyze and app tests pass in steps 01 to 03; password gate on a local copy (/info 200 without a code, run without or with a wrong code 401, right code passes, stop without a code 401, preflight 204); runtime_ci passes; the deployed /info answers 200 with CORS open.
Not tested: the deploy workflow itself (needs the secrets and only runs on main), Safari or Firefox, Windows.

Before merging (details inline):

setup.mjs can print the password in clear.
Cold start versus the doctor's 8 second timeout.
The CLAUDE_API_KEY rename breaks existing ANTHROPIC_API_KEY setups.

Suggestions: spend caps and using the session header, deployer permissions and pinned actions, tests for the gate, checking the password in setup, and the README's PowerShell command. The PR description also still mentions runtime/deploy/README.md, which the last commit removed.

Happy to send a follow-up with these if that helps.

Statyk7 · 2026-10-24T03:00:08Z

+const appArgs = ["flutter", "run", "-d", "chrome", ...(backend ? [`--dart-define=RUNTIME_URL=${backend.url}`, `--dart-define=AGENT_ID=${backend.agent}`, ...(workshopCode ? [`--dart-define=WORKSHOP_CODE=${workshopCode}`] : [])] : [])];
 const appCommand = `fvm ${appArgs.join(" ")}`;
+/** The same command for the screen: the password is never printed. */
+const shownCommand = workshopCode ? appCommand.replace(workshopCode, "<workshop password>") : appCommand;


Bug: this can print the password. String.replace masks the first occurrence in the whole command. If the password also appears earlier (fault is inside default, run is in flutter run), the wrong spot gets masked and the real value stays in the WORKSHOP_CODE= argument. Reproduced with the same logic and the password fault:

... --dart-define=AGENT_ID=de<workshop password> --dart-define=WORKSHOP_CODE=fault

Suggestion: mask by argument instead of by value, e.g. appArgs.map((arg) => arg.startsWith('--dart-define=WORKSHOP_CODE=') ? '--dart-define=WORKSHOP_CODE=<workshop password>' : arg).

Statyk7 · 2026-10-24T03:00:08Z

  if (kind === "shared") {
    if (!config.sharedBackendUrl) fail("The shared backend is not configured yet.", "Use --backend local, or --backend custom --url <url>.");
    await useExternal(config.sharedBackendUrl, config.sharedAgentId ?? "default");
+    workshopCode = option("code") ?? process.env.WORKSHOP_CODE ?? (await askSecret("Workshop password (nothing is shown as you type, empty if there is none):")) ?? "";


The password is never checked, so a typo only shows up later as "The copilot could not answer". An empty POST .../agent/<id>/run is refused with 401 before it reaches the model, and with the right code it returns 400 (tried on a local copy), so setup (and doctor.mjs) can verify the code up front. The same probe without a code tells you whether the backend needs a password at all, so a custom backend that needs none isn't asked.

Statyk7 · 2026-10-24T03:00:08Z

 const keyVariables: Record<string, string | null> = {
  openai: "OPENAI_API_KEY",
-  anthropic: "ANTHROPIC_API_KEY",
+  anthropic: "CLAUDE_API_KEY",


This breaks existing setups. After this change anyone with ANTHROPIC_API_KEY set, and every existing runtime/.env, gets "CLAUDE_API_KEY is not set". Suggestion: for the anthropic provider accept CLAUDE_API_KEY ?? ANTHROPIC_API_KEY (same in the 03 server, doctor.mjs and setup.mjs), and say in the README why the new name exists. My guess is it keeps tools like Claude Code from picking the key up from the shell, but the PR doesn't say.

I moved this to Secret manager and replaced the name by the ANTHROPIC_API_KEY one

Statyk7 · 2026-10-24T03:00:08Z

+
+// The shared deployment sets WORKSHOP_CODE; a runtime on your machine leaves it
+// out and asks for nothing. The app sends the code as X-Workshop-Code (and a
+// random id of its run as X-Workshop-Session, which this server ignores). `/info`


X-Workshop-Session is sent by every client but ignored here. Two suggestions:

Use it. A per-session cap on runs, plus a log line per run with the session id, means one leaked password can't burn the whole budget and you can see the sessions in the Cloud Run logs.

Nothing limits spend today: --max-instances=1 limits concurrency, not cost. A global cap and a spend limit on the Anthropic key would, and changing WORKSHOP_CODE after the event.

If you'd rather not, drop the header.

Session is meant to separate the state of each user, I was not thinking on implementing spend limits for session necessarily, but it is a good call for the shared backend, I added some limits there

Statyk7 · 2026-10-24T03:00:08Z

+// without the code.
+const workshopCode = process.env.WORKSHOP_CODE;
+
+function guarded(req: IncomingMessage, res: ServerResponse) {


This is the security-relevant part of the PR and has no tests. Suggestion: move it into its own module and cover it with node:test: /info and the preflight open, 401 for a missing or wrong code, pass with the right code, and stop requests also need the code. Then run npm test in runtime_ci.

By hand on a local copy it behaves as described: /info 200 without a code, run without or with a wrong code 401, right code passes, stop without a code 401, preflight 204.

Statyk7 · 2026-10-24T03:00:08Z

+          flags: >-
+            --service-account=${{ env.RUNTIME_SERVICE_ACCOUNT }}
+            --allow-unauthenticated
+            --max-instances=1


Cold start. With min instances at 0, the first request after a quiet spell waits for a cold start (tsx plus the runtime). Here the first call took longer than doctor.mjs's 8 second timeout and it reported "Can't reach the backend" before the instance was warm; workshop participants would see the same lag on their first message.

I'm not suggesting an always-on instance because of the cost. Cheaper options: a manually started workflow that pings /info every few minutes during the event window (Cloud Run stops idle instances after a while, so one warm-up isn't enough), --cpu-boost, and a longer timeout with a retry in doctor.mjs/setup.mjs for hosted backends.

Statyk7 · 2026-10-24T03:00:08Z

+    runs-on: ubuntu-latest
+    timeout-minutes: 15
+    steps:
+      - uses: actions/checkout@v4


Actions are pinned by tag (@v4, @v2, @v3) in a workflow that has id-token: write in a public repo, and tags can move. Suggestion: pin every uses: to a commit SHA (keep the version in a comment) and add Dependabot for github-actions so the pins keep moving.

Statyk7 · 2026-10-24T03:00:08Z

+          region: ${{ env.REGION }}
+          service: ${{ env.SERVICE }}
+          image: ${{ steps.image.outputs.name }}
+          env_vars: |


The key and the password end up as plain environment variables of the revision, readable by anyone with viewer access on the project. Secret Manager (secrets: here, secretAccessor for the runtime service account) would be better. Fine as a follow-up for a throwaway project.

Statyk7 · 2026-10-24T03:00:08Z

+  --member "serviceAccount:$DEPLOYER_EMAIL" --role roles/artifactregistry.writer >/dev/null
+# run.admin, not run.developer: the first deploy sets the service public.
+gc projects add-iam-policy-binding "$PROJECT" \
+  --member "serviceAccount:$DEPLOYER_EMAIL" --role roles/run.admin --condition=None >/dev/null


run.admin on the whole project is broader than needed, and the comment says it's there so the first deploy can make the service public. Alternative: give the deployer run.developer, drop --allow-unauthenticated from the workflow, and have this script grant allUsers the roles/run.invoker role on the service once it exists (the smoke test can print a hint when it gets a 403). Re-running the script should also remove the old run.admin binding, or the change has no effect. I haven't tried this on GCP.

Statyk7 · 2026-10-24T03:00:08Z

+request:
+
+```sh
+fvm flutter run -d chrome \


The \ line continuation doesn't work in PowerShell (it uses a backtick), and the README says Windows is supported. A single line works everywhere.

- key: name it ANTHROPIC_API_KEY everywhere again - gate: move the password gate to runtime/gate.ts, log and cap runs per session and in total, and test it with node:test in runtime_ci - setup, doctor: mask the password by argument, check it up front with an empty run, wait longer for a cold hosted backend - deploy: keep the key and password in Secret Manager, give the deployer run.developer and make the service public from setup_gcp.sh, use --cpu-boost, add a warm_runtime workflow - actions: pin to commit SHAs and add Dependabot - docs: one-line flutter run, note that a build made with the password contains it

pietroid added 5 commits October 23, 2026 16:20

adding authentication and individual sessions

fb45e86

docs: name the shared backend URL in the README

7dff95f

docs: drop the deploy README, the setup script lists what it needs

ee962a0

Statyk7 reviewed Oct 24, 2026

View reviewed changes

Conversation

pietroid commented Oct 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Statyk7 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pietroid Oct 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pietroid Oct 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

pietroid commented Oct 23, 2026 •

edited

Loading

pietroid Oct 26, 2026 •

edited

Loading

pietroid Oct 26, 2026 •

edited

Loading