Conversation
🧙 Wizard CIRun the Wizard CI and test your changes against wizard-workbench example apps by replying with a GitHub comment using one of the following commands: Test all apps:
Test all apps in a directory:
Test an individual app:
Show more apps
Results will be posted here when complete. |
|
Unrelated aside, I feel like you might know. What is the exact scope we need rn for the Wizard to run? I don't think we've got this written down anywhere. I'll definitely want to write this down. The minimum is |
|
Strict minimum to boot:
Without those three the wizard crashes before the agent runs. Minimum to be useful:
Current set in
Authoritative source for "what scopes does the wizard need" should be the union of:
#434 makes this declarative: wizard fetches |
gewenyu99
left a comment
gewenyu99
left a comment
There was a problem hiding this comment.
Thanks for doing this Matt!
| logToFile( | ||
| 'API key prefix:', | ||
| config.posthogApiKey | ||
| ? `${config.posthogApiKey.slice(0, 4)}***` |
There was a problem hiding this comment.
This should be fine but gonna check with @sarahxsanders. This won't trigger YARA right?
Context for you Matt, we have some pattern matching rules to prevent the agent from doing dumb stuff, like dumping API keys in logs. So checking to see if this will get yoked by Yara
There was a problem hiding this comment.
nope this shouldn't trigger it! the rules trigger on the shape of a full key
| } | ||
| // Warn (don't fail) on unexpected key prefix — `phx_` is the personal | ||
| // API key the LLM Gateway expects. We don't hard-fail because future | ||
| // PostHog releases may introduce new prefixes. |
There was a problem hiding this comment.
PostHog releases may introduce new prefixes.
Curious about this one, is this a known plan or speculative?
There was a problem hiding this comment.
Speculative - dropped that line in ee52ea1.
The wizard previously told every 401 error the user got was "Claude Code auth is conflicting with the wizard," even when no settings.json or managed-settings conflict existed. CI users hitting a 401 from a bad PAT prefix (e.g. passing pha_ OAuth tokens or phc_ project keys), missing llm_gateway:read scope, expired key, or region mismatch were sent down the wrong debug path. Re-runs checkAllSettingsConflicts at error time and only suggests 'claude auth logout' when a real conflict is detected. Otherwise lists the realistic CI-mode 401 causes. Always surfaces the verbose log path. Also warns (does not fail) on unexpected --api-key prefix in CI mode and adds a few diagnostic lines to the verbose log: API key prefix, gateway URL, and the conflict check result. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The ci flag was plumbed through showAuthError → store → session but never read anywhere. Removing it; both CI and TUI overlays already branch on hasSettingsConflict alone. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Vincent (Wen Yu) Ge <29069505+gewenyu99@users.noreply.github.com>
A leftover markdown triple-backtick after the line 225 string literal
broke tsdown's parser ("Cannot assign to this expression").
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
MattBro
force-pushed
the
fix/ci-401-error-ux
branch
from
63aee6f to
ba6511f
Compare
3 participants
Problem
When the wizard hits a 401 from the Claude agent in CI mode, it currently emits "Claude Code auth is conflicting with the wizard. Please try again after logging out: claude auth logout" regardless of the actual cause. That advice only applies when there's a real local conflict (
ANTHROPIC_*env var orapiKeyHelperin~/.claude/settings.json). For users running CI mode with a--api-keythat's the wrong token type or missing scopes, the message is misleading and sends them down the wrong path.A second issue:
--api-keyaccepts any string and silently runs to a 401 if the user passes a non-PAT token (e.g. apha_OAuth access token from a partner integration, or aphc_project key). There's no early signal that the wrong token type was passed.Changes
checkAllSettingsConflicts(options.installDir). Only emit the "Claude Code auth conflicting /claude auth logout" hint when a real conflict is detected. Otherwise emit a CI-specific message listing realistic causes: wrong token prefix (pha_/phc_instead ofphx_), missingllm_gateway:readscope, expired key, region mismatch. Always include the verbose log path so users can dig deeper. Both the CI logger and the TUI overlay use the same logic.--api-keyprefix warning. Non-phx_values now produce a visible warning identifyingpha_as an OAuth access token andphc_as a project key. The run continues — defensive, not gatekeeping.initializeAgentnow logs the API key prefix (phx_***), the gateway URL, and the result ofcheckAllSettingsConflicts. The 401 handler also logs the conflict-check verdict. Per-message SDK JSON dumps already capture upstream response bodies, so no separate body-logging entry was added.Test plan
I'm an agent (Claude Opus 4.7). Automated checks:
pnpm test— 589/589 passpnpm run lint— 0 errors (only pre-existing warnings)pnpm run buildand typecheck — cleanNo manual end-to-end testing.
Follow-up (not in this PR)
src/utils/setup-utils.ts:495requestsintrospectionin the interactive OAuth scope set. Per PostHog/posthog#56835,introspectionis not a grantable OAuth scope — it's the RFC 7662 token-introspection endpoint, exposed asintrospection_endpointper RFC 8414. Either the AS silently drops it or the wizard's interactive OAuth quietly errors withinvalid_scope. Either way, it's stale and should be removed. Leaving as a separate cleanup since this PR is scoped to CI 401 UX.LLM context
Authored by Claude Opus 4.7 via Claude Code. Driven by a real partner-integration 401 where a
pha_OAuth access token was rejected by the LLM Gateway and the wizard's "Claude Code auth conflicting" message sent the user down the wrong debugging path. Companion PR in posthog/posthog adds the missing scopes toALLOWED_PROVISIONING_SCOPES. This wizard PR makes the wizard more honest about what's actually wrong when a 401 hits.