test(extension): replace model-resolution it.todo placeholders - #2696
Open
HaoChiBao wants to merge 2 commits into
Open
test(extension): replace model-resolution it.todo placeholders#2696HaoChiBao wants to merge 2 commits into
HaoChiBao wants to merge 2 commits into
Conversation
Add controller-level coverage for model string and object resolution paths that were previously marked it.todo.
Contributor
There was a problem hiding this comment.
All reported issues were addressed across 1 file
Architecture diagram
sequenceDiagram
participant T as Test Suite
participant C as Stagehand Controller
participant R as Runtime
participant BS as Browserbase Service
participant S as act/extract/observe Services
participant LLM as LLM Service
Note over T,LLM: Model Resolution and Gateway Routing Paths
Note over T: Direct Inference with Provider Auth
T->>C: init(model={name, apiKey})
C->>R: Configure runtime
R-->>C: Runtime ready
T->>C: act/extract/observe
C->>S: call service(model, gateway=undefined)
S-->>T: Verify no gateway passed
Note over T: Browserbase Managed Inference
T->>C: init(apiKey, browser, keyless model)
C->>R: Configure runtime
R-->>C: Runtime ready
T->>C: act/extract/observe
C->>BS: Resolve gateway (apiUrl, sessionId)
BS-->>C: Gateway config
C->>S: call service(model, gateway)
S-->>T: Verify gateway routing
alt Local Browser with Keyless Model
T->>C: init(browserCdpUrl, keyless model)
C->>R: Configure runtime
R-->>C: Runtime ready
T->>C: act/extract/observe
C->>S: Forward model (gateway=undefined)
S->>LLM: Attempt inference
LLM-->>S: Reject (no gateway available)
S-->>T: Verify forwarding behavior
else Local Browser without Model
T->>C: init(browserCdpUrl)
C->>R: Configure runtime
R-->>C: Runtime ready
T->>C: act/extract/observe
C-->>T: Error: "An LLM was not configured"
end
Note over T: Client Inference Path
T->>C: init(model={source: "client"})
C->>R: Configure runtime with SDK callback
R-->>C: Runtime ready
T->>C: act/extract/observe
C->>S: call service(client callback)
S-->>T: Verify SDK callback usage
Note over T: Per-Call Model Overrides
T->>C: init(modelA)
C->>R: Configure runtime
R-->>C: Runtime ready
T->>C: act(model=modelB)
C->>S: call service(modelB)
S-->>T: Verify override, no credential inheritance
Reply with feedback, questions, or to request a fix.
Re-trigger cubic
Extract shared helpers so each case only states init params and expected forwarded fields, without repeating act/extract/observe setup.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
it.todoplaceholders inpackages/extension/tests/model-resolution.test.tswith controller-level coverage for model string and object resolution paths.llmService).Test plan
pnpm --filter @browserbasehq/stagehand-extension exec vitest run tests/model-resolution.test.ts(or equivalent package filter) passesit.todoin that fileSummary by cubic
Replaces it.todo tests in
packages/extension/tests/model-resolution.test.tswith controller-level coverage for model resolution and Browserbase gateway routing, and dedupes primitive spy scaffolding via shared helpers. Increases confidence in inference selection without changing runtime behavior.llmService).llmService); rejects a missing model with a local browser.model.source: "client"uses the connected SDK callback.Written for commit 74c6825. Summary will update on new commits.