Repository navigation
feat: trace Portkey chat completions - #2601
Draft
KarthikS (karthiksenv) wants to merge 1 commit into
Draft
KarthikS (karthiksenv) wants to merge 1 commit into
KarthikS (karthiksenv) wants to merge 1 commit into
Conversation
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Portkey's
chat.completions.create(...)calls currently produce no Braintrust generation spans. This adds the normal Orchestrion configuration, a Portkey channel/plugin, andwrapPortkey(client)forportkey-aiv3 chat completions.Both non-streaming and
stream: truecalls record messages, choices, model metadata, token usage, errors, and Portkey provenance. Streams aggregate text and tool-call arguments and record time to first token. Gateway configuration and HTTP options remain separate from logged metadata. Combining the wrapper with the automatic hook emits one span, andportkey/portkey-aidisable aliases are supported.Refs #2482. This change covers the primary chat-generation API; prompt management, legacy text completions, and other Portkey APIs are outside this PR's scope.
Validation on Windows with Node 24.18.0, pnpm 10.33.0, and the official
portkey-ai@3.1.0package:mainversion: all four original automatic-tracing regression cases failed because no Portkey spans were emitted. Restored the fix afterward.--import .../hook.mjssmoke test emits one Portkey span; disablingportkey-aiemits none.git diff --checkpass. ESLint reports no errors (two type-assertion warnings in the wrapper; test/config files follow the existing lint exclusions).Draft / remaining validation: No Portkey credentials were available to record real provider responses. The repository requires real-provider e2e cassettes, pinned/latest scenario variants, paired span-tree snapshots, and CI summary entries for new instrumentation. That provider-backed scenario coverage is still outstanding; the synthetic unit-test responses above are not provider recordings.