Add AI agent conversations landed by the AI Sessionizer - #14057
Open
wu-sheng wants to merge 1 commit into
Open
Conversation
wu-sheng
requested review from
peachisai and
wankai123
and removed request for
wankai123
September 5, 2026 04:51
wu-sheng
force-pushed
the
feat/ai-agent-conversation
branch
2 times, most recently
from
September 5, 2026 06:57
b5cae36 to
480ed7a
Compare
The AI Sessionizer (apache/skywalking-ai-sessionizer) pushes an agent
runtime's conversation as Session Data and Session Flow files over OTLP
logs under a new AI_AGENT layer. The OAP verifies each file's digest and
line count in the bundled LAL rule's ConversationFile output builder,
stores it verbatim in two record models, ai_agent_session_data and
ai_agent_session_flow, in a BanyanDB group of their own, recordsAIAgent,
and answers a conversation with one asz.view 1.0 document, equal key for
key to the one the Sessionizer prints for the same files.
The list page and the raw-file export are GraphQL queries,
listConversations and getConversationRawFiles. The document itself is
served by GET /ai-agent/conversations/{conversation}/v1/view on the same
HTTP server, streamed and compressed under its own timeout, because a
long conversation renders to tens of megabytes: measured, a 136 MB
session folds to a 70 MB document in about eleven seconds, past the
server's ten second default. Its Content-Type names the format and the
version, application/vnd.asz.view+json; version=1.0 or the +yaml twin,
and an error is an application/problem+json document.
The e2e builds sessions with the Sessionizer's own scenario tool, lands
one of them in three stages, pushes everything to the OAP, and compares
every document with the one the Sessionizer's viewer serves, over
HTTP/1.1 and HTTP/2, as JSON and YAML, gzipped or not.
wu-sheng
force-pushed
the
feat/ai-agent-conversation
branch
from
September 5, 2026 07:47
480ed7a to
9a51432
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add AI agent conversations landed by the AI Sessionizer
If this is non-trivial feature, paste the links/URLs to the design doc. The protocol design is in ai-agent: add conversation query protocol for the AI Sessionizer skywalking-query-protocol#166, ai-agent: serve the conversation view over a streamed HTTP route, not GraphQL skywalking-query-protocol#167 and ai-agent: the view route says format, version and errors the HTTP way skywalking-query-protocol#168 (the list and export queries, and the streamed view route); the file formats and the
asz.viewdocument are the Sessionizer's under https://skywalking.apache.org/docs/skywalking-ai-sessionizer/next/en/formats/asz-view/; the operator doc of this feature isdocs/en/setup/backend/ai-agent-conversation.md.Update the documentation to include this new feature.
Tests(including UT, IT, E2E) are added to verify the new feature.
If it's UI related, attach the screenshots below.
If this pull request closes/resolves/fixes an existing issue, replace the issue number. Closes #.
Update the
CHANGESlog.The AI Sessionizer collects an agent runtime's transcripts into two file formats, Session Data (
.sd) and Session Flow (.sf, an append-only chain of rounds), and pushes every file as one OTLP log record. This PR makes the OAP store them and serve a conversation whole.Ingest. A new
AI_AGENTlayer routes the records to the bundled LAL rulelal/ai-agent.yaml. Its output builder,ConversationFile, verifies each file's sha256 and line count against the record attributes and stores it verbatim, or drops it and counts the reason inai_agent_conversation_files_rejected. Nothing is decoded at ingest, so a cluster needs no shared state.Storage. Two record models,
ai_agent_session_dataandai_agent_session_flow, super datasets on every storage, in a BanyanDB group of their own,recordsAIAgent, with hot, warm and cold stages like the log group. A row belongs to its sender: its id is the service, the instance and the file digest, so a retry lands on the same row and another sender's copy is another row.Query. The list page and the raw-file export are GraphQL queries,
listConversationsandgetConversationRawFiles. The conversation itself isGET /ai-agent/conversations/{conversation}/v1/viewon the core HTTP server beside/graphql: oneasz.view1.0 document, equal key for key to the oneasz conversationprints for the same files, streamed as it is rendered,Content-Type: application/vnd.skywalking.asz.view+json; version=1.0or the+yamltwin, gzip onAccept-Encoding, errors asapplication/problem+json, under its own timeout. It is not a GraphQL query because a long conversation renders to tens of megabytes: measured, a 136 MB session folds to a 70 MB document in about eleven seconds, past the HTTP server's ten second default. Rounds and files are read window by window (roundReadWindow,fileReadWindow), across every sender when none is named, and over every retained stage on BanyanDB.Tests. Unit tests compare the built document with the Sessionizer's own output for the same files, cover the file builder, the formats, the raw-JSON preservation, and the route on an in-process server (JSON, YAML, gzip, chunking, UTF-8 at a chunk boundary, the error statuses). The e2e, one shared case on BanyanDB, Elasticsearch, MySQL and PostgreSQL, builds sessions with the Sessionizer's scenario tool, lands one in three stages, pushes everything to the OAP, and reads it back through
swctl ai-agent list,filesandview(apache/skywalking-cli#234, pinned asSW_CTL_COMMIT), comparing every document with the one the Sessionizer's viewer serves, over HTTP/1.1 and HTTP/2, as JSON and YAML, gzipped or not, plus the export's digests, a rejected file, and the three-round chain. All four passed locally.Known limit. On JDBC, a retry of the same file by the same sender hits the same primary key, which the shared record writer's plain
INSERTrejects; BanyanDB and Elasticsearch treat it as a duplicate the reader ignores or an overwrite. Content under a key is immutable by construction, so an insert-or-ignore in the common JDBC writer would be safe; that is a change outside this feature.