feat: add report turn navigator and restore standards column - #230
Conversation
WalkthroughThe change formats ChangesModel label formatting
Report rendering updates
Estimated code review effort: 3 (Moderate) | ~25 minutes Sequence Diagram(s)sequenceDiagram
participant User
participant TurnNavigationRail
participant TranscriptContent
participant IntersectionObserver
User->>TurnNavigationRail: Click a turn
TurnNavigationRail->>TranscriptContent: Scroll to the turn anchor
IntersectionObserver->>TranscriptContent: Observe the visible turn
TranscriptContent-->>TurnNavigationRail: Update the active turn
Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
🧹 Nitpick comments (2)
core/src/report/render.ts (2)
760-783: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winConsider adding tests for the new pure rendering helpers.
turnHeading,roleLabel,renderTurn, and the standards column construction are pure string-building functions with clear inputs and outputs. The PR objectives mention new coverage for model-label formatting but no test file is included for this rendering layer. A small snapshot or string-assertion test for these functions would catch regressions in the transcript markup or standards formatting without much setup effort.Do you want me to draft a starter test file for these functions?
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@core/src/report/render.ts` around lines 760 - 783, Add focused tests for the pure rendering helpers turnHeading, roleLabel, and renderTurn, including assertions for transcript markup and model-label formatting. Also test the standards column construction around formatStandardsLabel, covering both rendered standards output and its absent/empty case, using string assertions or snapshots without changing production behavior.
617-676: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winConsider extracting the shared prompt/tool-call markup between
renderTurnandsingleTurnTranscript.
renderTurn(Lines 635-676) andsingleTurnTranscript(Lines 679-711) render nearly identical prompt and tool-call markup. This change widens the gap between them:renderTurnnow adds a turn heading and an anchorid, whilesingleTurnTranscriptkeeps the old flat layout. A shared helper that accepts an optionalid/heading would remove the duplication and prevent the two paths from drifting further out of sync on future edits (for example, a future style or label change applied to one function but missed in the other).♻️ Sketch of a shared helper
-function renderTurn(turn: TurnViewModel, failingTurns: Set<number>, id: string): string { - const bubbleClass = failingTurns.has(turn.turnIndex) - ? "agent-bubble turn-highlight" - : "agent-bubble"; - const heading = turnHeading(turn.turnIndex); - ... -} - -function singleTurnTranscript(detail: DetailCard): string { - ... -} +function renderTurnBody( + detail: DetailCard, + bubbleClass: string, + heading: string +): string { + if (detail.kind === "prompt") { + return ` + ${heading} + <div class="turn-row attacker-row"> + ${roleLabel(ATTACKER_ICON, "Attacker")} + <pre>${esc(truncate(detail.prompt, 8000))}</pre> + </div> + <div class="turn-row agent-row"> + <div class="${bubbleClass}"> + ${roleLabel(AGENT_ICON, "Agent")} + <pre>${esc(truncate(detail.response, 8000))}</pre> + </div> + </div>`; + } + // ... shared tool-call branch +} + +function renderTurn(turn: TurnViewModel, failingTurns: Set<number>, id: string): string { + const bubbleClass = failingTurns.has(turn.turnIndex) ? "agent-bubble turn-highlight" : "agent-bubble"; + return `<div class="turn" id="${id}">${renderTurnBody(turn.detail, bubbleClass, turnHeading(turn.turnIndex))}</div>`; +} + +function singleTurnTranscript(detail: DetailCard): string { + return `<div class="turn">${renderTurnBody(detail, "agent-bubble", "")}</div>`; +}Also applies to: 679-711
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@core/src/report/render.ts` around lines 617 - 676, Extract the duplicated prompt/tool-call transcript markup from renderTurn and singleTurnTranscript into a shared helper, such as a helper accepting the turn, failing-turn styling, and optional heading/id inputs. Update both callers to use it, preserving renderTurn’s turn heading and anchor id while keeping singleTurnTranscript’s flat layout, and retain the existing escaping, truncation, labels, and error rendering behavior.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@core/src/report/render.ts`:
- Around line 760-783: Add focused tests for the pure rendering helpers
turnHeading, roleLabel, and renderTurn, including assertions for transcript
markup and model-label formatting. Also test the standards column construction
around formatStandardsLabel, covering both rendered standards output and its
absent/empty case, using string assertions or snapshots without changing
production behavior.
- Around line 617-676: Extract the duplicated prompt/tool-call transcript markup
from renderTurn and singleTurnTranscript into a shared helper, such as a helper
accepting the turn, failing-turn styling, and optional heading/id inputs. Update
both callers to use it, preserving renderTurn’s turn heading and anchor id while
keeping singleTurnTranscript’s flat layout, and retain the existing escaping,
truncation, labels, and error rendering behavior.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 16d16737-fbf4-4049-a326-9bd888ff7839
📒 Files selected for processing (3)
core/src/execute/aggregate.tscore/src/report/render.tscore/tests/orchestrator.equivalence.test.ts
2 participants
Problem
The report's "View more details" transcript had no way to jump to a specific turn on long multi-turn attacks. The Executive Summary risk pill had no explanation of how it's calculated. The evaluator standards mapping (OWASP/ATLAS/NIST/EU AI Act) had been dropped from the per-test detail cards during an earlier header redesign. Reports using the
openai-compatibleprovider showed a redundant model label (e.g.openai-compatible/deepseek/deepseek-chat).Solution
modelLabel()now drops theopenai-compatibleprefix and shows just the model name, since it's a generic custom-baseURL wrapper, not a real vendor identity.Changes
core/src/report/render.ts— turn navigator rail, turn heading, scrollable transcript panel + scroll-spy, risk-pill tooltip, standards columncore/src/execute/aggregate.ts—modelLabel()openai-compatible handlingcore/tests/orchestrator.equivalence.test.ts— new test for openai-compatible model labelIssue
N/A
How to test
npm run buildopfor run --config <any config>), or re-render an existing.opfor/reports/*/*-report.jsonthroughrenderReport()failingTurns), opening the card should auto-scroll to itowasp-llm: LLM01, atlas: AML.T0051) appears next to Confidence in each per-test cardprovider: "openai-compatible"and confirm the Attacker/Judge Model fields show just the model name, notopenai-compatible/<model>Screenshots
N/A — run locally to view, or see the attached preview report
Summary by CodeRabbit