Skip to content

feat(report): add hover explanations and polish run/hunt report UI - #237

Merged
arunSunnyKVS merged 3 commits into
KeyValueSoftwareSystems:masterfrom
jithin23-kv:feat/report-hover-explanations
Aug 6, 2026
Merged

feat(report): add hover explanations and polish run/hunt report UI#237
arunSunnyKVS merged 3 commits into
KeyValueSoftwareSystems:masterfrom
jithin23-kv:feat/report-hover-explanations

Conversation

@jithin23-kv

@jithin23-kv jithin23-kv commented Aug 6, 2026

Copy link
Copy Markdown
Collaborator

Problem

Both report renderers (opfor run and opfor hunt) had accumulated a set of UX and layout issues:

  • Several metrics had no explanation for a first-time reader: severity badges look like they describe this result when they actually describe the evaluator's category, Confidence % is easy to mistake for a severity score, and hunt's Exploration stats (leads, depth, expand/dismiss) are internal jargon with zero context.
  • The hunt report's "Reconnaissance" section mixed a genuinely live metric (probe count) with commander end-of-run synthesis (fingerprint, guardrails, weak points), so a run that ended before synthesis showed a confusing empty "recon found nothing" state.
  • The Verifier row always displayed a model name (falling back to the commander's), even on runs that never used --verify — implying verification happened when it didn't.
  • Several CSS layout bugs: exec-summary cells weren't bottom-aligned, single-turn transcripts had their text flush against the card edge (missing the turn-rail's implicit margin), the "View more/less details" toggle wasn't vertically centered, and the risk badge looked misaligned against the safety-score gauge (an optical illusion from font descenders, not an actual box-alignment bug).
  • Stale/inconsistent copy: "Custom Suite" label, a leftover page footer duplicating info already in the cover band, and the hunt report's objective sitting in the cover header instead of the executive summary.

Solution

Added a lightweight, reusable hover-tooltip system to both report renderers — a small "i" icon that shows an explanation the instant the pointer enters the card/row (not the ~1s native title delay), rather than always-on text cluttering the layout. Used it to explain every metric that isn't self-evident, and fixed the CSS bugs found while building it (bottom alignment via space-between, an inline-flex vs flex bug that silently broke margin math on the details toggle, and the badge/gauge optical alignment). Also fixed the Verifier row to omit itself entirely rather than show a misleading fallback, and moved the misplaced guardrails/weak-points data into its own section instead of implying it's live recon.

Changes

Report renderers (core/src/report/render.ts, core/src/autonomous/report/html.ts):

  • New hover-tooltip system: .info-icon + .info-tooltip/.info-hover, triggered by hovering the whole card/row, not just the icon.
  • Explanatory tooltips added: Overall Verdict + risk badge (combined into one tooltip with a divider), Safety Score, Token Usage, Testing Cost, Duration, Findings (hunt), severity badges, Confidence, Attack Agents roles (hunt), Exploration stats (hunt).
  • Fixed exec-summary cell bottom-alignment (space-between + body wrapper), single-turn transcript left margin (.no-rail), details-toggle vertical centering (display:flex, not inline-flex), and the verdict-badge/gauge optical misalignment.
  • Cover band: run report gained a Judge Model column; hunt report now shows Commander/Operator/Scout models instead of Objective Outcome/Run Cost (both still shown elsewhere).
  • Duration moved into the cover-date line, labeled "Ran for:".
  • Standards now render as individual pills instead of one comma-joined string.
  • Removed the redundant page footer (report ID/date already live in the cover band).
  • Hunt report: objective moved into an "Objective" line at the bottom of the executive summary; Observed Guardrails/Weak Points moved out of "Reconnaissance" into a new "Guardrails & Weak Points" section after Findings, since that data is the commander's end-of-run synthesis, not something recon itself produces — the section is now omitted entirely (not shown empty) on runs that ended before synthesis.
  • Verifier row is now omitted entirely when --verify wasn't used, instead of falling back to the commander's model.
  • Removed the standalone attack-success-rate callouts in favor of Safety Score alone.

Suite-label rename (core/src/execute/runAll.ts, runAllBrowser.ts, core/src/execute/types.ts, runners/sdk/src/types.ts, runners/extension/popup.js): "Custom Suite" → "Custom" — three independent implementations of this label (Node run path, browser-extension path, and the extension's own display logic) all needed the same rename.

CLI flag rename (runners/cli/src/commands/hunt.ts, docs/hunt.md): --model--commander-model for consistency with --operator-model/--scout-model.

Docs: core/src/autonomous/report/types.ts's verifierModel comment clarified to state it's only ever set when verification actually ran.

Issue

N/A

How to test

opfor run --config <path>            # then open the run report and hover the "i" icons
opfor hunt --endpoint <url> --objective "..." --verify   # then open the hunt report

Verified before every push: tsc --noEmit (core/cli/sdk), eslint, prettier --check, and npm test (277 passed / 0 failed) all clean, plus the full pre-commit pipeline (typecheck, lint, format, catalog build, skill validation, gitleaks) passed on commit.

Screenshots

Verified visually via headless-Chrome screenshots at each step (exec-strip cell alignment, single-turn transcript margin, combined verdict tooltip with divider, severity/confidence tooltips rendering inline) rather than by inspection alone — not attached here, but every layout claim above was screenshot-checked before merging into this branch.

Summary by CodeRabbit

  • New Features

    • Enhanced HTML reports with clearer Pass/Fail verdicts, risk explanations, evaluator tooltips, safety scores, findings, costs, durations, and model details.
    • Added objective callouts and dedicated sections for recon guardrails and weak points.
    • Improved responsive transcript layouts, standards presentation, and overall report readability.
  • Updates

    • Custom evaluation runs are now labeled “Custom.”
    • Renamed the hunt CLI option from --model to --commander-model.
  • Documentation

    • Updated hunt command guidance and model configuration documentation to reflect the new option name.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Aug 6, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 43145f84-8a30-4f8b-8f3f-171297827e57

📥 Commits

Reviewing files that changed from the base of the PR and between e2b558d and e2a1947.

📒 Files selected for processing (2)
  • core/src/autonomous/report/html.ts
  • core/src/report/render.ts
🚧 Files skipped from review as they are similar to previous changes (2)
  • core/src/report/render.ts
  • core/src/autonomous/report/html.ts

Walkthrough

The pull request redesigns autonomous and shared HTML reports, updates report metadata and transcript behavior, standardizes non-suite labels as Custom, and renames the hunt commander model option to --commander-model.

Changes

Report presentation

Layer / File(s) Summary
Autonomous report redesign
core/src/autonomous/report/html.ts, core/src/autonomous/report/types.ts
The autonomous report uses Fail/Pass verdicts, risk explanations, model metadata, revised executive cards, tooltips, and a dedicated Guardrails & Weak Points section.
Shared report renderer updates
core/src/report/render.ts
The shared renderer adds risk and severity explanations, standards pills, revised summary cards, duration metadata, and updated transcript behavior.
Hunt option and suite label consistency
runners/cli/src/commands/hunt.ts, docs/hunt.md, core/src/execute/*, runners/extension/popup.js, runners/sdk/src/types.ts
The commander option is renamed to --commander-model. Non-suite evaluator selections use the Custom label across reports and documentation.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Possibly related PRs

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly identifies the main report UI improvements and hover explanations.
Description check ✅ Passed The description includes all required sections and provides detailed problem, solution, changes, testing, and screenshot information.
Docstring Coverage ✅ Passed Docstring coverage is 80.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@core/src/autonomous/report/html.ts`:
- Around line 762-763: Update the Safety Score tooltip in
core/src/autonomous/report/html.ts lines 762-763 to say “scoreable findings”
instead of “scoreable threads,” and update the Findings tooltip at lines 771-772
to say “One entry per finding” instead of “One entry per attack thread.”
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 6beba5fa-7f74-4864-8ed2-bcbab4c063c2

📥 Commits

Reviewing files that changed from the base of the PR and between a67cca9 and 7953623.

📒 Files selected for processing (10)
  • core/src/autonomous/report/html.ts
  • core/src/autonomous/report/types.ts
  • core/src/execute/runAll.ts
  • core/src/execute/runAllBrowser.ts
  • core/src/execute/types.ts
  • core/src/report/render.ts
  • docs/hunt.md
  • runners/cli/src/commands/hunt.ts
  • runners/extension/popup.js
  • runners/sdk/src/types.ts

Comment thread core/src/autonomous/report/html.ts Outdated
@arunSunnyKVS
arunSunnyKVS merged commit 29471d5 into KeyValueSoftwareSystems:master Aug 6, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants