Skip to content

Rank registry leaderboard by equally weighted fixture wins - #174

Open
lukekarrys wants to merge 3 commits into
codex/issue-154from
codex/issue-155
Open

lukekarrys wants to merge 3 commits into
codex/issue-154from
codex/issue-155

Conversation

@lukekarrys

@lukekarrys lukekarrys commented Sep 16, 2026

Copy link
Copy Markdown
Member

Registry leaderboard cards now rank and display fixture wins, so one slow fixture cannot dominate the cross-fixture headline. Every selected fixture/variation pair has equal weight, including both registry variations on the average tab. The UI and README explain the metric, and fixture filtering recomputes wins and the shared denominator.

Exact ties each earn a win and share a displayed rank. Failed, partial, missing, non-finite, and non-positive results earn no wins or completions; cards show incomplete coverage without inventing timing penalties. Tool filters hide cards while wins remain measured against all registries. Fixture charts retain the underlying timings for judging the size of performance differences.

Validation:

  • All 33 combined data/app regression tests pass, including large-fixture domination, ties, invalid results, all-failed contests, and fixture/variation filtering.
  • App production build and targeted ESLint/Prettier checks pass.
  • The inherited pull-request CI runs data tests and the app build.

Depends on #173; merge the data stack in order #172#170#173#174. Partial-result wording is reconciled with the win metric. The independent warmup fix #171 also passes the combined integration suite (44 tests total and a production build).

Closes #155

@coderabbitai

coderabbitai Bot commented Sep 16, 2026

Copy link
Copy Markdown

Warning

Review limit reached

Next included review available in 50 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Advanced

Run ID: bb457b42-c406-4ccc-8344-dd2457fe214a

📥 Commits

Reviewing files that changed from the base of the PR and between 2a654d4 and 1b46b73.

📒 Files selected for processing (27)
  • .github/workflows/benchmark.yaml
  • .github/workflows/test.yml
  • README.md
  • app/package.json
  • app/src/components/header.tsx
  • app/src/components/history-chart.tsx
  • app/src/components/ui/chart.tsx
  • app/src/components/variation/chart.tsx
  • app/src/components/variation/index.tsx
  • app/src/components/variation/table.tsx
  • app/src/hooks/use-history-data.ts
  • app/src/lib/history-data.ts
  • app/src/lib/registry-leaderboard.ts
  • app/src/lib/run-completeness.ts
  • app/src/lib/utils.ts
  • app/src/types/chart-data.ts
  • app/tests/partial-results.test.js
  • app/tests/statistics.test.ts
  • scripts/benchmark-data.test.js
  • scripts/benchmark-statistics.js
  • scripts/benchmark-statistics.test.js
  • scripts/clean-benchmarks.js
  • scripts/generate-chart.js
  • scripts/history-data.test.js
  • scripts/partial-results.test.js
  • scripts/process-results.sh
  • scripts/registry-leaderboard.test.js

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@lukekarrys
lukekarrys changed the base branch from main to codex/issue-154 September 16, 2026 00:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] Make the registry leaderboard summary meaningful across fixtures

2 participants