last minute fixes to coherence leaderboard

[lsabor]

Summary by CodeRabbit

Release Notes

• New Features

  • Added enhanced display configuration for tournament leaderboards.

• Refactor

  • Updated tournament leaderboard scoring calculations.
  • Improved prize pool distribution and assignment system.

[coderabbitai]

📝 Walkthrough

Walkthrough

The update command now weights leaderboard scores based on per-question coverage instead of unused weights. Key changes include modifying gather_data flow, updating entry construction with squared score transformation and coverage calculation, computing minimum prize percentages, and deferring persistence until prize allocation is complete.

Changes

Cohort / File(s)

Summary

Leaderboard Scoring Update scoring/management/commands/update_coherence_tournament_leaderboard.py

Modified gather_data to return coverages; changed leaderboard creation to use score_type="relative_legacy_tournament" with fixed prize_pool and display_config; updated entry construction with entry.take = max(score, 0)^2 and per-question coverage calculation; refactored persistence pattern to buffer entries, compute minimum_prize_percent from prize amounts, assign prizes, then save entries and prune unseen ones.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

• Fix/coherence links switch h2h implementation #4274: Modifies the same command file with changes to gather_data baseline handling and per-competitor score aggregation logic, directly affecting the scoring pipeline refactored in this PR.
• feat/4214/coherence-tournament-manual-leaderboard #4215: Alters coherence leaderboard and entry creation in the same management command, including modifications to score_type and entry construction patterns.
• quickfix/coherence-leaderboard-bug #4275: Updates per-competitor aggregation logic in the same file, changing how individual competitor scores are combined in the leaderboard calculation.

Poem

🐰 Hops through the leaderboard with glee,
Coverage and prizes, now squared and free!
The tournament bounces with newfound grace,
Each competitor finds their rightful place. 🏆

🚥 Pre-merge checks | ✅ 3 | ❌ 1

❌ Failed checks (1 inconclusive)

Check name	Status	Explanation	Resolution
Title check	❓ Inconclusive	The title 'last minute fixes to coherence leaderboard' is vague and uses generic language ('last minute fixes') that doesn't convey the specific nature of the changes, making it unclear what actual modifications were made to the leaderboard.	Replace with a more specific title that describes the actual changes, such as 'Update coherence leaderboard scoring to use coverage-based weighting and relative tournament scoring' or similar.

✅ Passed checks (3 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch coherence-links-leaderboard-fix

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Review rate limit: 0/1 reviews remaining, refill in 60 minutes.}

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 1

🧹 Nitpick comments (1)

scoring/management/commands/update_coherence_tournament_leaderboard.py (1)

267-270: ⚡ Quick win

Keep the prize-threshold math in Decimal.

Casting both values back to float reintroduces binary rounding right before prize allocation, which can shift borderline entries around the minimum-prize cutoff. If the downstream helpers accept it, keep this calculation in Decimal and only convert at the boundary that needs a float.

Suggested fix

-    minimum_prize_percent = (
-        float(leaderboard.minimum_prize_amount) / float(prize_pool) if prize_pool else 0
-    )
+    minimum_prize_percent = (
+        leaderboard.minimum_prize_amount / prize_pool if prize_pool else Decimal("0")
+    )

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In `@scoring/management/commands/update_coherence_tournament_leaderboard.py`
around lines 267 - 270, The current calculation casts to float which
reintroduces binary rounding; compute minimum_prize_percent using Decimal
arithmetic instead: convert leaderboard.minimum_prize_amount and prize_pool to
Decimal and divide (or use Decimal('0') when prize_pool is falsy) so the
downstream call assign_prize_percentages_ receives a Decimal; if
assign_prize_percentages_ requires a float, defer the float conversion to the
final boundary inside that helper rather than here. Reference:
minimum_prize_percent, leaderboard.minimum_prize_amount, prize_pool, and
assign_prize_percentages_.

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@scoring/management/commands/update_coherence_tournament_leaderboard.py`:
- Around line 209-221: The current call to
Leaderboard.objects.get_or_create(...) only sets score_type, prize_pool, and
display_config on creation, leaving existing rows unchanged; change this to
ensure existing leaderboards are updated: use
Leaderboard.objects.update_or_create(...) with the same lookup keys (project and
name) and pass the desired defaults for score_type, prize_pool, bot_status and
display_config, or alternately keep get_or_create(...) and if not created assign
leaderboard.score_type, leaderboard.prize_pool, leaderboard.bot_status,
leaderboard.display_config = ... and call leaderboard.save() so the new
scoring/prize/display settings are applied on reruns.

---

Nitpick comments:
In `@scoring/management/commands/update_coherence_tournament_leaderboard.py`:
- Around line 267-270: The current calculation casts to float which reintroduces
binary rounding; compute minimum_prize_percent using Decimal arithmetic instead:
convert leaderboard.minimum_prize_amount and prize_pool to Decimal and divide
(or use Decimal('0') when prize_pool is falsy) so the downstream call
assign_prize_percentages_ receives a Decimal; if assign_prize_percentages_
requires a float, defer the float conversion to the final boundary inside that
helper rather than here. Reference: minimum_prize_percent,
leaderboard.minimum_prize_amount, prize_pool, and assign_prize_percentages_.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 1c45998f-e8cb-4efb-82d4-f6375c3eba18

📥 Commits

Reviewing files that changed from the base of the PR and between d4efe99 and e9b2c80.

📒 Files selected for processing (1)

• scoring/management/commands/update_coherence_tournament_leaderboard.py

[github-actions]

Cleanup: Preview Environment Removed

The preview environment for this PR has been destroyed.

Resource	Status
🌐 Preview App	Deleted
🗄️ PostgreSQL Branch	Deleted
⚡ Redis Database	Deleted
🔧 GitHub Deployments	Removed
📦 Docker Image	Retained (auto-cleanup via GHCR policies)

---

Cleanup triggered by PR close at 2026-05-07T14:24:26Z