Skip to content

fix(legacy): align RubricGroup.score_group metrics and advantages with score_rollout - #2340

Open
crazywriter1 wants to merge 1 commit into
PrimeIntellect-ai:mainfrom
crazywriter1:fix/rubric-group-legacy-score-group
Open

fix(legacy): align RubricGroup.score_group metrics and advantages with score_rollout#2340
crazywriter1 wants to merge 1 commit into
PrimeIntellect-ai:mainfrom
crazywriter1:fix/rubric-group-legacy-score-group

Conversation

@crazywriter1

@crazywriter1 crazywriter1 commented Aug 12, 2026

Copy link
Copy Markdown

Supersedes #2009 — same fix, rebased onto current main after #2303 moved v0 rubrics to verifiers/legacy/.

Summary

  • RubricGroup.score_group summed child rewards correctly but left per-child advantage on state and often left stale trajectory advantage/reward from earlier children.
  • Metrics were merged into preexisting keys, unlike score_rollout and Rubric.score_group, which fully replace state["metrics"].
  • After aggregation: recompute advantage from summed rewards, fill trajectory fields only when still None, and replace metrics entirely.
  • Restore advantage and trajectory fields between child rubrics so partial writes do not leak.

Risk / notes

  • Aggregate rewards were already correct; this is contract/consistency cleanup for legacy group scoring.
  • Default Rubric.has_advantages is still False; trainers that recompute advantages from rewards are largely unaffected.

Test plan

  • Manual: children [[1,0],[0,1]] → rewards [1,1], advantages [0,0]
  • Manual: preset trajectory step reward preserved; unset steps still get aggregate fill
  • Manual: preexisting stale metrics cleared after group scoring
  • Manual: child with empty metrics → state["metrics"] == {}

Note

Fix RubricGroup.score_group to align metrics and advantages with score_rollout

  • Adds an early return with a warning when score_group receives zero states.
  • Snapshots each state's reward, metrics, advantage, and per-step trajectory values before invoking child rubrics, then restores them after each child so all rubrics see identical inputs.
  • Accumulates rewards and metrics across child rubrics, then writes the final aggregated reward to each state and sets state['advantage'] as aggregated_reward - average_reward.
  • Per-step trajectory reward and advantage are only written when currently None, matching score_rollout behavior.
  • Behavioral Change: metrics aggregation now always writes a fresh dict per state rather than conditionally merging into an existing one.

Macroscope summarized b07b7f1.

…h score_rollout

Child rubrics left per-rubric advantages on state/trajectory and score_group merged metrics into preexisting keys, unlike score_rollout and Rubric.score_group.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant