[Siddhant Rajhans (siddhant-rajhans)]

Description

Fixes #4230. FuzzerGenerator counted a jailbreak for every passing score. A scorer that returns several scores per response, such as AzureContentFilterScorer with one per harm category, made one jailbroken response count several times, listed its conversation several times in jailbreak_conversation_ids, and could stop at target_jailbreak_goal_count before that many responses were jailbroken.

• _process_scoring_results counts a response once when any of its scores passes and records its conversation once. Attribution by the scored piece id (FIX: attribute fuzzer jailbreak scores through the scored piece id #3300) is unchanged.
• The MCTS reward is now jailbroken responses over responses. FIX: bound the Fuzzer MCTS reward by the scored query count #3628 kept it in [0, 1] by dividing by the number of scores; with each response counted once, dividing by the number of responses keeps the same bound and makes the reward the template's jailbreak rate, as in GPTFuzzer's MCTS-Explore (succ_num / (len(questions) * len(prompt_nodes)), one verdict per response). A response that gets no score now counts as a response that was not jailbroken, where before it was left out of the rate.

The repro from #4230 (goal of 2 jailbreaks, two of four categories above the threshold):

• main: queries: 1, jailbreak conversations: 2 (1 distinct)
• this PR: queries: 2, jailbreak conversations: 2 (2 distinct)

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.