BUG Scenario success rate counts ERROR attempts that were later retried or resumed

[u7k4rs6]

When an attempt errors, an ERROR AttackResult is saved, and a retry or resume then adds a new row for the same objective. Keeping those ERROR rows as history is intentional, but objective_achieved_rate and group_success_rate divide by every row, so each recovered error still drags the rate down. Running the existing Crescendo retry-and-resume test flow and printing the rate:

rows: ['success', 'error', 'error', 'success']  unique objectives: 2  objective_achieved_rate: 50

Both objectives succeeded, so that should be 100. This number feeds the console, JSON and HTML reports. The GUI already gets it right (calculate_progress_counts explicitly avoids "inflating retries or error attempts"), so the GUI and CLI show different rates for the same run.

Suggested fix: skip ERROR attempts that were replaced by a later attempt of the same unit, using the same unit identity the GUI uses.

[romanlutz]

The diagnosis is correct for this retry/resume case. These are two successful execution units, so the rate should be 100%. The earlier ERROR attempts should remain in history, but should not count as additional execution units.

We should do these calculations in pyrit.analytics, with shared result types in pyrit.models. The backend and direct SDK callers should use the same calculation. The GUI, CLI, JSON, and HTML reports should only present its results.

As currently written, #2820 fixes ordinary recovered errors, but the calculation paths still differ:

• ScenarioResult drops superseded errors but retains every non-error attempt.
• ScenarioProgressReadModel resolves execution-unit identity and selects one latest result per unit.
• Output helpers calculate group percentages separately from the SDK's overall percentage.

These are different policies, not just different presentation. Different technique configurations sharing a name and seed can produce 100% in SDK/reports versus 50% in backend progress. Older error records can produce the reverse when the saved plan lets the backend match them to a recovered unit but the SDK cannot.

The shared analytics implementation should own execution-unit identity, including technique configuration, attempt selection, overall and grouped counts, denominators, and rounding. Historical attempt, error, and retry counts should stay separate from effective-unit statistics. This also matches the responsibilities in doc/code/framework.md.

We should add parity coverage so the same saved history produces identical statistics across the SDK, API, and reports, including retry/resume, unrecovered errors, legacy identities, and empty histories. Copying similar filtering into another layer will leave us vulnerable to the same drift.

FYI @rlundeen2

[u7k4rs6]

Thanks, that makes sense. I reworked #2820 along those lines: the calculation (unit identity including technique config, latest-attempt selection, counts, rounding) now lives in pyrit.analytics.scenario_statistics, with the result types in pyrit.models. The backend read model, the reports and SDK callers all go through it now.

One thing to flag: pyrit.models can't import pyrit.analytics under test_import_boundary, so I deprecated ScenarioResult.objective_achieved_rate instead of adding a new lazy violation. Happy to switch if you'd prefer the other way.

There's also a parity test that saves a history and checks the SDK, run detail, run history, live progress and the JSON report all agree (retry/resume, unrecovered errors, legacy identities, shared-name techniques, empty history). Details in the PR.