mutate_scenario.py never flips != back to ==, so every '!=' assertion in the corpus goes operator-unmutated

[Yaraslaut]

Summary

scripts/scenario/mutate_scenario.py flips three operator pairs and no more:

for original, flipped in ((" == ", " != "), (" !~ ", " ~ "), (" ~ ", " !~ ")):

!= is a destination but never a source. So an assertion written
expect ok field X != Y is never mutated at the operator, and its only mutant
is the ok→err kind flip — which is caught by the reply's kind, not by
anything the comparison itself measures. The comparison is, for mutation
purposes, unverified: an != that would pass against literally any value
(because the named field is never equal to the constant, for a reason
unrelated to what the file claims) survives the whole mutation run silently.

The mutator's own docstring names the pairs it flips as
"==↔!=, ~↔!~" — the ↔ says bidirectional, and only one
direction is implemented.

Scope in the shipped corpus

13 assertions across four files, on this branch:

scripts/scenario/scenarios/bookmarks/the-live-model-cap-is-reached.scenario:52:expect ok field @modelId != 0
scripts/scenario/scenarios/bookmarks/the-live-model-cap-is-reached.scenario:816:expect ok field @modelId != 0
scripts/scenario/scenarios/bookmarks/the-live-model-cap-is-reached.scenario:863:expect ok field @modelId != 0
scripts/scenario/scenarios/ledger/bootstrap-a-book-over-the-wire.scenario:82:expect ok field id != $book
scripts/scenario/scenarios/ledger/open-account-transact-report-close.scenario:116:expect ok field status != 2
scripts/scenario/scenarios/ledger/submit-a-report-and-poll-it.scenario:69:expect ok field status != 2
scripts/scenario/scenarios/ledger/submit-a-report-and-poll-it.scenario:73:expect ok field status != 2
scripts/scenario/scenarios/ledger/submit-a-report-and-poll-it.scenario:80:expect ok field status != 2
scripts/scenario/scenarios/ledger/submit-a-report-and-poll-it.scenario:84:expect ok field status != 2
scripts/scenario/scenarios/ledger/submit-a-report-and-poll-it.scenario:99:expect ok field status != 2
scripts/scenario/scenarios/pastebin/wire-kinds-and-typeid-refusals.scenario:39:expect ok field @modelId != 0
scripts/scenario/scenarios/pastebin/wire-kinds-and-typeid-refusals.scenario:56:expect ok field @body != "[]"

Five of the ledger ones are status != 2 (ReportStatus::Failed), which
submit-a-report-and-poll-it.scenario's own header explains was chosen
because status == 0 raced the report runner's timer. Those are the
assertions carrying that file's central claim, and the operator in them is
exactly what nothing checks.

Verification status

Reproduced. Running the mutator over
scenarios/ledger/bootstrap-a-book-over-the-wire.scenario (which has one !=
line, at 82) produces 31 mutants; line 82 contributes exactly one, the kind
flip, and no operator flip:

  line   81  kind ok->err       caught
  line   82  kind ok->err       caught
  line   85  kind ok->err       caught
  line   85  op !~->~           caught

Every neighbouring ==/~/!~ line contributes both a kind flip and an
operator flip; line 82 contributes only the kind flip.

Not verified: whether adding the (" != ", " == ") pair leaves the shipped
corpus green. It very likely does not, and that is the point — field status != 2 mutated to field status == 2 would be a survivor if and only if the
job really has reached Failed, which it should never do, so that one should
be caught. @modelId != 0 mutated to @modelId == 0 should also be caught.
I did not run it.

Not verified: whether the same one-directional gap exists for any other
operator the runner supports. morph_scenario.py's compare handles
==, !=, ~, !~ and nothing else, so the set is closed, but I only
checked the mutator's source rather than exhausting the runner's.

What would close this

Add (" != ", " == ") to the pair list, run
python3 scripts/scenario/run_scenarios.py --mutate over every rung, and
either fix or explicitly document any assertion that then survives. A
survivor found this way is the finding, not the failure.

[Yaraslaut]

Triage: valid — implement as written. One number re-measured.

Premise: confirmed, and confirmable without a server. mutants() (mutate_scenario.py:25-41) is a pure function, so the gap can be measured by importing it rather than by running a mutation campaign. Over the shipped corpus on master (2a035fe8):

ledger/submit-a-report-and-poll-it.scenario: 23 mutants
  line  69  expect ok field status != 2      -> ['kind ok->err']
  line  73  expect ok field status != 2      -> ['kind ok->err']
  line  80  expect ok field status != 2      -> ['kind ok->err']
  line  84  expect ok field status != 2      -> ['kind ok->err']
  line  99  expect ok field status != 2      -> ['kind ok->err']

One mutant per != line, and it is the kind flip. No operator flip, exactly as reported. Adding (" != ", " == ") to the pair list yields exactly one new operator mutant per assertion, at every site and nowhere else — no double-flips, and no interaction with the ~/!~ pairs (!~ contains no ~ substring, so the third pair cannot re-fire on a line the second already matched).

Re-measured count: 11 assertions, not 13. The issue counted on scenario-corpus-both-axes, which carries morph#361's ledger/bootstrap-a-book-over-the-wire.scenario; that file is not on master. Current spread:

file	!= assertions
bookmarks/the-live-model-cap-is-reached.scenario	3
ledger/submit-a-report-and-poll-it.scenario	5
ledger/open-account-transact-report-close.scenario	1
pastebin/wire-kinds-and-typeid-refusals.scenario	2

The observation that carries the issue is unchanged: five of the eleven are the status != 2 assertions whose own file header explains they were written that way because status == 0 raced the report runner's timer — the operator is the whole claim, and it is the one thing nothing measures.

Second "not verified" item, settled: the operator set is closed and this is the only one-directional gap. morph_scenario.py:621 rejects anything outside ("==", "!=", "~", "!~"), and the mutator covers ~↔!~ in both directions. Only !=→== is missing.

This is invariant 7 in its purest form — a control that appears to work while measuring nothing — so the "what would close this" is right to insist the campaign is actually run rather than the pair merely added: a survivor found that way is the finding, not the failure. Worth being explicit that field status != 2 mutated to == 2 should be caught, so a survivor there means the report runner really can reach Failed on a healthy run, which is a ledger issue and not a mutator one.

What would change this verdict: a run showing the shipped mutator already produces an op !=->== mutant for any of those eleven lines. It produces none.