Repository navigation
test: kill the surviving mutants that matter - #19
Merged
Merged
Conversation
Add a test for `&foo=true` -> true: no test checked the `'true'` case in defaultTransform, so two mutants of it survived. Skip the 200 000-character timing tests during mutation runs (STRYKER_MUTATOR_WORKER is set). Stryker's instrumented code runs ~4x slower on all cores at once, so they failed without a mutant and killed random mutants, which made the score move between runs. The score is now 95.38% in every run. The 14 surviving mutants are equivalent. Stryker disable comments can't exclude them: a comment disables every mutant of that mutator on the line, which would also turn off 11 mutants the tests kill today. Set thresholds.break to 95.3 instead, so one new surviving mutant fails the run.
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #19 +/- ##
=========================================
Coverage 100.00% 100.00%
=========================================
Files 4 4
Lines 134 134
Branches 45 45
=========================================
Hits 134 134 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
- An empty object or array that a kept entry set (via transformEntry) must stay when a removed entry goes through it. Kills both mutants of `isRemoved && nextPathValue === undefined`. - With an empty-string property on Object.prototype, `a[]` entries stay: only object path parts are compared with Object.prototype. Kills `pathPart.type === 'object'` -> `true`. The score is 96.37% (292 of 303) in every run; the 11 survivors are equivalent. Raise thresholds.break to 96.3.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #18. Changes tests and the Stryker config only;
test:doesn't release.Changes
&foo=true→{foo: true}. No test checked the'true'case indefaultTransform(src/transform.ts:64), sovalue === 'true'→false/value === ""survived.transformEntrycan return{}/[]as a value (the suite already tests that,index.test.ts"removes the entry %s with the value %j"). If a removed entry (a.toString,a[].toString) then goes through it, undoing the removed entry must not delete it:{a: {}}/{a: []}. KillsisRemoved && nextPathValue === undefined→isRemoved && trueand&&→||(index.ts:407).Object.prototypedoesn't removea[]entries. Only object path parts are compared withObject.prototype(as the JSDoc says); the[]part's path is''. Same polluted-runtime setup and cleanup as the existing "a property added to Object.prototype" tests. KillspathPart.type === 'object'→true(index.ts:377).it.skipIf(process.env['STRYKER_MUTATOR_WORKER'] !== undefined). Stryker sets that variable in its worker processes;npm testand CI still run the tests. A localdeclare const processgives the type, because the project has no@types/node.thresholds: {high: 100, low: 95, break: 96.3}(wasbreak: 93).Why skip the timing tests
Measured with the instrumented code from a Stryker sandbox and no mutant active:
a[b]a[0]bStryker compiles all mutants into the source, and every expression runs through
stryMutAct_…/stryCov_…calls. The scanner loop runs once per character, so these tests went over 500 ms (7 of 30 in the parallel measurement) and "killed" whichever mutant was active. In #18's runs, every kill that varied between runs came from these tests. After the skip the score is the same in every run.Score and threshold
96.37% (292 of 303) in every run.
break: 96.3, so one new surviving mutant (291/303 = 96.04%) fails the run. Checked with the earlier threshold: removing thetruetest dropped the score belowbreakand the run exited 1.No
// Stryker disablecomments: the directive only accepts mutator names, not single replacements, and disables every mutant of that mutator on the line. On these lines that would also turn off mutants the tests kill today (e.g.idx < path.length→idx >= path.lengthnext to the equivalentidx <= path.length). If the score goes up,breakshould be raised.The 11 equivalent mutants
Each one was checked by a separate investigation: a proof from the code plus, for the scanner mutants, a differential search comparing the original and the mutant on millions of generated paths (all with 0 differences).
index.ts:166if (!isPrototypeKey(key))→if (false), and its block →{}Object.prototype,definePropertywith all flagstrueequals assignment on the library's own plain objects (same attributes, same key order)index.ts:176writable: true→falsedefinePropertyon aconfigurableproperty, and the key is deleted before returningindex.ts:177enumerable: true→falseObject.keyscheck (backwards walk)index.ts:367createdByRemoved = []→["Stryker was here"]container('r') with one key, so the loopcontinuespath-parts.ts:47if (open > lastClose) return -1→if (false)lastClose < openis returned, which the caller skips like-1; the guard is redundantpath-parts.ts:47>→>=open === lastCloseis impossible ([vs])path-parts.ts:49idx < lastClose→<=lastClosethe function returnslastCloseeither waypath-parts.ts:117idx < path.length→<=path[path.length]isundefined, which the loop body skipspath-parts.ts:119close < idx→<=closeis-1or afteridx, never equalpath-parts.ts:122close > idx→>=close === idxcan't happenindex.ts:176,index.ts:177andindex.ts:166can be killed only with tests we didn't add:Object.prototypefrom insidetransformEntrybetween two entries, or (177) inspecting an object passed in with a cast after the parse threw.Those tests would pin down unrealistic or undocumented behaviour.
npm run validatepasses (320 tests).