feat(cli): generate baseline of known failures#645
Closed
Chemaclass wants to merge 2 commits intomainfrom
Closed
Conversation
Capture the current run's failed/risky/incomplete tests into a baseline XML file so they no longer fail the suite. Newly introduced issues still fail the run, while baselined ones are reported separately and excluded from the failure count. Closes #284
Bash 5.2+ enables patsub_replacement by default, which makes & in a parameter expansion replacement string refer to the matched text. That broke baseline entity encoding/decoding under recent Linux runners while still passing on macOS (Bash 3.2). Switch to sed for deterministic behavior across Bash versions.
Member
Author
|
Reconsidered scope. Closing — see issue #284. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🤔 Background
Related #284
Inspired by PHPUnit's baseline feature: capture currently failing/risky/incomplete tests so the suite can focus on newly introduced issues without rewriting or skipping legacy ones.
💡 Changes
--generate-baseline <file>(andBASHUNIT_BASELINE_GENERATE) writing an XML list of failed/risky/incomplete tests; the run exits 0 so the file can be committed.--use-baseline <file>(andBASHUNIT_BASELINE_USE) which loads the XML and reports matching tests asbaselined, excluded from the failure count; unmatched failures still fail the run.[[ ]]).