[jeanbisutti]

With an annotation that needs a specific JVM (@HeapSize, @Xmx, @JvmOptions, @MeasureHeapAllocation, @ExpectMaxHeapAllocation, @ProfileJvm...), QuickPerf executes the JUnit 5 test in a new JVM. The new JVM only got the class name and the method name, and selected the test with selectMethod(testClass, methodName). As a consequence:

• A @RepeatedTest(N) or a @ParameterizedClass ran all its invocations in each new JVM: N x N executions. Each invocation reported the measure of the last execution in its JVM, and the failures of the other executions in its JVM.
• A test method having parameters (@ParameterizedTest, RepetitionInfo, TestInfo, TestReporter, @TempDir, a parameter resolved by an extension) could not be found in the new JVM: the test failed with a JvmError, without executing the test method.
• With overloaded test methods, each new JVM executed the overload without parameters.
• The new JVM only collected the failures. A test that JUnit did not execute there (disabled by a condition) passed. A failed assumption there did not abort the test: the test failed, for example with an UnexpectedQuickPerfException when the assumption failed in a @BeforeEach or @BeforeAll method of a test having an allocation annotation.

This PR:

• Saves the JUnit unique id of the test (ExtensionContext.getUniqueId()) in the working folder before forking (junit5-test-to-execute.ser). The new JVM executes the test having this unique id, with selectUniqueId. The unique id of a test template invocation contains its index, for example [test-template-invocation:#2]. For a test executed through a @Suite, the new JVM removes the segments of the suite engine: as before, it executes the test with the Jupiter engine only, without the @BeforeSuite and @AfterSuite methods.
• Reports a test aborted in the new JVM as aborted:
  • before the test method (assumption failing in a @BeforeEach or @BeforeAll method, or in an arguments source): no performance record is evaluated;
  • in or after the test method (assumption failing in the test method or in an @AfterEach method): the performance records are evaluated, as when the test executes in the test JVM;
  • when an extension turned a failure of the test method into an abort, as retry extensions do, the test JVM gets the failure, so that the extension handles it there.
• Otherwise, fails the test with an explicit message when JUnit does not start exactly one test in the new JVM. The message gives what JUnit reported there and the possible causes. No performance record is evaluated.
• Throws what the test method threw in the new JVM, instead of an InvocationTargetException, as JUnit does, so that JUnit and the other extensions handle it as usual.
• Fails the test with the stack trace of an error that JUnit rethrows instead of reporting it, such as an OutOfMemoryError. The new JVM then still ends with System.exit, even if the test left non-daemon threads.
• Makes test template invocations use the allocation offset of the other tests.

Every JUnit 5 test executed in a new JVM now goes through the unique id selection and the check, not only the test template invocations. The main code changes are in junit5-extension only, with three new package-private classes: no change to core, to JUnit 4, to TestNG, or to a public API.

Behavior changes for users

• @RepeatedTest, @ParameterizedTest, @ParameterizedClass and test methods having parameters now work with the annotations that need a specific JVM.
• The new JVM finds the invocation of a parameterized test by its index, not by its arguments: for [test-template-invocation:#2], it calls the arguments source (@MethodSource...) again and executes the test with the second arguments it returns. So the arguments source has to return the same arguments, in the same order, in every JVM. With Set.of(...), whose iteration order changes from one JVM to another, with a HashSet of objects that do not override hashCode(), or with random values, the new JVM can execute the test with other arguments than the invocation that JUnit reports, without any warning. When the arguments source returns fewer arguments in the new JVM, the test fails with an explicit message.
• Retry extensions such as junit-pioneer @RetryingTest cannot retry a test executed in a new JVM: only the first attempt exists there. A test failing its first attempt fails, with an explicit message.
• A test that the new JVM does not execute, for example because a condition depends on a system property of the test JVM, now fails with an explicit message. The message suggests @JvmOptions("-Dname=value"), because the new JVM does not get the system properties of the test JVM.
• An assumption failing in the new JVM now aborts the test.