AuTest timing improvements - #13540
Open
bneradt wants to merge 1 commit into
Open
Conversation
Parallel AuTest runs were repeating similarly named tests, while timing collection reset shared port allocation by starting a process per test. Several tests also waited on client defaults, retries, and log buffering rather than behavior under verification. Exact test selection -------------------- The runner now anchors worker filters to test paths. This avoids suffix collisions, duplicate runs, and parallel execution of serial tests. Batch timing collection ----------------------- The runner now measures completions from persistent worker batches. This preserves shared port allocation and removes per-test process startup. Rate-limit expiration --------------------- The rate-limit test bounds the queued client wait and requires the ATS expiration diagnostic, retaining queue-expiration coverage. Server-abort timeouts --------------------- The server-abort test disables retries and shortens connection and inactivity timeouts that are unrelated to the expected abort. JAX fingerprint log flushing ---------------------------- The JAX fingerprint test flushes log buffers once per second so its existing log assertions do not wait on production buffering defaults. Stale-response log flushing --------------------------- The stale-response test similarly flushes logs once per second while retaining its existing traffic and log assertions.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Parallel AuTest runs were repeating similarly named tests, while timing
collection reset shared port allocation by starting a process per test.
Several tests also waited on client defaults, retries, and log buffering
rather than behavior under verification.
Exact test selection
The runner now anchors worker filters to test paths. This avoids suffix
collisions, duplicate runs, and parallel execution of serial tests.
Batch timing collection
The runner now measures completions from persistent worker batches. This
preserves shared port allocation and removes per-test process startup.
Rate-limit expiration
The rate-limit test bounds the queued client wait and requires the ATS
expiration diagnostic, retaining queue-expiration coverage.
Server-abort timeouts
The server-abort test disables retries and shortens connection and
inactivity timeouts that are unrelated to the expected abort.
JAX fingerprint log flushing
The JAX fingerprint test flushes log buffers once per second so its
existing log assertions do not wait on production buffering defaults.
Stale-response log flushing
The stale-response test similarly flushes logs once per second while
retaining its existing traffic and log assertions.
Local timing results
In the
asfats5container with a Debug build, an instrumented full-suiteautest.sh -j8 -vinvocation decreased from 18m43s (1122.8 seconds) to15m09s (908.8 seconds). This saved 3m34s of wall-clock time, or 19%.
rate_limit_sniserver_abortjax_fingerprintstale_response