Skip to content

Flaky under load: run note tests, session temp-dir races, workspace and provider timing #1441

Description

@santoshkumarradha

These tests fail intermittently under load on santos/dev2. They also fail on a clean trunk, so they are not caused by the changes they were seen beside. Each was rerun alone to confirm.

Test Package Seen Rate on clean trunk
TestNotesBeyondTheBoundWaitForTheNextBoundary, TestANoteIsHandedToAWorkerOnceAndTheScreenStillReadsIt, TestANoteDoesNotChangeWhatATaskWasAskedFor internal/run "stopped at its step cap after 9 steps" about half the runs at load 27 to 35 on 20 cores (3 of 5 alone)
TestTwoWorkersLiveAtOnceEachGetOnlyItsOwnNote internal/run note routing 4 of 20
TestARunsRowNamesTheStoreTaskThePlanReadAnswersUnder internal/session temp directory cleanup race 3 of 20
TestANewDirectionArrivesWithTheClaimUnspent internal/session 1 of 10
TestEightyConcurrentAddsFromSeparateHandlesAllLand internal/workspace only under load; passes alone
TestAWireBelowItsPaceWithNothingToSpendSaysSoAndIsSpent internal/provider failed in 2 full runs; passed alone 3 of 3

The internal/run note tests are the costly ones: they turn a full make check red about half the time, so a green check on trunk is partly luck.

Suggested order: the run note tests first (a step cap reached under a slow scheduler reads as a real failure), then the two temp-directory races.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions