Skip to content

Support randomized test order with reproducible seeds #14

Closed

Description

@glennmichael123

Motivation

Randomized execution can expose order-dependent tests, but failures must be reproducible.

Proposed work

  • Add options to shuffle suites and tests.
  • Accept an explicit seed and generate one when omitted.
  • Print the seed at run start and on failure.
  • Preserve the exact ordering for a given seed.
  • Include the seed in JSON, JUnit properties, and test history.

Acceptance criteria

  • Re-running with the printed seed reproduces test order.
  • Sequential and parallel modes document their ordering guarantees.
  • Unit tests verify stable ordering for a fixed seed.
  • Shuffle remains opt-in by default.

Activity

  1. glennmichael123 commented on Sep 30, 2026

    @glennmichael123
    ContributorAuthor

    Implemented in #29 and rebase-merged as dd87ffe. The pull-request workflow and the post-merge main workflow are both green across documentation, formatting, coverage, macOS, Ubuntu, and Windows.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions