Feature or enhancement

While working on #110160 I've noticed that it is rather hard to reproduce random test failures. So, I want to propose a new feature / fix for that.

First of all, there's existing prior work of @vstinner who added --randseed flag.
Right now it is used together with -r to randomize test order and seeding random.

I propose to:

• Split --randseed into two options: test order randomization and random.seed usage
• Let's keep -r as-is
• Let's add --no-use-randseed flag to disable setting random.seed, let's always seed random by default
• Always print information about current random seed to be able to reuse it in the next runs

Example:

» ./python.exe -m test test_regrtest
Using random seed 65906482
0:00:00 load avg: 1.55 Run 1 test sequentially
0:00:00 load avg: 1.55 [1/1] test_regrtest

== Tests result: SUCCESS ==

1 test OK.

Total duration: 12.7 sec
Total tests: run=102 skipped=2
Total test files: run=1/1
Result: SUCCESS

Basically, this is how https://github.com/pytest-dev/pytest-randomly works.

I have a PR ready.

Linked PRs

• gh-110171: libregrtest always sets random.seed unless --no-use-randseed is provided #110172