Repository navigation
Test parallellization uses a stupid strategy #20980
Description
Description
When running ./run-test.php with the -j argument, all the tests are scheduled to the workers before even running them. This means that if you have a worker which finishes all its tests quickly it'll sit idle even if there are many more other tests to run. One can observe this often because at the end of a test run the number of concurrent working test runners drops quickly.
PHP Version
8.4+
Operating System
N/A
Reacted by Michael Voříšek
Activity
iluuu1994 commentedon Jan 20, 2026
Another dumb thing that has bitten me before: When running tests without
-j, tests are printed to the console before they are executed (without--no-progress). With-j, they are printed after.One note though is that many of the last tests are marked as
CONFLICTS: all. So they are intentionally unparallelized.ndossche commentedon Jan 20, 2026
Yes. I was in particular looking at the OpenSSL tests, which has a CONFLICTS file but does no longer seem necessary, after removing that the number of concurrent workers still drop pretty quickly.
arnaud-lb commentedon Feb 6, 2026
Apparently we send work by batches of 33 tests. Slow tests happen to be grouped together, like the lstat variation ones, so if a worker receives a batch of those tests it will take more time to complete.
On my machine, all workers are busy for about 10 seconds, then only 2-3 workers for about the same time, and then sequential tests run for about 4 seconds.
Reducing batch size to
1doesn't reduce total time due to communication overhead. I've also tried using batch sizes of min(33, #remainingTests/#workers), but no improvements either.Using
--shufflereduces the total time by about 30%.Reduce parallel test batch size