Run tests in parallel with pytest-xdist by hugovk · Pull Request #9342 · python-pillow/Pillow
Follow on from #8972.
This runs tests by default with pytest-xdist, except disables for valgrind.
Locally, tox -e py314 goes from 44.79s for pytest to 22.78s.
And we save about an hour on the CI:
| workflow | before | after | saving | faster |
|---|---|---|---|---|
| test | 46.7m | 28.9m | 17.8m | 1.62x |
| test-docker | 133.7m | 102.0m | 31.8m | 1.31x |
| test-windows | 19.1m | 13.1m | 6.0m | 1.46x |
| test-mingw | 1.7m | 1.0m | 0.7m | 1.73x |
| Total | 201.2m | 145.0m | 56.2m | 1.39x |
This means the test order is no longer deterministic, so we don't need pytest-reverse.
Another option is to add another "extra" or "dependency group" to install pytest-xdist/pytest-sugar for certain invocations (and it'll need some config only to only set the options for these).
I put the imagegrab tests in a single group, which means they'll be allocated the same runner, to avoid some resource clashes on the CI. And I put an AVIF test into its own group because it sometimes failed locally when my machine didn't have much memory to spare.