The attached patch adds the '-X' and '-Y' options to the regrtest tool, allowing to select a range of tests and a range of their subtests. The patch is missing the test cases for the moment.
Limitation:
Does not work very well with nested subtest (nested subtests are currently only used by the unittest test suite itself): subtest numbers are flattened across all the nested subtests.
Here is the sequence of commands that would have allowed to find the subtest responsible for the crash in test_capi at issue 22588, without modifying any code:
./python -m test -X "[]" test_capi # Get the test count: 18 tests.
./python -m test -X "range(1,10)" -R 23:23 test_capi # pass
./python -m test -X "range(10,15)" -R 23:23 test_capi # pass
./python -m test -X "range(15,17)" -R 23:23 test_capi # pass
./python -m test -X "[17]" -R 23:23 test_capi # pass
./python -m test -X "[18]" -R 23:23 test_capi # fail
./python -m test -X "[18]" -Y "[]" test_capi # Test 18 has 35 subtests.
./python -m test -X "[18]" -Y "range(1,19)" -R 23:23 test_capi # fail
./python -m test -X "[18]" -Y "range(1,10)" -R 23:23 test_capi # fail
./python -m test -X "[18]" -Y "range(1,6)" -R 23:23 test_capi # pass
./python -m test -X "[18]" -Y "range(6,8)" -R 23:23 test_capi # pass
./python -m test -X "[18]" -Y "[8]" -R 23:23 test_capi # pass
./python -m test -X "[18]" -Y "[9]" -R 23:23 test_capi # fail
Output of the last command:
Test# 18: test__testcapi
subTest# 9: internal, {'name': 'test_incref_decref_API'} |