As it is decalted now it is not trivial to make contributor head what are the tested cases...
So first sing a dictionary written in a single line to its closed-to-table reading which is easier for humans; then rather define rules for run with exceptions then special cases...

This also will increase the number of running suits, but I believe it is in favor of extension stability

cc: @bsipocz @pllim

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your contributon!

However, the updated matrix is much harder for me to read, which is opposite of "making things easier for contributors" as you claimed. But I am just one data point.

The pytestoldest naming and exact pinning was actually requested by @bsipocz .

The only thing that I 100% agree on right now is adding pytest81: pytest==8.1.* job.

p.s. If we deem this PR is ok to go forward, I can approve the CI run then. For now, I will just leave it unapproved. Thank you for your patience!

the updated matrix is much harder for me to read, which is opposite of "making things easier for contributors" as you claimed. But I am just one data point.

you are right, it can be very subjective, but at least I would convert it to d61a564

The pytestoldest naming and exact pinning was actually requested by @bsipocz .

it is set as explicit 4.6 so once it fails you do not need to dive to tox try to understand what the oldest version is

Thanks for the quick response! Let's see what @bsipocz say first. 🐱

I find the matrix part more legible, but it doesn't do what we had before. Otoh, I don't find the curly brackets and longer lines etc around the one-off combinations helpful, quite the opposite.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bsipocz not sure if I got your reply correctly, sounds like you do not agree with any part of this PR, correct? in such case, let's just close it... :)

Adding pytest81: pytest==8.1.* should be uncontroversial though. Do you want to just address that part in a separate PR? :)

I like the matrix form better but unfortunately it doesn't do the same thing, see a ci run of 19 jobs on main vs 157 jobs in this PR.

So if we take away those changes that don't work, unfortunately, it's indeed just the pytest addition that remains in this PR.

I like the matrix form better but unfortunately it doesn't do the same thing, see a ci run of 19 jobs on main vs 157 jobs in this PR.

You are right as I did all combinations (and it was a demonstration not necessarily final state), but also you may want to define rich combinations for all Python and pytest versions on Ubuntu and then add a few includes for the latest and oldest configurations for MacOS and Windows.

Even that maybe an overkill. The only thing we definitely need to test for is all the (major) pytest versions as those sometimes change suddenly. Then the combination of the OS or python version can be much sparer as pytest itself already test for those. Given the incompatibilities (new pytest on old python vs old pytest on new python) the matrix form will get rather complicated to make it work, and then we'll need to add enough other OS jobs.
So maybe this all comes down to some documentation lines in the config file that summarize the above logic, and also separates the "walking through the versions" jobs from the more special cases?

I didn't mean to close just yet it with the other PR.