We still have several other jobs that use ILP64, and this setup.py
based job does nothing special. Hence, let's remove it.

[skip cirrus] [skip circle] [skip azp]

The use of conda and testing on macOS is nice to combine;
no need for the separate jobs. Same for the almost 100%
duplication of the two macOS Azure jobs - we can use only
one here.

Code coverage is removed, because it's done in another job on
Linux already, and uploading to Azure isn't all that useful.
Some more cleanups to comments and code structure, so that the
job is reasonably readable now.

[skip cirrus] [skip circle]

[skip cirrus] [skip circle]