Fix check version timeout Version 2 (compatible with Windows and MacOS) by maximpavliv · Pull Request #2820 · DeepLabCut/DeepLabCut
For info:
The difference in multiprocessing between Linux and Windows or MacOS is how it starts new processes (by default):
- On Linux, the default method is
"fork" - On Windows and MacOS, the default method is
"spawn"
This can be verified by doing
import multiprocessing
multiprocessing.get_start_method()
With "fork", the child process inherits the memory space, file descriptors, and environment from the parent process. With "spawn", a fresh Python interpreter is started, and only the necessary objects (like the target function) are passed to the child process.
Therefore, to successfully run the tests on all OSes, the methods passed to multiprocessing cannot be defined in a local scope, because they cannot be pickled if it is the case, which then is not compatible with the "spawn" method.
Another consequence is that, in practice, deeplabcut needs to be reimported in each child process if using the "spawn" method (both when running test_call_with_timeout() and when running _check_for_updates()), which might take some time. This is not the case if new processes are started using "fork" method (Linux).
@n-poulsen do you see any potential issue with this, that I didn't think of?