Issue 3598: multiprocessing.Pool windows/linux behaviour difference

Created on 2008-08-19 06:50 by mark, last changed 2022-04-11 14:56 by admin. This issue is now closed.

Messages (4) msg71408 - (view) Author: Mark Summerfield (mark) * Date: 2008-08-19 06:50
When the attached program is run on Linux it runs "instantly" and
outputs one line, e.g.:

$ python3 mtest.py
100 files, 1702627142 bytes

(The number of bytes will vary depending on the system.)

When run on Windows XP there is no output at all; many processes seem to
be created but nothing seems to actually get done.

In both cases I'm using Py30b2.
msg71409 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * (Python committer) Date: 2008-08-19 08:49
There is a problem in your script.

On Windows, the remote processes have to import the mtest module, in
order to execute the get_size() function.
This in turn calls the main() function, which creates a new pool of
processes, etc... exponential explosion.
Linux forks and does not have this problem.

The solution of course is

if __name__ == '__main__':
    main()
msg71410 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2008-08-19 09:06
For what it's worth, this is documented in
http://docs.python.org/dev/library/multiprocessing.html#windows
msg71411 - (view) Author: Mark Summerfield (mark) * Date: 2008-08-19 09:09
On 2008-08-19, Antoine Pitrou wrote:
> Antoine Pitrou <pitrou@free.fr> added the comment:
>
> For what it's worth, this is documented in
> http://docs.python.org/dev/library/multiprocessing.html#windows

Ah yes, sorry I missed that.
History Date User Action Args 2022-04-11 14:56:37adminsetgithub: 47848 2008-08-19 09:09:58marksetmessages: + msg71411 2008-08-19 09:06:09pitrousetnosy: + pitrou
messages: + msg71410 2008-08-19 08:49:37amaury.forgeotdarcsetstatus: open -> closed
resolution: not a bug
messages: + msg71409
nosy: + amaury.forgeotdarc 2008-08-19 06:50:56markcreate