Issue33884
Created on 2018-06-17 10:45 by Yoni Rozenshein, last changed 2022-04-11 14:59 by admin.
| Files | ||||
|---|---|---|---|---|
| File name | Uploaded | Description | Edit | |
| mp_pickle_issues.py | Yoni Rozenshein, 2018-06-17 10:45 | |||
| Messages (2) | |||
|---|---|---|---|
| msg319811 - (view) | Author: Yoni Rozenshein (Yoni Rozenshein) | Date: 2018-06-17 10:45 | |
multiprocessing will attempt to pickle things using ForkingPickler when starting a new process in spawn mode (in Windows this is the only mode, in Linux this is a non-default but settable mode). When run within the context of a unit test, if it has to pickle a TestCase subclass, it encounters objects that can't be pickled. The attached file shows a minimum working example (uncomment the two commented lines under __main__ to run with pytest). When run with unittest.main(), it raises: TypeError: cannot serialize '_io.TextIOWrapper' object When run with pytest.main(), it raises: AttributeError: Can't pickle local object 'ArgumentParser.__init__.<locals>.identity' Note that changing the _child_process in my example to a classmethod/staticmethod or moving it to a top-level function outside the class works around this issue (both with unittest and with pytest). |
|||
| msg319826 - (view) | Author: Antoine Pitrou (pitrou) * ![]() |
Date: 2018-06-17 18:51 | |
While this is limitation of the spawn and forkserver modes, I don't see how this is a bug. Also, as you point out, making the method a classmethod or staticmethod works around the limitation. |
|||
| History | |||
|---|---|---|---|
| Date | User | Action | Args |
| 2022-04-11 14:59:01 | admin | set | status: pending -> open github: 78065 |
| 2018-06-17 18:51:59 | pitrou | set | status: open -> pending resolution: not a bug messages: + msg319826 stage: resolved |
| 2018-06-17 17:51:58 | ned.deily | set | nosy:
+ pitrou, davin |
| 2018-06-17 10:45:32 | Yoni Rozenshein | create | |
