Issue36512
Created on 2019-04-02 19:52 by stefanhoelzl, last changed 2022-04-11 14:59 by admin.
| Pull Requests | |||
|---|---|---|---|
| URL | Status | Linked | Edit |
| PR 12668 | open | stefanhoelzl, 2019-04-02 20:04 | |
| Messages (6) | |||
|---|---|---|---|
| msg339364 - (view) | Author: Stefan Hölzl (stefanhoelzl) * | Date: 2019-04-02 19:52 | |
adding a future_factory argument to Thread/ProcessPoolExecutor to control which type of Future should be created by Executor.submit |
|||
| msg341341 - (view) | Author: Andrew Svetlov (asvetlov) * ![]() |
Date: 2019-05-03 15:06 | |
What is your use case? |
|||
| msg342079 - (view) | Author: Antoine Pitrou (pitrou) * ![]() |
Date: 2019-05-10 17:14 | |
I'd like to know about the use case too :-) |
|||
| msg342114 - (view) | Author: Stefan Hölzl (stefanhoelzl) * | Date: 2019-05-10 17:57 | |
It would allow to use Futures with a customized interface for a specific domain. e.g. to not only save the result of a task but also some context informations or provide properties/methods which are result specific. But when subclassing Future the builtin Thread/ProcessExecutor cannot be reused anymore, because the returned class on submit cannot be customized to be the subclassed Future. With my change it would be possible to customize the Future object returned by the Executor. As it is now the Future class has to be wrapped and the Executor subclassed to return a wrapped Future object. The Future object cannot be extended without completely wrapping it. This change would make the Executor class more versatile. It would allow something like this: class CustomExecutor: ... custom_executor = CustomExecutor() custom_future = custom_executor.submit(workload, context=context_information) ... assert custom_future.context_has_some_property() assert custom_future.result_has_some_property() factories are also used in other places in standard library: logging.setLogFactory asyncio.loop.set_task_factory |
|||
| msg342178 - (view) | Author: Andrew Svetlov (asvetlov) * ![]() |
Date: 2019-05-11 09:24 | |
Sorry, but in the provided snippet I see a class with an interface which looks close to executor. There is no clear relationship with standard executors (e.g. is it inherited from ThreadExecutor?). Proposed `future_factory` is also absent. Please, provide a real example if you want to convince us that the requested feature is important and desired. |
|||
| msg342184 - (view) | Author: Stefan Hölzl (stefanhoelzl) * | Date: 2019-05-11 11:26 | |
Here is a complete example.
Please consider this is a very simple example just to demonstrate what would be possible with my proposed changes.
In the following example you can see two things:
* saving informations about the context in which a workload was submitted to the Executor in the Future
* having a custom methods on the Future returned by the Executor
from concurrent.futures import Future, ThreadPoolExecutor
def workload(n):
# does some heavy calculations
return n
class Context:
def __init__(self, name):
self.name = name
class CustomFactory(Future):
def __init__(self):
super().__init__()
self.context = None
def is_from_context(self, ctx):
return self.context.name == ctx
def processed_result(self):
# processes the result
return self.result()
class ContextExecutor(ThreadPoolExecutor):
def __init__(self):
super().__init__(future_factory=CustomFactory)
def submit(self, workload, arg, context):
future = super().submit(workload, arg)
future.context = context
return future
context_executor = ContextExecutor()
futures = [
context_executor.submit(workload, 0, context=Context("A")),
context_executor.submit(workload, 1, context=Context("B")),
context_executor.submit(workload, 2, context=Context("A")),
]
futures_from_context_a = [f for f in futures if f.is_from_context("A")]
if all(f.done() for f in futures_from_context_a):
print("All Futures in context A are done")
processed_results = [f.processed_result() for f in futures_from_context_a]
print("Results:", processed_results)
|
|||
| History | |||
|---|---|---|---|
| Date | User | Action | Args |
| 2022-04-11 14:59:13 | admin | set | github: 80693 |
| 2019-05-11 11:26:15 | stefanhoelzl | set | messages: + msg342184 |
| 2019-05-11 09:24:48 | asvetlov | set | messages: + msg342178 |
| 2019-05-10 17:57:31 | stefanhoelzl | set | messages: + msg342114 |
| 2019-05-10 17:14:59 | pitrou | set | messages: + msg342079 |
| 2019-05-10 17:14:42 | pitrou | set | nosy:
+ tomMoral |
| 2019-05-03 15:06:24 | asvetlov | set | nosy:
+ asvetlov messages: + msg341341 |
| 2019-04-03 03:36:56 | xtreak | set | nosy:
+ bquinlan, pitrou |
| 2019-04-02 20:04:34 | stefanhoelzl | set | keywords:
+ patch stage: patch review pull_requests: + pull_request12596 |
| 2019-04-02 19:52:53 | stefanhoelzl | create | |
