Messages (9) msg338576 - (view) Author: Brian McCutchon (Brian McCutchon) Date: 2019-03-22 01:39

Currently, it is possible to make a basic single-threaded executor for unit testing:

class FakeExecutor(futures.Executor):

  def submit(self, f, *args, **kwargs):
    future = futures.Future()
    future.set_result(f(*args, **kwargs))
    return future

  def shutdown(self, wait=True):
    pass

However, this evaluates the provided function eagerly, which may be undesirable for tests. It prevents the tests from catching a whole class of errors (those where the caller forgot to call .result() on a future that is only desirable for its side-effects). It would be great to have an Executor implementation where the function is only called when .result() is called so tests can catch those errors.

I might add that, while future.set_result is documented as being supported for unit tests, a comment in the CPython source says that Future.__init__() "Should not be called by clients" (https://github.com/python/cpython/blob/master/Lib/concurrent/futures/_base.py#L317), suggesting that even the above code is unsupported and leaving me wondering how I should test future-heavy code without using mock.patch on everything.

------ Alternatives that don't work ------

One might think it possible to create a FakeFuture to do this:

class FakeFuture(object):

  def __init__(self, to_invoke):
    self._to_invoke = to_invoke

  def result(self, timeout=None):
    return self._to_invoke()

However, futures.wait is not happy with this:

futures.wait([FakeFuture(lambda x: 1)])  # AttributeError: 'FakeFuture' object has no attribute '_condition'

If FakeFuture is made to extend futures.Future, the above line instead hangs:

class FakeFuture(futures.Future):

  def __init__(self, to_invoke):
    super(FakeFuture, self).__init__()
    self._to_invoke = to_invoke

  def result(self, timeout=None):
    return self._to_invoke()

I feel like I shouldn't have to patch out wait() in order to get good unit tests.

msg341616 - (view) Author: Brian Quinlan (bquinlan) * Date: 2019-05-06 19:29

Hey Brian, why can't you use threads in your unit tests? Are you worried about non-determinism or resource usage? Could you make a ThreadPoolExecutor with a single worker?

msg341624 - (view) Author: Brian McCutchon (Brian McCutchon) Date: 2019-05-06 19:42

Mostly nondeterminism. It seems like creating a ThreadPoolExecutor with one worker could still be nondeterministic, as there are two threads: the main thread and the worker thread. It gets worse if multiple executors are needed.

Another option would be to design and document futures.Executor to be extended so that I can make my own fake executor.

msg341625 - (view) Author: Brian Quinlan (bquinlan) * Date: 2019-05-06 19:46

Do you have a example that you could share?

I can't think of any other fakes in the standard library and I'm hesitant to be the person who adds the first one ;-)

msg341660 - (view) Author: Brian McCutchon (Brian McCutchon) Date: 2019-05-06 23:05

I understand your hesitation to add a fake. Would it be better to make it possible to subclass Executor so that a third party implementation of this can be developed?

As for an example, here is an example of nondeterminism when using a ThreadPoolExecutor with a single worker. It sometimes prints "False" and sometimes "True" on my machine.

from concurrent import futures
import time

complete = False

def complete_eventually():
  global complete
  for _ in range(150000):
    pass
  complete = True

with futures.ThreadPoolExecutor(max_workers=1) as pool:
  pool.submit(complete_eventually)
  print(complete)

msg341740 - (view) Author: Brian Quinlan (bquinlan) * Date: 2019-05-07 14:53

Hey Brian,

I understand the non-determinism. I was wondering if you had a non-theoretical example i.e. some case where the non-determinism had impacted a real test that you wrote?

msg341797 - (view) Author: Brian McCutchon (Brian McCutchon) Date: 2019-05-07 18:55

No, I do not have such an example, as most of my tests try to fake the executors.

msg341890 - (view) Author: Brian Quinlan (bquinlan) * Date: 2019-05-08 15:31

Brian, I was looking for an example where the current executor isn't sufficient for testing i.e. a useful test that would be difficult to write with a real executor but would be easier with a fake.

Maybe you have such an example from your tests?

msg369603 - (view) Author: Leonardo Santagada (santagada) Date: 2020-05-22 14:18

I have a single example:

Profiling. As most python profilers don't support threads or processes, it would be very convenient to have a in process executor in those cases.

History Date User Action Args 2022-04-11 14:59:12adminsetgithub: 80576 2020-05-22 14:18:17santagadasetnosy: + santagada
messages: + msg369603
2019-05-08 15:31:39bquinlansetmessages: + msg341890 2019-05-07 18:55:39Brian McCutchonsetmessages: + msg341797 2019-05-07 14:53:47bquinlansetmessages: + msg341740 2019-05-06 23:05:13Brian McCutchonsetmessages: + msg341660 2019-05-06 19:46:49bquinlansetmessages: + msg341625 2019-05-06 19:42:16Brian McCutchonsetmessages: + msg341624 2019-05-06 19:29:26bquinlansetmessages: + msg341616 2019-03-22 01:55:07xtreaksetnosy: + bquinlan, pitrou
2019-03-22 01:39:20Brian McCutchoncreate