7 Common Errors When Using the ThreadPool

Last Updated on October 29, 2022

You may encounter one among a number of common errors when using the ThreadPool in Python.

These errors are often easy to identify and often involve a quick fix.

In this tutorial, you will discover the common errors when using the ThreadPool in Python and how to fix each in turn.

Let’s get started.

There are a number of common errors when using the ThreadPool.

These errors are typically made because of bugs introduced by copy-and-pasting code, or from a slight misunderstanding of how the ThreadPool works.

We will take a closer look at some of the more common errors made when using the ThreadPool, such as:

Using a Function Call in submit()
Using a Function Call in map()
Incorrect Function Signature for map()
Incorrect Function Signature for Future Callbacks
Tasks Fail Silently
Joining Pool While Running
Issuing Tasks to a Closed Pool

Do you have an error using the ThreadPool?
Let me know in the comments so I can recommend a fix and add the case to this tutorial.

Error 1: Using a Function Call in apply_async()

A common error is to call your function when using the apply_async() function.

For example:

...

# issue the task

result = pool.apply_async(task())

A complete example of this error is listed below.

# SuperFastPython.com

# example of calling submit with a function call

from time import sleep

from multiprocessing.pool import ThreadPool

# custom function executed in another thread

def task():

# block for a moment

sleep(1)

return 'all done'

# protect the entry point

if __name__ == '__main__':

# start the thread pool

with ThreadPool() as pool:

# issue the task

result = pool.apply_async(task())

# get the result

value = result.get()

print(value)

Running this example will fail with an error.

Traceback (most recent call last):

...

TypeError: 'str' object is not callable

You can fix the error by updating the call to apply_async() to take the name of your function and any arguments, instead of calling the function in the call to execute.

For example:

...

# issue the task

result = pool.apply_async(task)

Error 2: Using a Function Call in map()

A common error is to call your function when using the map() function.

For example:

...

# issue all tasks

for result in pool.map(task(), range(5)):

print(result)

A complete example of this error is listed below.

# SuperFastPython.com

# example of calling map with a function call

from time import sleep

from multiprocessing.pool import ThreadPool

# custom function executed in another thread

def task(value):

# block for a moment

sleep(1)

return 'all done'

# protect the entry point

if __name__ == '__main__':

# start the thread pool

with ThreadPool() as pool:

# issue all tasks

for result in pool.map(task(), range(5)):

print(result)

Running the example results in a TypeError.

Traceback (most recent call last):

...

TypeError: task() missing 1 required positional argument: 'value'

This error can be fixed by changing the call to map() to pass the name of the target task function instead of a call to the function.

...

# issue all tasks

for result in pool.map(task, range(5)):

print(result)

Free Python ThreadPool Course

Download your FREE ThreadPool PDF cheat sheet and get BONUS access to my free 7-day crash course on the ThreadPool API.

Discover how to use the ThreadPool including how to configure the number of worker threads and how to execute tasks asynchronously

Learn more

Error 3: Incorrect Function Signature for map()

Another common error when using map() is to provide no second argument to the function, e.g. the iterable.

For example:

...

# issue all tasks

for result in pool.map(task):

print(result)

A complete example of this error is listed below.

# SuperFastPython.com

# example of calling map without an iterable

from time import sleep

from multiprocessing.pool import ThreadPool

# custom function executed in another thread

def task(value):

# block for a moment

sleep(1)

return 'all done'

# protect the entry point

if __name__ == '__main__':

# start the thread pool

with ThreadPool() as pool:

# issue all tasks

for result in pool.map(task):

print(result)

Running the example does not issue any tasks to the ThreadPool as there was no iterable for the map() function to iterate over.

Running the example results in a TypeError.

Traceback (most recent call last):

...

TypeError: map() missing 1 required positional argument: 'iterable'

The fix involves providing an iterable in the call to map() along with your function name.

...

# issue all tasks

for result in pool.map(task, range(5)):

print(result)

Error 4: Incorrect Function Signature for Callbacks

Another common error is forgetting to include the result in the signature for the callback function when issuing tasks asynchronously.

For example:

# result callback function

def handler():

print(f'Callback got: {result}')

A complete example of this error is listed below.

# SuperFastPython.com

# example of a callback function for apply_async()

from time import sleep

from multiprocessing.pool import ThreadPool

# result callback function

def handler():

print(f'Callback got: {result}')

# custom function executed in another thread

def task():

# block for a moment

sleep(1)

return 'all done'

# protect the entry point

if __name__ == '__main__':

# create and configure the thread pool

with ThreadPool() as pool:

# issue tasks to the thread pool

result = pool.apply_async(task, callback=handler)

# get the result

value = result.get()

print(value)

Running this example will result in an error when the callback is called by the ThreadPool.

This will break the ThreadPool and the program will have to be killed manually with a Control-C.

Exception in thread Thread-11:

Traceback (most recent call last):

...

TypeError: handler() takes 0 positional arguments but 1 was given

Fixing this error involves updating the signature of your callback function to include the result from the task.

# result callback function

def handler(result):

print(f'Callback got: {result}')

You can learn more about using callback functions with asynchronous tasks in the tutorial:

ThreadPool Callback Functions in Python

This error can also happen with the error callback and forgetting to add the error as an argument in the error callback function.

Loving The Tutorials?

Why not take the next step? Get the book.

Learn more

Error 5: Tasks Fail Silently

A common error is when tasks are issued to the ThreadPool but fail silently.

The expected result or output does not occur and no message is provided by the ThreadPool.

For example:

# SuperFastPython.com

# example of asynchronous tasks failing silently in the thread pool

from time import sleep

from multiprocessing.pool import ThreadPool

# task executed in a worker thread

def task():

# block for a moment

sleep(1)

# fail

raise Exception('Something bad happened')

# report a message

print(f'Task done')

# protect the entry point

if __name__ == '__main__':

# create and configure the thread pool

with ThreadPool() as pool:

# issue an asynchronous task into the thread pool

result = pool.apply_async(task)

# wait for all tasks to finish

result.wait()

Running the example results in no message from the task or the ThreadPool itself.

In order to trigger the error, we must attempt to retrieve the result from the asynchronous task.

For example:

...

# get the result

result.get()

Alternatively, we can register an error callback function with the task.

For example:

# error callback function

def callback(error):

print(f'Error: {error}')

...

# issue task and register an error callback

result = pool.apply_async(task, error_callback=callback)

You can learn more about tasks failing silently in the ThreadPool in the tutorial:

ThreadPool Error Callback Functions in Python

Error 6: Joining Pool While Running

Another common error occurs when attempting to join the ThreadPool in order to wait for all running tasks to complete.

This is achieved by calling the join() method.

For example:

# SuperFastPython.com

# example of an error while joining the pool

from time import sleep

from multiprocessing.pool import ThreadPool

# custom function executed in another thread

def task():

# block for a moment

sleep(1)

return 'all done'

# protect the entry point

if __name__ == '__main__':

# start the thread pool

with ThreadPool() as pool:

# issue the task

result = pool.apply_async(task)

# wait for all tasks to finish

pool.join()

Running the example results in an exception.

Traceback (most recent call last):

...

ValueError: Pool is still running

This error occurs because you attempt to join the ThreadPool while it is still running.

You can fix this error by first closing the pool by calling close() or terminate().

For example:

...

# close the pool

pool.close()

# wait for all tasks to finish

pool.join()

You can learn more about joining the ThreadPool in the tutorial:

How to Join a ThreadPool in Python

Error 7: Issuing Tasks to a Closed Pool

A common error occurs when attempting to issue tasks to the ThreadPool.

This can happen if the pool was inadvertently closed before the task was issued.

For example:

# SuperFastPython.com

# example of issuing tasks to a pool that is closed

from time import sleep

from multiprocessing.pool import ThreadPool

# custom function executed in another thread

def task():

# block for a moment

sleep(1)

return 'all done'

# protect the entry point

if __name__ == '__main__':

# start the thread pool

with ThreadPool() as pool:

# issue the task

result = pool.apply_async(task)

# close the pool

pool.close()

# wait for all tasks to finish

pool.join()

# issue another task

result = pool.apply_async(task)

Running the example results in an exception.

Traceback (most recent call last):

...

ValueError: Pool not running

This error occurs because you have closed the ThreadPool and then attempted to issue tasks to execute.

The pool cannot execute tasks if it is not running.

You must start a new pool or issue tasks before closing the pool.

You can learn more about correctly shutting down the ThreadPool in the tutorial:

How to Shutdown the ThreadPool in Python

Takeaways

You now know about the common errors when using the ThreadPool in Python.

Do you have any questions?
Ask your question in the comments below and I will do my best to answer.

Photo by ZHENYU LUO on Unsplash

Error 1: Using a Function Call in apply_async()

Error 2: Using a Function Call in map()

Error 3: Incorrect Function Signature for map()

Error 4: Incorrect Function Signature for Callbacks

Error 5: Tasks Fail Silently

Error 6: Joining Pool While Running

Error 7: Issuing Tasks to a Closed Pool

Further Reading

Takeaways