How to Handle Asyncio Task Exceptions

An asyncio task may fail with an unhandled exception.

The exception will unwind the task, although may not impact other tasks or the broader asyncio program.

As such, we need a way of checking if a task has failed and of retrieving any unhandled exceptions in the task, if they occurred.

In this tutorial, you will discover exception handling in asyncio tasks.

After completing this tutorial, you will know:

How to check if a task failed due to an unhandled exception.
How to retrieve the exception from a task and what happens if we get the exception while the task is running.
How and when the exception in the task may be propagated to another coroutine or task.

Let’s get started.

What is an Asyncio Task

An asyncio Task is an object that schedules and independently runs an asyncio coroutine.

It provides a handle on a scheduled coroutine that an asyncio program can query and use to interact with the coroutine.

A Task is an object that manages an independently running coroutine.
— PEP 3156 – Asynchronous IO Support Rebooted: the “asyncio” Module

An asyncio task is represented via an instance of the asyncio.Task class.

A task is created from a coroutine. It requires a coroutine object, wraps the coroutine, schedules it for execution, and provides ways to interact with it.

A task is executed independently. This means it is scheduled in the asyncio event loop and will execute regardless of what else happens in the coroutine that created it. This is different from executing a coroutine directly, where the caller must wait for it to complete.

Tasks are used to schedule coroutines concurrently. When a coroutine is wrapped into a Task with functions like asyncio.create_task() the coroutine is automatically scheduled to run soon
— Coroutines and Tasks

We can create a task using the asyncio.create_task() function.

This function takes a coroutine instance and an optional name for the task and returns an asyncio.Task instance.

Wrap the coro coroutine into a Task and schedule its execution. Return the Task object.
— Coroutines and Tasks

For example:

...

# create and schedule a task

task = asyncio.create_task(coro)

You can learn more about asyncio tasks in the tutorial:

What is an Asyncio Task

Now that we know about asyncio tasks, let’s look at how we might handle and check for exceptions.

How to Check for Exceptions in Tasks

A coroutine wrapped by a task may raise an exception that is not handled.

This will fail the task, in effect.

We can retrieve an unhandled exception in the coroutine wrapped by a task via the exception() method.

For example:

...

# get the exception raised by a task

exception = task.exception()

If an unhandled exception was not raised in the wrapped coroutine, then a value of None is returned.

If the task was canceled, then a CancelledError exception is raised when calling the exception() method and may need to be handled.

For example:

...

try:

# get the exception raised by a task

exception = task.exception()

except asyncio.CancelledError:

# task was canceled

As such, it is a good idea to check if the task was canceled first.

For example:

...

# check if the task was not canceled

if not task.cancelled():

# get the exception raised by a task

exception = task.exception()

else:

# task was canceled

If the task is not yet done, then an InvalidStateError exception is raised when calling the exception() method and may need to be handled.

For example:

...

try:

# get the exception raised by a task

exception = task.exception()

except asyncio.InvalidStateError:

# task is not yet done

As such, it is a good idea to check if the task is done first.

For example:

...

# check if the task is not done

if not task.done():

await task

# get the exception raised by a task

exception = task.exception()

Next, let’s look at when an unhandled exception in a task is propagated to the caller.

When Are Task Exceptions Propagated to the Caller

Exceptions that occur within a task can be propagated to the caller.

This can happen in two situations, they are:

When the caller awaits the task.
When the caller gets the result from the task.

When a coroutine awaits a task that raises an unhandled exception, the exception is propagated to the caller.

For example:

...

# wait for the task to finish

await task

Therefore, if an unhandled exception is possible in a Task’s coroutine, it may need to be handled when awaiting the task.

For example:

...

try:

# wait for the task to finish

await task

except Exception as e:

# ...

Similarly, if the task is done and the caller tempts to retrieve the return value from the task via the result() method, any unhandled exceptions are propagated.

For example:

...

# get the return value from the task

value = task.result()

Therefore, if an unhandled exception is possible in a Task‘s coroutine, it may need to be handled when awaiting the task.

For example:

...

try:

# get the return value from the task

value = task.result()

except Exception as e:

# ...

Now that we know when exceptions in tasks are propagated, let’s look at some worked examples of checking for and handling exceptions in tasks.

Free Python Asyncio Course

Download your FREE Asyncio PDF cheat sheet and get BONUS access to my free 7-day crash course on the Asyncio API.

Discover how to use the Python asyncio module including how to define, create, and run new coroutines and how to use non-blocking I/O.

Learn more

Example of Checking for an Exception in a Done Task

We can explore how to check for and get an exception from a successfully done task.

That is, check for an exception in a task that does not raise an exception.

The expectation is that the exception() method will return None after the task is done.

The complete example is listed below.

# SuperFastPython.com

# example of getting an exception from a successful task

import asyncio

# define a coroutine for a task

async def task_coroutine():

# report a message

print('executing the task')

# block for a moment

await asyncio.sleep(1)

# custom coroutine

async def main():

# report a message

print('main coroutine started')

# create and schedule the task

task = asyncio.create_task(task_coroutine())

# wait for the task to complete

await task

# get the exception

ex = task.exception()

print(f'exception: {ex}')

# report a final message

print('main coroutine done')

# start the asyncio program

asyncio.run(main())

Running the example starts the asyncio event loop and executes the main() coroutine.

The main() coroutine reports a message, then creates and schedules the task coroutine.

It then suspends and awaits the task to be completed.

The task runs, reports a message, and sleeps for a moment before terminating normally.

The main() coroutine resumes and retrieves an exception from the task.

The task did not raise an unhandled exception, so the exception() method returns None.

This example highlights that a successful task will return None if an unhandled exception was not raised. This could be checked for, e.g. checking to see if a task failed or not.

main coroutine started

executing the task

exception: None

main coroutine done

Next, we can look at an example of retrieving an exception from a failed task.

Example of Checking for an Exception in a Failed Task

We can explore getting an exception from a task that failed with an unhandled exception.

This is the exact use case for the exception() method.

In this example, we can update the task coroutine to explicitly raise an exception that is not handled.

This will cause the task coroutine to fail.

The main coroutine will sleep to wait for the task to be completed. This is to avoid using the await expression which will propagate the exception back to the caller.

Once the task is done, the main coroutine will retrieve and report the exception raised in the task.

The complete example is listed below.

# SuperFastPython.com

# example of getting an exception from a failed task

import asyncio

# define a coroutine for a task

async def task_coroutine():

# report a message

print('executing the task')

# block for a moment

await asyncio.sleep(1)

# raise an exception

raise Exception('Something bad happened')

# custom coroutine

async def main():

# report a message

print('main coroutine started')

# create and schedule the task

task = asyncio.create_task(task_coroutine())

# wait for the task to complete

await asyncio.sleep(1.1)

# get the exception

ex = task.exception()

print(f'exception: {ex}')

# report a final message

print('main coroutine done')

# start the asyncio program

asyncio.run(main())

Running the example starts the asyncio event loop and executes the main() coroutine.

The main() coroutine reports a message, then creates and schedules the task coroutine.

It then suspends and awaits the task to be completed.

The task runs, reports a message, and sleeps for a moment. The task resumes and raises an exception.

The exception does not terminate the application or the asyncio event loop.

Instead, the exception is captured by the asyncio event loop and stored in the task.

The main() coroutine resumes and then retrieves the exception from the task, which is reported.

main coroutine started

executing the task

exception: Something bad happened

main coroutine done

Next, let’s look at what happens if we try to retrieve an exception from a running task.

Loving The Tutorials?

Why not take the next step? Get the book.

Learn more

Example of Checking for an Exception in a Running Task

We cannot retrieve an exception from a running asyncio task.

Instead, we can only retrieve the exception from a task after it is done.

If we call the exception() method on a task that is scheduled or running, an InvalidStateError exception is raised in the caller.

The example below demonstrates this.

# SuperFastPython.com

# example of getting an exception from a running task

import asyncio

# define a coroutine for a task

async def task_coroutine():

# report a message

print('executing the task')

# block for a moment

await asyncio.sleep(1)

# custom coroutine

async def main():

# report a message

print('main coroutine started')

# create and schedule the task

task = asyncio.create_task(task_coroutine())

# wait a moment

await asyncio.sleep(0.2)

# get the exception

ex = task.exception()

print(f'exception: {ex}')

# report a final message

print('main coroutine done')

# start the asyncio program

asyncio.run(main())

Running the example starts the asyncio event loop and executes the main() coroutine.

The main() coroutine reports a message, then creates and schedules the task coroutine.

It then suspends and sleeps for a moment.

The task runs, reports a message, and sleeps for a moment.

The main() coroutine resumes and attempts to retrieve the exception from the task while the task is running, even though the task is suspended.

This fails with an InvalidStateError that breaks the asyncio event loop in this case.

This example highlights that we must always retrieve a Task exception after the task is done.

main coroutine started

executing the task

Traceback (most recent call last):

...

asyncio.exceptions.InvalidStateError: Exception is not set.

We can check if a task is done before retrieving the exception via the done() method that will return True if the task is done, or False otherwise.

Next, we can look at the case of attempting to get a task exception for a canceled task.

Example of Checking for an Exception in a Canceled Task

We cannot retrieve an exception from a canceled task.

Although a canceled task is done, an exception will not be available and cannot be retrieved.

Instead, a CancelledError exception is raised when calling the exception() method if the task was canceled.

We can demonstrate this with a worked example.

# SuperFastPython.com

# example of getting an exception from a running task

import asyncio

# define a coroutine for a task

async def task_coroutine():

# report a message

print('executing the task')

# block for a moment

await asyncio.sleep(1)

# custom coroutine

async def main():

# report a message

print('main coroutine started')

# create and schedule the task

task = asyncio.create_task(task_coroutine())

# wait a moment

await asyncio.sleep(0.1)

# cancel the task

task.cancel()

# wait a moment

await asyncio.sleep(0.1)

# get the exception

ex = task.exception()

print(f'exception: {ex}')

# report a final message

print('main coroutine done')

# start the asyncio program

asyncio.run(main())

Running the example starts the asyncio event loop and executes the main() coroutine.

The main() coroutine reports a message, then creates and schedules the task coroutine.

It then suspends and sleeps for a moment.

The main() coroutine resumes and cancels the task. It then suspends and waits a moment for the task to respond to the request for being canceled.

The task is canceled by raising a CancelledError within the wrapped coroutine.

The main() coroutine resumes and attempts to retrieve an exception.

This fails and the CancelledError exception is re-raised in the caller.

This breaks the event loop in this case.

main coroutine started

executing the task

Traceback (most recent call last):

...

asyncio.exceptions.CancelledError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

File "...", line 25, in main

ex = task.exception()

asyncio.exceptions.CancelledError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

...

asyncio.exceptions.CancelledError

Next, let’s look at how we might handle an exception propagated by awaiting a task.

Example of Handling a Task Exception with await

Awaiting a task that fails with an exception will cause the exception to be propagated to the caller.

As such, awaiting a task may require that the unhandled but possible exceptions be handled.

The example below demonstrates this with a task that fails with an exception that is awaited in a main coroutine that expects and then handles the exception

# SuperFastPython.com

# example of handling a task exception when await

import asyncio

# define a coroutine for a task

async def task_coroutine():

# report a message

print('executing the task')

# block for a moment

await asyncio.sleep(1)

# fail with an exception

raise Exception('Something bad happened')

# custom coroutine

async def main():

# report a message

print('main coroutine started')

# create and schedule the task

task = asyncio.create_task(task_coroutine())

try:

# wait for the task to complete

await task

except Exception as e:

print(f'Failed with: {e}')

# report a final message

print('main coroutine done')

# start the asyncio program

asyncio.run(main())

Running the example starts the asyncio event loop and executes the main() coroutine.

The main() coroutine reports a message, then creates and schedules the task coroutine.

It then suspends and awaits the task to be completed. Importantly, the main() coroutine awaits the task within a try-except block.

The task runs, reports a message and sleeps for a moment, and then fails with an exception.

The main() coroutine resumes and handles the exception that was raised in the wrapped coroutine. The exception is propagated to the caller, caught, and the details are reported.

This highlights that we may need to handle unhandled exceptions because they can be propagated back to any coroutines waiting on the task.

main coroutine started

executing the task

Failed with: Something bad happened

main coroutine done

Next, we will look at how to handle task exceptions propagated to the caller when getting task results.

Example of Handling a Task Exception with result()

We can get the return value from a task via the result() method.

Care must be taken with this method because any exception that was raised in the Task‘s coroutine that was not handled will be propagated back and re-raised in the caller.

We can demonstrate this with a worked example.

The task coroutine returns a value, but the line is never reached because it fails with an exception.

The main coroutine attempts to retrieve the result from the task and handles the exception that may be raised and propagated.

# SuperFastPython.com

# example of handling a task exception when getting the result

import asyncio

# define a coroutine for a task

async def task_coroutine():

# report a message

print('executing the task')

# block for a moment

await asyncio.sleep(1)

# fail with an exception

raise Exception('Something bad happened')

# return a value (never reached)

return 100

# custom coroutine

async def main():

# report a message

print('main coroutine started')

# create and schedule the task

task = asyncio.create_task(task_coroutine())

# wait for the task to complete

await asyncio.sleep(1.1)

try:

# get the result

value = task.result()

except Exception as e:

print(f'Failed with: {e}')

# report a final message

print('main coroutine done')

# start the asyncio program

asyncio.run(main())

Running the example starts the asyncio event loop and executes the main() coroutine.

The main() coroutine reports a message, then creates and schedules the task coroutine.

It then suspends and sleeps a moment to allow the task to be completed.

The task runs, reports a message and sleeps for a moment, and then fails with an exception.

The main() coroutine resumes and attempts to retrieve the return value from the task.

This fails and the unhandled exception raised in the task’s coroutine is re-raised in the caller.

The main() coroutine handles the exception, catching it and reporting the details.

This highlights that we may need to handle unhandled exceptions when getting task results because they can be propagated back to any coroutines waiting on the task.

main coroutine started

executing the task

Failed with: Something bad happened

main coroutine done

Takeaways

You now know how to handle exceptions in asyncio tasks in Python.

Do you have any questions?
Ask your questions in the comments below and I will do my best to answer.

Photo by William Chiesurin on Unsplash

What is an Asyncio Task

How to Check for Exceptions in Tasks

When Are Task Exceptions Propagated to the Caller

Example of Checking for an Exception in a Done Task

Example of Checking for an Exception in a Failed Task

Example of Checking for an Exception in a Running Task

Example of Checking for an Exception in a Canceled Task

Example of Handling a Task Exception with await

Example of Handling a Task Exception with result()

Further Reading

Takeaways