How to Use Asyncio wait() in Python

After issuing many tasks on asyncio, we may need to wait for a specific condition to occur in the group.

For example, we may want to wait until all tasks are complete, or for the first task to complete or fail and know which task it was.

This can be achieved via the asyncio.wait() function.

In this tutorial, you will discover how to wait for asyncio tasks to complete.

After completing this tutorial, you will know:

How to use the wait() function to wait on a collection of asyncio tasks.
How to configure the condition waited for by the wait() function.
How to wait for a target condition with a timeout.

Let’s get started.

What is asyncio.wait()

The asyncio.wait() function can be used to wait for a collection of asyncio tasks to complete.

Recall that an asyncio task is an instance of the asyncio.Task class that wraps a coroutine. It allows a coroutine to be scheduled and executed independently, and the Task instance provides a handle on the task for querying status and getting results.

You can learn more about asyncio tasks in the tutorial:

What is an Asyncio Task

The wait() function allows us to wait for a collection of tasks to be done.

The call to wait can be configured to wait for different conditions, such as all tasks being completed, the first task completed and the first task failing with an error.

Next, let’s look at how we might use the wait() function.

The asyncio.wait() function takes a collection of awaitables, typically Task objects.

This could be a list, dict or set of task objects that we have created, such as via calls to the asyncio.create() task function in a list comprehension.

For example:

...

# create many tasks

tasks = [asyncio.create_task(task_coro(i)) for i in range(10)]

The asyncio.wait() will not return until some condition on the collection of tasks is met.

By default, the condition is that all tasks are completed.

The wait() function returns a tuple of two sets. The first set contains all task objects that meet the condition, and the second contains all other task objects that do not yet meet the condition.

These sets are referred to as the “done” set and the “pending” set.

For example:

...

# wait for all tasks to complete

done, pending = await asyncio.wait(tasks)

Technically, the asyncio.wait() is a coroutine function that returns a coroutine.

We can then await this coroutine which will return the tuple of sets.

For example:

...

# create the wait coroutine

wait_coro = asyncio.wait(tasks)

# await the wait coroutine

tuple = await wait_coro

The condition waited for can be specified by the “return_when” argument which is set to asyncio.ALL_COMPLETED by default.

For example:

...

# wait for all tasks to complete

done, pending = await asyncio.wait(tasks, return_when=asyncio.ALL_COMPLETED)

We can wait for the first task to be completed by setting return_when to FIRST_COMPLETED.

For example:

...

# wait for the first task to be completed

done, pending = await asyncio.wait(tasks, return_when=asyncio.FIRST_COMPLETED)

When the first task is complete and returned in the done set, the remaining tasks are not canceled and continue to execute concurrently.

We can wait for the first task to fail with an exception by setting return_when to FIRST_EXCEPTION.

For example:

...

# wait for the first task to fail

done, pending = await asyncio.wait(tasks, return_when=asyncio.FIRST_EXCEPTION)

In this case, the done set will contain the first task that failed with an exception. If no task fails with an exception, the done set will contain all tasks and wait() will return only after all tasks are completed.

We can specify how long we are willing to wait for the given condition via a “timeout” argument in seconds.

If the timeout expires before the condition is met, the tuple of tasks is returned with whatever subset of tasks do meet the condition at that time, e.g. the subset of tasks that are completed if waiting for all tasks to complete.

For example:

...

# wait for all tasks to complete with a timeout

done, pending = await asyncio.wait(tasks, timeout=3)

If the timeout is reached before the condition is met, an exception is not raised and the remaining tasks are not canceled.

Now that we know how to use the asyncio.wait() function, let’s look at some worked examples.

Example of Waiting for All Tasks

We can explore how to wait for all tasks using asyncio.wait().

In this example, we will define a simple task coroutine that generates a random value, sleeps for a fraction of a second, then reports a message with the generated value.

The main coroutine will then create many tasks in a list comprehension with the coroutine and then wait for all tasks to complete.

The complete example is listed below.

# SuperFastPython.com

# example of waiting for all tasks to complete

from random import random

import asyncio

# coroutine to execute in a new task

async def task_coro(arg):

# generate a random value between 0 and 1

value = random()

# block for a moment

await asyncio.sleep(value)

# report the value

print(f'>task {arg} done with {value}')

# main coroutine

async def main():

# create many tasks

tasks = [asyncio.create_task(task_coro(i)) for i in range(10)]

# wait for all tasks to complete

done,pending = await asyncio.wait(tasks)

# report results

print('All done')

# start the asyncio program

asyncio.run(main())

Running the example first creates the main() coroutine and uses it as the entry point into the asyncio program.

The main() coroutine then creates a list of ten tasks in a list comprehension, each providing a unique integer argument from 0 to 9.

The main() coroutine is then suspended and waits for all tasks to complete.

The tasks execute. Each generates a random value, sleeps for a moment, then reports its generated value.

After all tasks have been completed, the main() coroutine resumes and reports a final message.

This example highlights how we can use the wait() function to wait for a collection of tasks to be completed.

This is perhaps the most common usage of the function.

Note, that the results will differ each time the program is run given the use of random numbers.

>task 5 done with 0.0591009105682192

>task 8 done with 0.10453715687017351

>task 0 done with 0.15462838864295925

>task 6 done with 0.4103492027393125

>task 9 done with 0.45567100006991623

>task 2 done with 0.6984682905809402

>task 7 done with 0.7785363531316224

>task 3 done with 0.827386088873161

>task 4 done with 0.9481344994700972

>task 1 done with 0.9577302665040541

All done

Next, let’s look at how we might wait for the first task to complete.

Free Python Asyncio Course

Download your FREE Asyncio PDF cheat sheet and get BONUS access to my free 7-day crash course on the Asyncio API.

Discover how to use the Python asyncio module including how to define, create, and run new coroutines and how to use non-blocking I/O.

Learn more

Example of Waiting for First Task

We can explore how to wait for the first task to complete using asyncio.wait().

In this example, we will reuse the same simple task coroutine that generates a random value, sleeps for a fraction of a second, then reports a message with the generated value.

The main coroutine will then create many tasks in a list comprehension with the coroutine as before. It will then wait for the first task to complete.

Once a task is completed, it is retrieved from the set of done tasks and its details are reported.

The complete example is listed below.

# SuperFastPython.com

# example of waiting for the first task to complete

from random import random

import asyncio

# coroutine to execute in a new task

async def task_coro(arg):

# generate a random value between 0 and 1

value = random()

# block for a moment

await asyncio.sleep(value)

# report the value

print(f'>task {arg} done with {value}')

# main coroutine

async def main():

# create many tasks

tasks = [asyncio.create_task(task_coro(i)) for i in range(10)]

# wait for the first task to complete

done,pending = await asyncio.wait(tasks, return_when=asyncio.FIRST_COMPLETED)

# report result

print('Done')

# get the first task to complete

first = done.pop()

print(first)

# start the asyncio program

asyncio.run(main())

Running the example first creates the main() coroutine and uses it as the entry point into the asyncio program.

The main() coroutine then creates a list of ten tasks in a list comprehension, each providing a unique integer argument from 0 to 9.

The main() coroutine is then suspended and waits for the first task to complete

The tasks execute. Each generates a random value, sleeps for a moment, then reports its generated value.

As soon as the first task completes, the wait() function returns and the main() coroutine resumes.

The “done” set contains a single task that is finished, whereas the “pending” set contains all other tasks that were provided in the collection to the wait() function.

The single finished task is then retrieved from the “done” set and is reported. The printed status of the task confirms that indeed it is done.

The other tasks are not canceled and continue to run concurrently. Their execution is cut short because the asyncio program is terminated.

This example highlights how we can use the wait() function to wait for the first task to complete.

Note, that the results will differ each time the program is run given the use of random numbers.

>task 9 done with 0.04034360933451242

Done

<Task finished name='Task-11' coro=<task_coro() done, defined at ...> result=None>

Next, let’s look at how we might wait for the first task to fail with an exception.

Example of Waiting for First Failure

We can explore how to wait for the first task to fail using asyncio.wait().

In this example, we will reuse the same simple task coroutine that generates a random value, sleeps for a fraction of a second, then reports a message with the generated value.

The task coroutine is updated so that conditionally it will fail with an exception if the generated value is below a threshold.

The main coroutine will then create many tasks. It will then wait for the first task to fail with an exception.

Once a task has failed, it is retrieved from the set of done tasks and its details are reported.

The complete example is listed below.

# SuperFastPython.com

# example of waiting for the first task to fail

from random import random

import asyncio

# coroutine to execute in a new task

async def task_coro(arg):

# generate a random value between 0 and 1

value = random()

# block for a moment

await asyncio.sleep(value)

# report the value

print(f'>task {arg} done with {value}')

# conditionally fail

if value < 0.5:

raise Exception(f'Something bad happened in {arg}')

# main coroutine

async def main():

# create many tasks

tasks = [asyncio.create_task(task_coro(i)) for i in range(10)]

# wait for the first task to fail, or all tasks to complete

done,pending = await asyncio.wait(tasks, return_when=asyncio.FIRST_EXCEPTION)

# report result

print('Done')

# get the first task to fail

first = done.pop()

print(first)

# start the asyncio program

asyncio.run(main())

Running the example first creates the main() coroutine and uses it as the entry point into the asyncio program.

The main() coroutine then creates a list of ten tasks in a list comprehension, each providing a unique integer argument from 0 to 9.

The main() coroutine is then suspended and waits for the first task to complete

The tasks execute. Each generates a random value, sleeps for a moment, then reports its generated value.

Conditionally, some tasks raise an exception if their generated value is above a threshold value. It is possible that all tasks generate a value below the threshold and none raise an exception, although this situation is extremely unlikely.

As soon as the first task fails with an exception, the wait() function returns and the main() coroutine resumes.

The “done” set contains a single task that failed first, whereas the “pending” set contains all other tasks that were provided in the collection to the wait() function.

The single failed task is then retrieved from the “done” set and is reported. The printed status of the task confirms that it indeed failed with an exception in this case.

The other tasks are not canceled and continue to run concurrently. Their execution is cut short because the asyncio program is terminated.

This example highlights how we can use the wait() function to wait for the first task to fail.

Note, that the results will differ each time the program is run given the use of random numbers.

>task 5 done with 0.13168449673381888

Done

<Task finished name='Task-7' coro=<task_coro() done, defined at ...> exception=Exception('Something bad happened in 5')>

Next, let’s look at how we might wait for tasks with a timeout.

Loving The Tutorials?

Why not take the next step? Get the book.

Learn more

Example of Waiting with a Timeout

We can explore how to wait for all tasks to complete with a timeout using asyncio.wait().

In this example, we will reuse the same simple task coroutine that generates a random value, sleeps for a fraction of a second, then reports a message with the generated value.

The main coroutine will then create many tasks and wait for all tasks to complete. In this case, a timeout is specified, setting an upper limit on how long the caller is willing to wait.

Once all tasks are complete, or the timeout expires, the sets of done and pending tasks are returned. The total number of tasks completed within the timeout is then reported.

The complete example is listed below.

# SuperFastPython.com

# example of waiting for all tasks to be completed with a timeout

from random import random

import asyncio

# coroutine to execute in a new task

async def task_coro(arg):

# generate a random value between 0 and 10

value = random() * 10

# block for a moment

await asyncio.sleep(value)

# report the value

print(f'>task {arg} done with {value}')

# main coroutine

async def main():

# create many tasks

tasks = [asyncio.create_task(task_coro(i)) for i in range(10)]

# wait for all tasks to complete

done,pending = await asyncio.wait(tasks, timeout=5)

# report results

print(f'Done, {len(done)} tasks completed in time')

# start the asyncio program

asyncio.run(main())

Running the example first creates the main() coroutine and uses it as the entry point into the asyncio program.

The main() coroutine then creates a list of ten tasks in a list comprehension, each providing a unique integer argument from 0 to 9.

The main() coroutine is then suspended and waits for all tasks to complete.

It specifies a timeout of 5 seconds. This is about half the duration of the longest tasks that could be generated.

The tasks execute. Each generates a random value between 0 and 10, sleeps for the generated number of seconds, then reports its generated value.

The timeout expires and the main() coroutine resumes. It reports a message indicating the number of tasks that were completed within the timeout, which was 6 in this case.

This example highlights how we can use the wait() function to wait for a collection of tasks to be completed with a timeout.

Note, that the results will differ each time the program is run given the use of random numbers.

>task 7 done with 0.16485206249285955

>task 0 done with 0.73241529734688

>task 4 done with 1.1137310743743878

>task 6 done with 2.396915437441108

>task 5 done with 3.375537014759735

>task 2 done with 4.821848023696365

Done, 6 tasks completed in time

Example of Waiting for Coroutines

We can explore waiting for coroutines instead of tasks with asyncio.wait().

The wait() function is used to support coroutines as well as tasks. It was since updated to only support tasks.

Support for coroutines in the wait() function is deprecated at the time of writing and will result in a DeprecationWarning in Python 3.10. This support will soon be removed, in Python 3.11.

The example below creates a list of coroutines and then waits for them all to complete.

If you are using Python 3.10 or lower, the coroutines are converted into tasks and scheduled for execution.

The complete example is listed below.

# SuperFastPython.com

# example of waiting for all coroutines to complete

from random import random

import asyncio

# coroutine to execute in a new task

async def task_coro(arg):

# generate a random value between 0 and 1

value = random()

# block for a moment

await asyncio.sleep(value)

# report the value

print(f'>task {arg} done with {value}')

# main coroutine

async def main():

# create many coroutines

tasks = [task_coro(i) for i in range(10)]

# wait for all tasks to complete

done,pending = await asyncio.wait(tasks)

# report results

print('All done')

# start the asyncio program

asyncio.run(main())

Running the program assumes you are using Python 3.10 or lower. If not, please let me know the results you see in the comments below.

In this case, we can see that a DeprecationWarning is reported.

The example first creates the main() coroutine and uses it as the entry point into the asyncio program.

The main() coroutine then creates a list of ten coroutines (not tasks) in a list comprehension, each providing a unique integer argument from 0 to 9.

The main() coroutine is then suspended and waits for all coroutines to complete. The wait() function converts each coroutine into a Task, which is then scheduled for execution and then awaited.

The tasks execute. Each generates a random value, sleeps for a moment, then reports its generated value.

After all tasks have been completed, the main() coroutine resumes and reports a final message.

This example highlights how we can use the wait() function to wait for a collection of coroutines to complete.

Note, that the results will differ each time the program is run given the use of random numbers.

DeprecationWarning: The explicit passing of coroutine objects to asyncio.wait() is deprecated since Python 3.8, and scheduled for removal in Python 3.11.

done,pending = await asyncio.wait(tasks)

>task 3 done with 0.05716637980412176

>task 2 done with 0.21893288741084582

>task 9 done with 0.2702814255302467

>task 4 done with 0.2989980107703637

>task 6 done with 0.40462737710888264

>task 8 done with 0.5675796788798889

>task 5 done with 0.6219456587051952

>task 1 done with 0.8236981712898166

>task 7 done with 0.8408137669173977

>task 0 done with 0.9844164290735692

All done

Takeaways

You now know how to wait for asyncio tasks to complete in Python.

Do you have any questions?
Ask your questions in the comments below and I will do my best to answer.

Photo by Chris Stein on Unsplash

What is asyncio.wait()

Example of Waiting for All Tasks

Example of Waiting for First Task

Example of Waiting for First Failure

Example of Waiting with a Timeout

Example of Waiting for Coroutines

Further Reading

Takeaways