You can use a shared Event with Processes via the multiprocessing.Event class.
In this tutorial you will discover how to use an event object with processes in Python.
Let’s get started.
Need for an Event Object
A process is a running instance of a computer program.
Every Python program is executed in a Process, which is a new instance of the Python interpreter. This process has the name MainProcess and has one thread used to execute the program instructions called the MainThread. Both processes and threads are created and managed by the underlying operating system.
Sometimes we may need to create new child processes in our program in order to execute code concurrently.
Python provides the ability to create and manage new processes via the multiprocessing.Process class.
You can learn more about multiprocessing in the tutorial:
In concurrent programming, sometimes we need to coordinate processes with a boolean variable. This might be to trigger an action or signal some result.
This could be achieved with a mutual exclusion lock (mutex) and a boolean variable, but provides no way for processes to wait for the variable to be set True.
Instead, this can be achieved using an event object.
What is an event object and how can we use it with processes in Python?
How to Use an Event Object
Python provides an event object for processes via the multiprocessing.Event class.
An event is a simple concurrency primitive that allows communication between processes.
A multiprocessing.Event object wraps a boolean variable that can either be “set” (True) or “not set” (False). Processes sharing the event instance can check if the event is set, set the event, clear the event (make it not set), or wait for the event to be set.
The multiprocessing.Event provides an easy way to share a boolean variable between processes that can act as a trigger for an action.
First, an event object must be created and the event will be in the “not set” state.
... # create an instance of an event event = multiprocessing.Event() |
Once created we can check if the event has been set via the is_set() function which will return True if the event is set, or False otherwise.
For example:
... # check if the event is set if event.is_set(): # do something... |
The event can be set via the set() function. Any process waiting on the event to be set will be notified.
For example:
... # set the event event.set() |
The event can be marked as “not set” (whether it is currently set or not) via the clear() function.
... # mark the event as not set event.clear() |
Finally, processes can wait for the event to set via the wait() function. Calling this function will block until the event is marked as set (e.g. another process calling the set() function). If the event is already set, the wait() function will return immediately.
... # wait for the event to be set event.wait() |
A “timeout” argument can be passed to the wait() function which will limit how long a process is willing to wait in seconds for the event to be marked as set.
The wait() function will return True if the event was set while waiting, otherwise a value of False returned indicates that the event was not set and the call timed-out.
... # wait for the event to be set with a timeout event.wait(timeout=10) |
Event objects can also be used with threads, you can learn more about this in the tutorial:
Now that we know how to use a multiprocessing.Event, let’s look at a worked example.
Example of Using a Shared Event with Processes
We can explore how to use a multiprocessing.Event object.
In this example we will create a suite of processes that each will perform some processing and report a message. All processes will use an event to wait to be set before starting their work. The main process will set the event and trigger the child processes to start work.
First, we can define a target task function that takes the shared multiprocessing.Event instance and a unique integer to identify the process.
# target task function def task(event, number): # ... |
Next, the function will wait for the event to be set before starting the processing work.
... # wait for the event to be set print(f'Process {number} waiting...', flush=True) event.wait() |
Once triggered, the process will generate a random number, block for a moment and report a message.
... # begin processing value = random() sleep(value) print(f'Process {number} got {value}', flush=True) |
Tying this together, the complete target task function is listed below.
# target task function def task(event, number): # wait for the event to be set print(f'Process {number} waiting...', flush=True) event.wait() # begin processing value = random() sleep(value) print(f'Process {number} got {value}', flush=True) |
The main process will first create the shared multiprocessing.Event instance, which will be in the “not set” state by default.
... # create a shared event object event = Event() |
Next, we can create and configure five new processes specifying the target task() function with the event object and a unique integer as arguments.
This can be achieved in a list comprehension.
... # create a suite of processes processes = [Process(target=task, args=(event, i)) for i in range(5)] |
We can then start all child processes.
... # start all processes for process in processes: process.start() |
Next, the main process will block for a moment, then trigger the processing in all of the child processes via the event object.
... # block for a moment print('Main process blocking...') sleep(2) # trigger all child processes event.set() |
The main process will then wait for all child processes to terminate.
... # wait for all child processes to terminate for process in processes: process.join() |
Tying this all together, the complete example is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 |
# SuperFastPython.com # example of using an event object with processes from time import sleep from random import random from multiprocessing import Process from multiprocessing import Event # target task function def task(event, number): # wait for the event to be set print(f'Process {number} waiting...', flush=True) event.wait() # begin processing value = random() sleep(value) print(f'Process {number} got {value}', flush=True) # entry point if __name__ == '__main__': # create a shared event object event = Event() # create a suite of processes processes = [Process(target=task, args=(event, i)) for i in range(5)] # start all processes for process in processes: process.start() # block for a moment print('Main process blocking...') sleep(2) # trigger all child processes event.set() # wait for all child processes to terminate for process in processes: process.join() |
Running the example first creates and starts five child processes.
Each child process waits on the event before it starts its work, reporting a message that it is waiting.
The main process blocks for a moment, allowing all child processes to begin and start waiting on the event.
The main process then sets the event. This triggers all five child processes that perform their simulated work and report a message.
Note, your specific results will differ given the use of random numbers.
Main process blocking... Process 0 waiting... Process 1 waiting... Process 2 waiting... Process 3 waiting... Process 4 waiting... Process 0 got 0.06198821143561384 Process 4 got 0.219334069761699 Process 3 got 0.7335552378594119 Process 1 got 0.7948771419640999 Process 2 got 0.8713839353896263 |
Free Python Multiprocessing Course
Download your FREE multiprocessing PDF cheat sheet and get BONUS access to my free 7-day crash course on the multiprocessing API.
Discover how to use the Python multiprocessing module including how to create and start child processes and how to use a mutex locks and semaphores.
Further Reading
This section provides additional resources that you may find helpful.
Python Multiprocessing Books
- Python Multiprocessing Jump-Start, Jason Brownlee (my book!)
- Multiprocessing API Interview Questions
- Multiprocessing API Cheat Sheet
I would also recommend specific chapters in the books:
- Effective Python, Brett Slatkin, 2019.
- See: Chapter 7: Concurrency and Parallelism
- High Performance Python, Ian Ozsvald and Micha Gorelick, 2020.
- See: Chapter 9: The multiprocessing Module
- Python in a Nutshell, Alex Martelli, et al., 2017.
- See: Chapter: 14: Threads and Processes
Guides
- Python Multiprocessing: The Complete Guide
- Python Multiprocessing Pool: The Complete Guide
- Python ProcessPoolExecutor: The Complete Guide
APIs
References
Takeaways
You now know how to use a multiprocessing.Event Object in Python
Do you have any questions?
Ask your questions in the comments below and I will do my best to answer.
Photo by Alina Kacharho on Unsplash