Add if __name__ == ‘__main__’ When Spawning Processes

You can fix a RuntimeError when starting child processes by adding if name == ‘main’ to checking for the top-level environment in your code.

In this tutorial you will discover the RuntimeError common on Windows and MacOS when creating child processes and how to fix it.

Let’s get started.

RuntimeError When Starting a Child Process

It is common to get a RuntimeError when starting a new Process in Python.

The content of the error often looks as follows:

An attempt has been made to start a new process before the

current process has finished its bootstrapping phase.

This probably means that you are not using fork to start your

child processes and you have forgotten to use the proper idiom

in the main module:

if __name__ == '__main__':

freeze_support()

...

The "freeze_support()" line can be omitted if the program

is not going to be frozen to produce an executable.

This will happen on Windows and MacOS where the default start method is ‘spawn‘. It may also happen when you configure your program to use the ‘spawn‘ start method on other platforms.

We can demonstrate this error with a worked example.

In this example we will configure a new child process to execute our custom task() function. We will then start the child process by calling the start() function.

The complete example is listed below.

# SuperFastPython.com

# example of missing __name__ == '__main__'

from multiprocessing import Process

# function executed in a new process

def task():

print('Hello from a child process', flush=True)

# create and configure a new process

process = Process(target=task)

# start the new process

process.start()

# wait for the new process to finish

process.join()

Running the example raises a RuntimeError on Windows and MacOS.

The RuntimeError is raised when we try to start the child process.

Specifically on the line:

A sample of the message reported by the RuntimeError is listed below.

Traceback (most recent call last):

...

RuntimeError:

An attempt has been made to start a new process before the

current process has finished its bootstrapping phase.

This probably means that you are not using fork to start your

child processes and you have forgotten to use the proper idiom

in the main module:

if __name__ == '__main__':

freeze_support()

...

The "freeze_support()" line can be omitted if the program

is not going to be frozen to produce an executable.

This is a common error and is easy to fix.

Fix RuntimeError When Starting New Process

The fix involves checking if the code is running in the top-level environment and only then, attempt to start a new process.

This is a best practice.

The idiom for this fix, as stated in the message of the RuntimeError, is to use an if-statement and check if the name of the module is equal to the string ‘__main__‘.

For example:

...

# check for top-level environment

if __name__ == '__main__':

# ...

This is called “protecting the entry point” of the program.

Recall, that __name__ is a variable that refers to the name of the module executing the current code.

When a Python module or package is imported, __name__ is set to the module’s name. Usually, this is the name of the Python file itself without the .py extension:
— __main__ — Top-level code environment

Also, recall that ‘__main__‘ is the name of the top-level environment used to execute a Python program.

__main__ is the name of the environment where top-level code is run. “Top-level code” is the first user-specified Python module that starts running. It’s “top-level” because it imports all other modules that the program needs.
— __main__ — Top-level code environment

Using an if-statement to check if the module is the top-level environment and only starting child processes within that block will resolve the RuntimeError.

It means that if the Python file is imported, then the code protected by the if-statement will not run. It will only run when the Python file is run directly, e.g. is the top-level environment.

Some modules contain code that is intended for script use only, like parsing command-line arguments or fetching data from standard input. If a module like this was imported from a different module, for example to unit test it, the script code would unintentionally execute as well. This is where using the if __name__ == ‘__main__’ code block comes in handy. Code within this block won’t run unless the module is executed in the top-level environment.
— __main__ — Top-level code environment

The if-statement idiom is required, even if the entry point of the program calls a function that itself starts a child process.

This is only used for child processes created by the main process. Child processes themselves can create child processes without this check.

Why Do We Need to Protect Entry Point?

We must protect the entry point when starting new processes from the main process using the ‘spawn‘ method.

The reason is because our Python script will be loaded and imported automatically for us by child processes.

This is required to execute our custom code and functions in new child processes.

If the entry point was not protected with an if-statement idiom checking for the top-level environment, then the script would execute again directly, rather than run a new child process as expected.

Protecting the entry point ensures that the program is only started once, that the tasks of the main process are only executed by the main process and not the child processes.

Free Python Multiprocessing Course

Download your FREE multiprocessing PDF cheat sheet and get BONUS access to my free 7-day crash course on the multiprocessing API.

Discover how to use the Python multiprocessing module including how to create and start child processes and how to use a mutex locks and semaphores.

Learn more

Example of Fix By Protecting Entry Point

We can demonstrate how to check for the top-level environment when starting a new process.

This can be achieved by updating the example above that resulted in a RuntimeError and adding a check if __name__ is equal to ‘__main__’.

For example:

# check for top-level environment

if __name__ == '__main__':

# create and configure a new process

process = Process(target=task)

# start the new process

process.start()

# wait for the new process to finish

process.join()

Tying this together, the complete example is listed below.

# SuperFastPython.com

# example of starting a child process only in __main__

from multiprocessing import Process

# function executed in a new process

def task():

print('Hello from a child process', flush=True)

# check for top-level environment

if __name__ == '__main__':

# create and configure a new process

process = Process(target=task)

# start the new process

process.start()

# wait for the new process to finish

process.join()

Running the example successfully starts a new child process.

The message is reported from the child process as expected.

Hello from a child process

Example of Fix by Changing Start Method

We can also fix the issue by changing the start method from ‘spawn‘ to ‘fork‘.

This fix only works on those platforms where the ‘fork’ method is supported, e.g. Unix and MacOS.

This can be achieved via the multiprocessing.set_start_method() function.

For example:

# SuperFastPython.com

# example of missing __name__ == '__main__'

from multiprocessing import set_start_method

from multiprocessing import Process

# function executed in a new process

def task():

print('Hello from a child process', flush=True)

# set the start method

set_start_method('fork')

# create and configure a new process

process = Process(target=task)

# start the new process

process.start()

# wait for the new process to finish

process.join()

Running the example successfully starts a new child process.

The message is reported from the child process as expected.

Hello from a child process

Loving The Tutorials?

Why not take the next step? Get the book.

Learn more

Takeaways

You now know how to fix the RuntimeError caused by starting child processes with the ‘spawn‘ start method.

Do you have any questions?
Ask your questions in the comments below and I will do my best to answer.

Photo by Ivan Ragozin on Unsplash

RuntimeError When Starting a Child Process

Fix RuntimeError When Starting New Process

Why Do We Need to Protect Entry Point?

Example of Fix By Protecting Entry Point

Example of Fix by Changing Start Method

Further Reading

Takeaways