faster shut down by snnn · Pull Request #24891 · microsoft/onnxruntime

github-actions[bot]

bot reviewed May 28, 2025

github-actions[bot]

github-actions[bot]

@snnn snnn mentioned this pull request

May 29, 2025

Changming Sun added 4 commits

May 29, 2025 20:50

github-actions[bot]

github-actions[bot]

Changming Sun added 2 commits

May 31, 2025 01:49

github-actions[bot]

Changming Sun added 4 commits

May 30, 2025 19:47

@snnn snnn marked this pull request as ready for review

May 31, 2025 03:17

Changming Sun added 2 commits

May 30, 2025 20:42

adrianlizarraga

skottmckay

github-actions[bot]

Changming Sun added 2 commits

June 3, 2025 20:06

Changming Sun added 4 commits

June 4, 2025 15:39

github-actions[bot]

Changming Sun and others added 3 commits

June 4, 2025 17:51
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

Changming Sun added 2 commits

June 4, 2025 18:43

@snnn snnn marked this pull request as draft

June 5, 2025 04:39

Changming Sun added 2 commits

June 4, 2025 21:44

@snnn snnn marked this pull request as ready for review

June 5, 2025 23:00

skottmckay

@snnn snnn deleted the snnn/shutdown branch

June 6, 2025 02:41

khoover added a commit to khoover/onnxruntime that referenced this pull request

Jun 9, 2025

javier-intel pushed a commit to intel/onnxruntime that referenced this pull request

Jun 15, 2025
This PR is to reduce the chance of having crashes when a process is
shutting down. The main idea is: if we know the process is shutting
down(or if we know the ORT DLL won't be reloaded), we do not need to run
C++ object destructors. The approach is recommended by Windows SDK's
official document and many Windows developers. For example, 18 years ago
Raymond Chen wrote a blog [The old-fashioned theory on how processes
exit](https://devblogs.microsoft.com/oldnewthing/20070502-00/?p=27023).
This is ORT's current behavior. Raymond Chen also wrote a blog [what a
better approach
is](https://devblogs.microsoft.com/oldnewthing/20120105-00/?p=8683)

In our case, when onnxruntime is built as a python package, the
DLL(onnxruntime_pybind11_state.pyd ) will never be manually unloaded.
Same on Linux. Python does not unload the DLLs on exit. Therefore, we do
not need to worry about the potential memory leaks caused by any global
variables. Therefore, we do not need to call OrtEnv's destructors, and we
do not need to unload any EP dlls.

In most cases, people do not unload DLLs on Windows. And, on Linux it is
even more complicated because GCC needs to maintain a unique table to
avoid odr violations, and this feature makes most C++ shared library
cannot be unloaded.

So, this change detects if the os is Windows and if the process is
shutdown when calling destructors. If yes, the destructor will do
nothing.

After this change on Windows in most cases OrtEnv will not be destroyed.
The only exception is: if someone manually load the DLL and manually
unload the DLL, and also do not have a global threadpool. Then I think
the user is an advanced user and they should know that they need to
destroy all inference session objects and the OrtEnv singleton object
before unloading the DLL. Besides, if they have enabled global thread
pool, the DLL won't be unloaded if they haven't shutdown the thread pool
and delete the OrtEnv object. And, even if the user has manually
loaded/unloaded the DLL, there would still be some memory leaks(that are not
related to this change). It's hard to get 100% clean.

quic-ankus pushed a commit to CodeLinaro/onnxruntime that referenced this pull request

Nov 25, 2025
This PR is to reduce the chance of having crashes when a process is
shutting down. The main idea is: if we know the process is shutting
down(or if we know the ORT DLL won't be reloaded), we do not need to run
C++ object destructors. The approach is recommended by Windows SDK's
official document and many Windows developers. For example, 18 years ago
Raymond Chen wrote a blog [The old-fashioned theory on how processes
exit](https://devblogs.microsoft.com/oldnewthing/20070502-00/?p=27023).
This is ORT's current behavior. Raymond Chen also wrote a blog [what a
better approach
is](https://devblogs.microsoft.com/oldnewthing/20120105-00/?p=8683)

In our case, when onnxruntime is built as a python package, the
DLL(onnxruntime_pybind11_state.pyd ) will never be manually unloaded.
Same on Linux. Python does not unload the DLLs on exit. Therefore, we do
not need to worry about the potential memory leaks caused by any global
variables. Therefore, we do not need to call OrtEnv's destructors, and we
do not need to unload any EP dlls.

In most cases, people do not unload DLLs on Windows. And, on Linux it is
even more complicated because GCC needs to maintain a unique table to
avoid odr violations, and this feature makes most C++ shared library
cannot be unloaded.

So, this change detects if the os is Windows and if the process is
shutdown when calling destructors. If yes, the destructor will do
nothing.

After this change on Windows in most cases OrtEnv will not be destroyed.
The only exception is: if someone manually load the DLL and manually
unload the DLL, and also do not have a global threadpool. Then I think
the user is an advanced user and they should know that they need to
destroy all inference session objects and the OrtEnv singleton object
before unloading the DLL. Besides, if they have enabled global thread
pool, the DLL won't be unloaded if they haven't shutdown the thread pool
and delete the OrtEnv object. And, even if the user has manually
loaded/unloaded the DLL, there would still be some memory leaks(that are not
related to this change). It's hard to get 100% clean.