GetProcAddress and GetModuleHandle do block in the same way as LoadLibrary and FreeLibrary - they acquire the loader lock too.

Yes, ideally the application would terminate its threads cleanly, however when Python is embedded in another application it may not have control on when or how the threads are terminated (see original problem description).

I would be surprised if there is any code that assumes the GIL is acquired during any of those functions. When going though the code I found that sometimes the GIL was acquired when calling one of those four functions and at other times it wasn't.

See https://docs.microsoft.com/en-us/dotnet/framework/debug-trace-profile/loaderlock-mda for a (possibly incomplete) list of functions that acquire the loader lock.