The following thread program will cause Python3.10 parser "core dump" due to missing “libgcc_s.so.1”. "pthread_cancel" cannot work correctly. I am wondering is there any possible to install or link to libgcc_s.so.1 during Python installation?

===================================
import socket
from threading import Thread
class thread(Thread):
      def run(self):
          for res in socket.getaddrinfo('www.google.com',8080):
              sock = socket.socket()
              sock.connect(res[4])

for i in range(2000):
    thread().start()
====================================
Error message:
---------------------------------------------------------------
........
Exception in thread Thread-1346:
Traceback (most recent call last):
  File "/usr/local/python310/lib/python3.10/threading.py", line 960, in _bootstrap_inner
  File "/usr/local/python310/lib/python3.10/threading.py", line 960, in _bootstrap_inner
Exception in thread Thread-1172:
Traceback (most recent call last):
.......
Exception in thread Thread-1509:
Exception in thread Thread-1510:
libgcc_s.so.1 must be installed for pthread_cancel to work
Exception in thread Thread-1511:
Traceback (most recent call last):
Aborted (core dumped)
---------------------------------------------------------------

What's your platform and distribution?

>>python310 -V
Python 3.10.0a2

>>uname -v
#73~16.04.1-Ubuntu SMP Fri Sep 13 09:56:18 UTC 2019

Thank you!

Please provide more information. What Linux distribution and distro version are you using? How did you install Python?

>>uname -a
Linux xxm-System-Product-Name 4.15.0-64-generic #73~16.04.1-Ubuntu SMP Fri Sep 13 09:56:18 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

>>uname -r
4.15.0-64-generic


>>lsb_release -a
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 16.04.3 LTS
Release:	16.04
Codename:	xenial

I download Python from https://www.python.org/downloads/. Then I extract it,run install command ./configure,make,sudo make install. I also try the versions of Python 3.5-3.10 and even cPython on GitHub. The results are the same. But when I try this program on Python 2, the program will not cause core dump. I think Python 2 is initially installed with the system, not by me. So I guess no link is referred to libgcc_s.so.1 when I install new version of Python.

It might be a packaging or documentation issue. I'm assiging the ticket to Matthias. He is the Debian and Ubuntu package maintainer.

I've encountered this issue too. My use case was a 32-bit Python on a 64-bit CentOS system, and my understanding of the issue was that 64-bit libgcc_s is somehow counted as a "provider" of libgcc_s for 32-bit libc by the package manager, so 32-bit libgcc_s is never installed even when "yum install glibc.i686" is used because everything seems "already OK":

$ yum deplist glibc

...
  dependency: libgcc
   provider: libgcc.x86_64 4.1.2-55.el5
   provider: libgcc.i386 4.1.2-55.el5
...

(The above is for CentOS 5, but at the time I tested 6 and 7 as well, with the same result).

But I suggest not to dismiss this issue as a packaging bug. Glibc needs libgcc_s for pthread_exit() because it's implemented in terms of pthread_cancel(), and the latter wants to do stack unwinding (probably to cleanup resources for each stack frame). But the code for unwinding lives in libgcc_s. The non-intuitive thing here is that glibc tried to optimize this dependency by dlopen'ing libgcc_s when pthread_cancel() is called instead of making it a startup dependency. Many people consider this a terrible design choice since your program can now fail at an arbitrary moment due to dlopen() failure (and missing libgcc_s is not the only possible reason[1]).

Since CPython doesn't use pthread_cancel() directly, I propose a simple solution  -- just `return NULL` from thread functions instead of calling pthread_exit(). The last time I looked, pthread_exit() in CPython is only called from top frames of the threads, so a direct `return` should suffice. If the top-level thread function simply returns, no stack unwinding is needed, so glibc never uses its cancellation machinery.

I've tested that this solution worked at the time (for 3.8 I think), and the test suite passed. If there is interest in going this way, I can test again.

[1] https://www.sourceware.org/bugzilla/show_bug.cgi?id=13119

I've made a PR to remove most calls to pthread_exit().

@xxm: could you test it in your environment?

No crash is reported anymore. The result is like following:
-----------------------------------------
....... 
OSError: [Errno 24] Too many open files
OSError: [Errno 24] Too many open files
OSError: [Errno 24] Too many open files
OSError: [Errno 24] Too many open files
OSError: [Errno 24] Too many open files
  File "/home/xxm/Downloads/cpython-a4afb55fb226e1debcdaaf80890b790198ba14cc/Lib/socket.py", line 953, in getaddrinfo
  File "/home/xxm/Downloads/cpython-a4afb55fb226e1debcdaaf80890b790198ba14cc/Lib/socket.py", line 953, in getaddrinfo
OSError: [Errno 24] Too many open files
  File "/home/xxm/Downloads/cpython-a4afb55fb226e1debcdaaf80890b790198ba14cc/Lib/socket.py", line 953, in getaddrinfo
OSError: [Errno 24] Too many open files
OSError: [Errno 24] Too many open files
OSError: [Errno 24] Too many open files
-----------------------------------------------------------

I think the crash is fixed by this PR. I try other arguments. No crash is reported. Thank you!

Thank you for testing. I've added a NEWS entry to the PR, so it's ready for review by the core devs.

Note that PyThread_exit_thread() can still be called by daemon threads if they try to take the GIL after Py_Finalize(), and also via C APIs like 
PyEval_RestoreThreads() (see bpo-42969), so, in general, libgcc_s is still required for CPython.

messages: + msg384834
nosy: + doko