When Python is built without --enable-shared, the "Py_FrozenMain" is not exported.

When Python is built with --enable-shared, the "Py_FrozenMain" is exported as expected.

I can reproduce the issue with attached reproduce.tar.gz example:

* build.sh exports "func1" symbol
* build_ar.sh doesn't export the "func1" symbol

The difference is that build.sh links directly two object files (.o) into a binary, whereas build_ar.sh creates a static library (.a) and then link the static library into a binary.

Python is always built with a static library (libpythonVERSION.a) which causes the issue.

A solution is to pass -Wl,--whole-archive option to the linker to export *all* symbols exported by all object files, and not only export symbols of the object files which are used.

I'm not sure why, but "Py_FrozenMain" is the *only* impacted symbol of the whole C API.

This issue was discovered by Petr Viktorin who works on the stable ABI:
https://mail.python.org/archives/list/capi-sig@python.org/thread/5QLI3NUP3OSGLCCIBAQOTX4GEJQBWJ6F/

The "python" binary exports 1610 symbols. With the attached fix, it exports 1611 symbols (+1): "Py_FrozenMain" is also exported :-)

The symbol is not exported on Windows neither. My PR 26126 ("Test Py_FrozenMain()") fails on Windows with:

"_testembed.obj : error LNK2001: unresolved external symbol __imp_Py_FrozenMain"

("__imp_" prefix is added by Windows DLL loader.)

> I'm not sure why, but "Py_FrozenMain" is the *only* impacted symbol of the whole C API.

Apparently, on some platforms `PyModule_Create2` and `PyModule_FromDefAndSpec2` don't appear either.

Should I rename the issue to cover these as well?

New changeset 276468dddb46c54980c782c09cdb53bd90755752 by Petr Viktorin in branch 'main':
bpo-43795: Add a test for Stable ABI symbol availability using ctypes (GH-26354)
https://github.com/python/cpython/commit/276468dddb46c54980c782c09cdb53bd90755752

Repurposing this bug to track all symbols that don't appear in some configurations.

- `Py_FrozenMain` should be added to stable ABI when this is fixed. It was left out of the 3.10 stable ABI for only this reason. See discussion here: https://mail.python.org/archives/list/capi-sig@python.org/thread/5QLI3NUP3OSGLCCIBAQOTX4GEJQBWJ6F/

- `PyModule_Create2` and `PyModule_FromDefAndSpec2` are "temporarily" excluded from tests in `Tools/scripts/stable_abi.py`'s `gen_ctypes_test`, so that all the other symbols can get regression testing. When this issue is fixed, that special case should be removed.

On Solaris (and most likely several other platforms), `PyThread_get_thread_native_id` is also not available.

> On Solaris (and most likely several other platforms), `PyThread_get_thread_native_id` is also not available.

Oh, I added an explicit test for that in my PR 30556.

> `Py_FrozenMain` should be added to stable ABI

I suggest to first fix the issue (export the symbol), and then write a second PR to add it to the stable ABI.

New changeset 6be848922bc0f4c632c255c39de82a45b6480286 by Victor Stinner in branch 'main':
bpo-44133: Link Python executable with object files (GH-30556)
https://github.com/python/cpython/commit/6be848922bc0f4c632c255c39de82a45b6480286

> New changeset 6be848922bc0f4c632c255c39de82a45b6480286 by Victor Stinner in branch 'main':
> bpo-44133: Link Python executable with object files (GH-30556)

Sadly, Py_FrozenMain() is still missing on Windows. See:
https://github.com/python/cpython/pull/30556#issuecomment-1012032712

Until Windows also exports the symbol, I don't think that we can add the symbol to the stable ABI.

I prefer to not backport the change since it's always risky to break the build system on a stable branch.

If someone wants to fix the Windows build to also export Py_FrozenMain(), please open a new issue.

I consider that the initial issue is fixed, so I close the issue.

>> On Solaris (and most likely several other platforms), `PyThread_get_thread_native_id` is also not available.

> Oh, I added an explicit test for that in my PR 30556.

Now it started failing on a different place:

======================================================================
FAIL: test_export_symbols (test.test_capi.CAPITest) (name='PyThread_get_thread_native_id')
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/..../cpython-main/Lib/test/test_capi.py", line 662, in test_export_symbols
    self.assertTrue(hasattr(ctypes.pythonapi, name))
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: False is not true

Looking at the test, is the expectation that all OSes must implement it since 3.11?

Oh no, I expected that the new way to build Python would also export PyThread_get_thread_native_id() on Solaris. I reopen the issue.

Can you please specify your configure command? Can you check without Python if the symbol is exported or not? If you use --enable-shared, check in libpython, otherwise check in the "python" binary.

Ah, sorry, I could have described the issue better. It's not a problem with exporting, PyThread_get_thread_native_id() isn't available on Solaris (and possibly other platforms) at all.

https://github.com/python/cpython/blob/main/Include/pythread.h#L28
https://github.com/python/cpython/blob/main/Python/thread_pthread.h#L329

The reason I didn't implement it yet is that Solaris doesn't expose anything like native thread id. We do have functions like `_lwp_self()` or `pthread_self()` or `thr_self()` but neither of them returns id where "value may be used to uniquely identify this particular thread system-wide". (I presume that means that no other thread of no other process running on a given system would return the same number - all these functions return single digit numbers so there is no way they are unique system wide).

If necessary, I guess that such a number can be created by masking pid and thread id together, but then there's a question of how it is supposed to be used (because OS would not understand it).

> Ah, sorry, I could have described the issue better. It's not a problem with exporting, PyThread_get_thread_native_id() isn't available on Solaris (and possibly other platforms) at all.

Oh ok, it's a simple bug in my test. I wrote GH-30636 to fix it.

New changeset 16901c0482734dbd389b09ca3edfcf3e22faeed7 by Victor Stinner in branch 'main':
bpo-44133: Skip PyThread_get_thread_native_id() if not available (GH-30636)
https://github.com/python/cpython/commit/16901c0482734dbd389b09ca3edfcf3e22faeed7

> Ah, sorry, I could have described the issue better. It's not a problem with exporting, PyThread_get_thread_native_id() isn't available on Solaris (and possibly other platforms) at all.

Jakub, does this mean test_stable_abi_ctypes fails on Solaris?
Are there any other missing functions?
I opened bpo-47169 to improve the situation.

Yes, it still does, and PyThread_get_thread_native_id is the only symbol missing.

stage: patch review -> resolved