Issue45881
Created on 2021-11-23 16:37 by christian.heimes, last changed 2022-04-11 14:59 by admin.
| Pull Requests | |||
|---|---|---|---|
| URL | Status | Linked | Edit |
| PR 29752 | merged | christian.heimes, 2021-11-24 16:18 | |
| PR 29753 | merged | christian.heimes, 2021-11-24 17:57 | |
| PR 29754 | merged | christian.heimes, 2021-11-24 18:01 | |
| PR 29835 | merged | christian.heimes, 2021-11-29 11:45 | |
| Messages (7) | |||
|---|---|---|---|
| msg406853 - (view) | Author: Christian Heimes (christian.heimes) * ![]() |
Date: 2021-11-23 16:37 | |
Cross compiling is an approach to compile a program for a different CPU architecture and platform, e.g. compile for an ARM64 (aarch64) or WASM on a x86_64 build system. Python configure script, Makefile, and setup.py have multiple references to cross compiling. However I could not find any documentation in the devguide or Python docs how to cross compile. We also lack CI (buildbot) to test cross compiling. This lack of awareness and testing leads to breakage of the feature. For example the design of Programs/_freeze_module in main (3.11-dev) is incompatible with cross compiling. I kinda got cross compiling working with 3.10, but only with some additional hacks and patches. I also ran into other problems like _PYTHON_HOST_PLATFORM env var is not automatically forwarded to setup.py. The helper functions add_multiarch_paths() and add_cross_compiling_paths() break builds for me, too. Cross compiling only works when the methods are commented out. |
|||
| msg406860 - (view) | Author: Christian Heimes (christian.heimes) * ![]() |
Date: 2021-11-23 17:00 | |
I have uploaded a reproducer at https://github.com/tiran/cpython_builddep/compare/cross-aarch64 $ podman run -ti --rm -v $(pwd):/cpython:Z quay.io/tiran/cpythonbuild:ubuntu-impish-aarch64 ... aarch64-linux-gnu-gcc -fPIC -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -Werror=implicit-function-declaration -fvisibility=hidden -I/cpython/Include/internal -I/usr/include/x86_64-linux-gnu -I. -IObjects -IPython -I/usr/include -I/usr/include/x86_64-linux-gnu -I/usr/local/include -I/cpython/Include -I/cpython/builddep/ubuntu-impish-x86_64 -c /cpython/Modules/_ctypes/_ctypes.c -o build/temp.linux-aarch64-3.10/cpython/Modules/_ctypes/_ctypes.o -DPy_BUILD_CORE_MODULE -DHAVE_FFI_PREP_CIF_VAR=1 -DHAVE_FFI_PREP_CLOSURE_LOC=1 -DHAVE_FFI_CLOSURE_ALLOC=1 In file included from /usr/aarch64-linux-gnu/include/features.h:508, from /usr/include/x86_64-linux-gnu/bits/libc-header-start.h:33, from /usr/aarch64-linux-gnu/include/limits.h:26, from /usr/lib/gcc-cross/aarch64-linux-gnu/11/include/limits.h:203, from /usr/lib/gcc-cross/aarch64-linux-gnu/11/include/syslimits.h:7, from /usr/lib/gcc-cross/aarch64-linux-gnu/11/include/limits.h:34, from /cpython/Include/Python.h:11, from /cpython/Modules/zlibmodule.c:8: /usr/include/x86_64-linux-gnu/gnu/stubs.h:7:11: fatal error: gnu/stubs-32.h: No such file or directory 7 | # include <gnu/stubs-32.h> |
|||
| msg406935 - (view) | Author: Christian Heimes (christian.heimes) * ![]() |
Date: 2021-11-24 16:13 | |
bpo-45886 addresses the cross build issue with freeze_module command. The wrong header files come from the fact that setup.py uses CC variable from sysconfig instead of environment. The sysconfig variable contains the C compiler of the build interpreter instead of the host interpreter. The correct value is in os.environ. |
|||
| msg406942 - (view) | Author: Christian Heimes (christian.heimes) * ![]() |
Date: 2021-11-24 17:53 | |
New changeset b30bf4520ae9d6e7eca09d812dd8a86c020b9202 by Christian Heimes in branch 'main': bpo-45881: Use CC from env first for cross building (GH-29752) https://github.com/python/cpython/commit/b30bf4520ae9d6e7eca09d812dd8a86c020b9202 |
|||
| msg407016 - (view) | Author: Christian Heimes (christian.heimes) * ![]() |
Date: 2021-11-25 19:53 | |
New changeset cd6d2577fadc4cc0275017f27f46b0a628216353 by Christian Heimes in branch '3.9': [3.9] bpo-45881: Use CC from env first for cross building (GH-29752) (GH-29754) https://github.com/python/cpython/commit/cd6d2577fadc4cc0275017f27f46b0a628216353 |
|||
| msg407165 - (view) | Author: Christian Heimes (christian.heimes) * ![]() |
Date: 2021-11-27 20:14 | |
New changeset 545aebd2ecef9f6c3b2ca1973e3e0515d8355ce3 by Christian Heimes in branch '3.10': [3.10] bpo-45881: Use CC from env first for cross building (GH-29752). (GH-29753) https://github.com/python/cpython/commit/545aebd2ecef9f6c3b2ca1973e3e0515d8355ce3 |
|||
| msg407292 - (view) | Author: Christian Heimes (christian.heimes) * ![]() |
Date: 2021-11-29 16:23 | |
New changeset 992565f7f72fd8250b788795f76eedcff5636a64 by Christian Heimes in branch 'main': bpo-45881: configure --with-freeze-module --with-build-python (GH-29835) https://github.com/python/cpython/commit/992565f7f72fd8250b788795f76eedcff5636a64 |
|||
| History | |||
|---|---|---|---|
| Date | User | Action | Args |
| 2022-04-11 14:59:52 | admin | set | github: 90039 |
| 2021-11-29 16:23:38 | christian.heimes | set | messages: + msg407292 |
| 2021-11-29 11:45:56 | christian.heimes | set | pull_requests: + pull_request28066 |
| 2021-11-27 20:14:12 | christian.heimes | set | messages: + msg407165 |
| 2021-11-27 12:06:24 | christian.heimes | set | dependencies: + sysconfig --generate-posix-vars creates wrong file when cross compiling |
| 2021-11-26 09:41:56 | christian.heimes | link | issue40280 dependencies |
| 2021-11-25 19:53:23 | christian.heimes | set | messages: + msg407016 |
| 2021-11-24 18:01:05 | christian.heimes | set | pull_requests: + pull_request27991 |
| 2021-11-24 17:57:55 | christian.heimes | set | pull_requests: + pull_request27990 |
| 2021-11-24 17:53:41 | christian.heimes | set | messages: + msg406942 |
| 2021-11-24 16:18:41 | christian.heimes | set | keywords:
+ patch stage: patch review pull_requests: + pull_request27989 |
| 2021-11-24 16:13:08 | christian.heimes | set | dependencies:
+ Fix Program/_freeze_module for cross compiling Python messages: + msg406935 |
| 2021-11-23 17:00:57 | christian.heimes | set | messages: + msg406860 |
| 2021-11-23 16:37:45 | christian.heimes | create | |
