Isolate macOS wheel builds from Homebrew by freakboy3742 · Pull Request #8497 · python-pillow/Pillow
The existing macOS cibuildwheel configuration relies on Homebrew to provide some dependencies. This clearly works in practice, but there are some issues associated with this choice.
Firstly, it is destructive when used on a local build machine. The wheel dependency script invokes brew remove --ignore-dependencies (which can leave the host machine in a broken, and potentially difficult to restore state); and requires the use of sudo to make changes in the /usr/local tree. This means it is difficult to test cibuildwheel changes locally, or to recommend using cibuildwheel as a local build solution.
Secondly, Homebrew builds could be incompatible with Pillow builds. The MACOSX_DEPLOYMENT_TARGET for Homebrew dependencies is fixed by the Homebrew build chain, rather than the Pillow build tools, so a change in Pillow's configuration may not be satisfied by Homebrew's build configuration. This isn't an issue at present, but it could easily become one in future.
Related to this, there is also the potential that a change in a Homebrew recipe could lead to the inadvertent introduction of additional binary libraries into the delocated macOS binaries. This is especially problematic with the fribidi, raqm and freetype dependedencies which are included as "vendor" sources.
However, the real motivation for proposing this change is that it is a precursor for PEP 730-compliant iOS builds. If Homebrew is on the build path when compiling iOS binaries, compiler tooling will often find an ARM64 Homebrew binary and attempt link it into an iOS library - which then (predictably) breaks. So - Homebrew isolation is essential for iOS compilation. Since the build processes for iOS and macOS are very similar, and Homebrew isolation was necessary for iOS, adding Homebrew isolation for macOS will simplify any future iOS patch. It also provides a way to submit iOS support in a series of smaller pieces, rather than a single "monster" patch. Plus, it provides all the side benefits of isolation described above.
An overview of the changes:
-
Uses the
buildfolder as a location for all dependency builds. This prevents the root checkout from becoming dirtied by dependency artefacts, simplifying cleanup, and avoiding the need for extensive modifications to the.gitignorefile. -
Sets up an isolated build prefix in the
buildfolder (./build/deps/darwin) where dependencies can be installed, andmake installsall dependencies into that path. -
Forces
PATHto be a "clean" environment that only includes bare system tools, plus the Python binary being used for the build, and the isolated build prefix. macOS doesn't include most of it's development libraries in /usr or /usr/local, instead using a path provided by the macOS SDK (which is configured as part of the compiler toolchain). -
Adds a build of
pkg-configto the dependencies. This is the one build tool that autotools and cmake often require that Xcode doesn't provide. Providing a custom build ofpkg-configalso ensures that no dependencies other than the ones provided by cibuildwheel are included. The build recipe that is used is derived from the Homebrew recipe. -
Adds a build oflibXdmcp, to avoid a dependency on the Homebrew-provided version. -
Adds builds offribidiandraqm, rather than using the Homebrew versions as "vendored" versions. Thesetup.pyconfiguration passed in bycibuildwheelhas been modified from the default on other platforms to not use thevendorversion. -
Updates the pinned multibuild version to the current
develbranch hash. This is to incorporate Python3.13 support, plus a number of fixes required to support installs into locations other than/usr/local, and corrections to LIBDIR handling for platforms that use thelib64suffix. -
Adds adependencies-prefixconfiguration option tosetup.py. This allows the user to provide a specific location to look for dependencies, rather than the series of Fink/MacPorts/Homebrew fallbacks that currently exist.