[mypyc] Move API table definitions to .c files by p-sawicki · Pull Request #21183 · python/mypy
lib-rt API tables are defined as static variables in lib-rt headers. This means that each translation unit gets its own independent instance of these variables.
That becomes a problem in multi-file compilation mode when using BytesWriter.write, as this function is translated to CPyBytesWriter_Write by mypyc which is defined in bytes_writer_extra_ops.c. With multi-file compilation, bytes_writer_extra_ops.c is compiled as its own translation unit and gets linked with the C extension compiled from python files.
The C extension TU copies the librt.strings capsule contents into the global table but because it's static this is not visible in the table in the bytes_writer_extra_ops.c TU, which stays zero-initialized. This results in a seg fault with the following call chain:
CPyBytesWriter_Write -> CPyBytesWriter_EnsureSize -> LibRTStrings_ByteWriter_grow_buffer_internal -> LibRTStrings_API[5]-> oops all zeros
To fix this, declare the tables as extern and define them in .c files so there's only one global version of each variable. There's one new .c file for each lib-rt module that is compiled or included when mypyc detects a dependency on that module.