[WIP] bpo-36142: Rework Python init to add _PyPreConfig by vstinner · Pull Request #12087 · python/cpython
I added a _PyCoreConfig structure to Python 3.7 which contains almost
all parameters used to configure Python. Problems: _PyCoreConfig uses
bytes and Unicode strings (char* and wchar_t*) whereas it is also
used to setup the memory allocator and (filesystem, locale and stdio)
encodings.
I propose to add a new _PyPreConfig which is the "strict minimum"
configuration to setup encodings and the memory allocator. In
practice, it also contains parameters which directly or indirectly
impacts the allocator and encodings. For example, isolated impacts
use_environment which impacts the allocator (PYTHONMALLOC environment
variable). Another example: dev_mode=1 sets the allocator to "debug".
The command line arguments are now parsed twice. _PyPreConfig only
parses a few parameters like -E, -I and -X. A temporary _PyPreCmdline
is used to store command line arguments like -X options.
I moved structures closer to where they are used. "Global" _PyMain
structure has been removed. _PyCmdline now lives way shorter than
previously and is moved from main.c to coreconfig.c. The idea is to
better control when and how memory is allocated.
In term of API, we get something like:
_PyCoreConfig config = _PyCoreConfig_INIT;
config.preconfig.stdio_encoding = "iso8859-1";
config.preconfig.stdio_errors = "replace";
config.user_site_directory = 0;
...
_PyInitError err = _Py_InitializeFromConfig(&config);
if (_Py_INIT_FAILED(err)) {
_Py_ExitInitError(err);
}
...
Py_Finalize();
return 0;
"config.preconfig.stdio_errors" syntax isn't great, but it's simpler
to implement than duplicating all _PyPreConfig fields into
_PyCoreConfig.
Changes:
* Add Python/preconfig.c file
* Rename Include/coreconfig.h to Include/cpython/coreconfig.h.
* Add Include/internal/pycore_coreconfig.h
* Add _PyPreConfig structure: pre-initialization configuration for
allocator and encodings.
* _PyCoreConfig now has a "_PyPreConfig preconfig;" field
* Add _PyArgv structure: helper to pass command line argument in
bytes (char*) or Unicode (wchar_t*)
* Add exitcode to _PyInitError
* Add 'exitcode' field to _PyInitError structure
* Add _Py_INIT_EXIT() macro
* Rename _Py_FatalInitError() to _Py_ExitInitError()
* Add new fields to _PyCoreConfig:
* run_command
* run_filename
* run_module
* skip_source_first_line
* Add _PyMain_Usage()
* _Py_InitializeCore() no longer changes the memory allocator.
* Hardcode short and long options in _PyOS_GetOpt().
* Make _PyCoreConfig_xxx() functions internal (Py_BUILD_CORE).
* Move _PyCmdline from main.c to coreconfig.c
* main.c: Remove _PyMain structure