mgr: load modules in separate python sub-interpreters by tserong · Pull Request #14971 · ceph/ceph

@tserong tserong changed the title [DNM] mgr: load modules in separate python sub-interpreters mgr: load modules in separate python sub-interpreters

May 10, 2017

@tserong tserong changed the title mgr: load modules in separate python sub-interpreters [DNM] mgr: load modules in separate python sub-interpreters

May 11, 2017

@tserong tserong changed the title [DNM] mgr: load modules in separate python sub-interpreters mgr: load modules in separate python sub-interpreters

May 12, 2017

chardan

chardan

chardan

chardan

chardan

chardan

chardan

chardan

chardan

chardan

chardan

chardan

chardan

chardan

chardan

chardan

chardan

chardan

chardan

chardan

chardan

Prep work for loading modules in separate python sub-interpreters.
According to the Python C API docs, mixing the PyGILState_*() API
with sub-interpreters is unsupported, so I've replaced these calls
with a Gil class, which will acquire and release the GIL against
a specific python thread state.

Note the manual python thread state creation in MgrPyModule::serve().
The PyGILState_*() APIs would have done that for us, but we're not
using them anymore, so have to do it by hand (see
https://docs.python.org/2/c-api/init.html#non-python-created-threads
for some description of this).

Signed-off-by: Tim Serong <tserong@suse.com>
This provides a reasonable amount of isolation between mgr
modules.  Notably, with this change, it's possible to have more
than one mgr module use cherrypy without conflicts.

Each MgrPyModule gets its own python sub-interpreter.  The logger,
the ceph_state module and sys.path need to be set up separately
for each sub-interpreter, so all that happens in MgrPyModule's
constructor (previously this was done on the main interpreter
in PyModules::init()).

Each sub-interpreter has its own python thread state.  The main
interpreter also has a python thread state, but that is almost
unused except for during setup and teardown of the whole beast.

On top of that, an additional python thread state is necessary
for each OS thread that calls into python code (e.g. the serve()
method).

Some care needs to be taken to ensure that the right thread state
is active at the right time; note how the call to handle_pyerror()
in PyModules::init() had to be moved inside MgrPyModle::load().
PyModules should be using the main thread state, and MgrPyModule
should usually be using its sub-interpreter thread state, except
for very early in its constructor (before the sub-interpreter has
been created), and in the serve() method where another python
thread state is created to map to the separate OS thread that is
responsible for invoking this method.

The ceph_state module (PyState.cc) naturally has no idea what
context it's being run in, so uses PyThreadState_Get() when it
needs to know the python thread state.

Signed-off-by: Tim Serong <tserong@suse.com>

@jcsp jcsp self-requested a review

May 24, 2017 10:24

jcsp

jcsp approved these changes May 30, 2017

@tserong tserong deleted the wip-mgr-py-sub-interpreter branch

June 5, 2017 06:39

@tserong tserong restored the wip-mgr-py-sub-interpreter branch

January 13, 2018 09:40

@tserong tserong deleted the wip-mgr-py-sub-interpreter branch

January 13, 2018 09:41