Attached dagpool_test.py script triggers a crash on Python 3.7, but works well on Python 3.6. The script is a highly simplified code of tests/dagpool_tests.py of eventlet.

To reproduce the crash, you only need a single dependency: greenlet. Example using a venv:

    python3.7 -m venv ENV
    ENV/bin/python -m pip install greenlet

Then run the script:

    $ test_venv/bin/python dagpool_test.py 
    Segmentation fault (core dumped)

eventlet bug report:
https://github.com/eventlet/eventlet/issues/475


I suspected that the bug is caused by the new exc_info attribute of PyThreadState: commit ae3087c6382011c47db82fea4d05f8bbf514265d.

$ gdb -args test_venv/bin/python -X faulthandler dagpool_test.py
(gdb) run

Program received signal SIGSEGV, Segmentation fault.
0x000000000056c9d2 in PyErr_SetObject (exception=<type at remote 0x9709a0>, value=0x0) at Python/errors.c:101
101	        Py_INCREF(exc_value);
(gdb) where
#0  0x000000000056c9d2 in PyErr_SetObject (exception=<type at remote 0x9709a0>, value=0x0) at Python/errors.c:101
#1  0x000000000056cd4e in PyErr_SetNone (exception=<type at remote 0x9709a0>) at Python/errors.c:162
#2  0x000000000067cb0c in gen_send_ex (gen=0x7fffea651d78, arg=0x0, exc=0, closing=0) at Objects/genobject.c:241
#3  0x000000000067dd86 in gen_iternext (gen=0x7fffea651d78) at Objects/genobject.c:542
#4  0x00000000005461b1 in _PyEval_EvalFrameDefault (...)
...

(gdb) p tstate
$1 = (PyThreadState *) 0xa132a0

(gdb) p tstate->exc_info
$2 = (_PyErr_StackItem *) 0x7fffea651930

(gdb) p *tstate->exc_info
$3 = {
  exc_type = 0x0, 
  exc_value = 0x0, 
  exc_traceback = 0x0, 
  previous_item = 0x7fffea651d20
}

(gdb) p *tstate->exc_info->previous_item
$4 = {
  exc_type = <unknown at remote 0xdbdbdbdbdbdbdbdb>, 
  exc_value = <unknown at remote 0xdbdbdbdbdbdbdbdb>, 
  exc_traceback = <unknown at remote 0xdbdbdbdbdbdbdbdb>, 
  previous_item = 0xdbdbdbdbdbdbdbdb
}

> I suspected that the bug is caused by the new exc_info attribute of PyThreadState: commit ae3087c6382011c47db82fea4d05f8bbf514265d.

The change comes from bpo-25612.

Strange. 
I could only reproduce this intermittently with 3.7 build.
But with a debug build, I cannot reproduce this at all.
Rebuilding Python (optimised, but without PGO) I cannot reproduce at all now.

This looks like a Greenlet bug to me.
Possibly https://github.com/python-greenlet/greenlet/commit/780a12b51fcef9adcc4f2c9a4cc5b05c2d652ba4 is incomplete.

It is not clear to me why greenlets hold their own exception state, but I suspect that if they didn't then this issue might go away.

> I could only reproduce this intermittently with 3.7 build.

Oh, I forgot to mention that I have to run the test many times to reproduce the crash. Between 1 and 10 times.

I used the master branch compiled in debug mode to report this issue.

It's not a bug in Python but in greenlet:

* greenlet bug: https://github.com/python-greenlet/greenlet/issues/131
* my greenlet fix: https://github.com/python-greenlet/greenlet/pull/132
* Red Hat bug: https://bugzilla.redhat.com/show_bug.cgi?id=1594248

resolution: third party
stage: resolved