I initially reported this on launchpad at https://bugs.launchpad.net/bugs/1782647.

I'm running a test for a project that hangs and requires a Control-C to cancel it. The results look like this: https://paste.ubuntu.com/p/SwXsCcghjt/

In narrowing down on this some more, the use of queue.SimpleQueue() seems to make the difference that causes the hang. If I switch that back to queue.Queue() (which was the case for py3.6) the hang goes away:

- self._work_queue = queue.Queue()
+ self._work_queue = queue.SimpleQueue()

To reproduce with OpenStack designate:

git clone https://github.com/openstack/designate.

Then, update tox.ini with the following:

--- a/tox.ini
+++ b/tox.ini
@@ -1,6 +1,6 @@
 [tox]
 minversion = 2.0
-envlist = py35,py27,flake8
+envlist = py35,py37,py27,flake8
 skipsdist = True

 [testenv]
@@ -39,6 +39,12 @@ commands =
   {[testenv]commands}
   stestr run '{posargs}'

+[testenv:py37]
+basepython = python3.7
+commands =
+ {[testenv]commands}
+ stestr run 'designate\.tests\.test_workers\.test_processing\.TestProcessingExecutor\.(test_execute_multiple_tasks)'
+
 [testenv:docs]
 basepython = python3
 deps =


And run: tox -e py37

I looked into the code of test_execute_multiple_tasks and designate/worker/processing.py from which the executor was initialized. I have converted the test case code removing the project related code as a standalone script. I hope the attached code is the minimal code required. The change from Queue to SimpleQueue was made with commit : ab74504346a6e2569b3255b7b621c589716888c4 and relevant issue is https://bugs.python.org/issue32576 .

I ran the test code by checking out each commit obtained by `git log --oneline --format="%h" Lib/concurrent/futures/thread.py` . The tests finish successfully without any hanging as below

Testing on commit :  c4b695f
Finished Tasks ['UnnamedTask', 'UnnamedTask', 'UnnamedTask', 'UnnamedTask', 'UnnamedTask']s in 0.0012586116790771484fs
Testing on commit :  ab74504
Finished Tasks ['UnnamedTask', 'UnnamedTask', 'UnnamedTask', 'UnnamedTask', 'UnnamedTask']s in 0.0013058185577392578fs
Testing on commit :  63ff413
Finished Tasks ['UnnamedTask', 'UnnamedTask', 'UnnamedTask', 'UnnamedTask', 'UnnamedTask']s in 0.0012793540954589844fs
Testing on commit :  bc61315
Finished Tasks ['UnnamedTask', 'UnnamedTask', 'UnnamedTask', 'UnnamedTask', 'UnnamedTask']s in 0.0013325214385986328fs
Testing on commit :  a3d91b4
Finished Tasks ['UnnamedTask', 'UnnamedTask', 'UnnamedTask', 'UnnamedTask', 'UnnamedTask']s in 0.0012831687927246094fs
Testing on commit :  15f44ab
Finished Tasks ['UnnamedTask', 'UnnamedTask', 'UnnamedTask', 'UnnamedTask', 'UnnamedTask']s in 0.0013515949249267578fs
Testing on commit :  50abe87
Finished Tasks ['UnnamedTask', 'UnnamedTask', 'UnnamedTask', 'UnnamedTask', 'UnnamedTask']s in 0.0020570755004882812fs
Testing on commit :  cfd4661
Finished Tasks ['UnnamedTask', 'UnnamedTask', 'UnnamedTask', 'UnnamedTask', 'UnnamedTask']s in 0.0012753009796142578fs
Testing on commit :  20efceb
Finished Tasks ['UnnamedTask', 'UnnamedTask', 'UnnamedTask', 'UnnamedTask', 'UnnamedTask']s in 0.0012905597686767578fs
Testing on commit :  6b97374
Finished Tasks ['UnnamedTask', 'UnnamedTask', 'UnnamedTask', 'UnnamedTask', 'UnnamedTask']s in 0.001275777816772461fs
Testing on commit :  04842a8
Finished Tasks ['UnnamedTask', 'UnnamedTask', 'UnnamedTask', 'UnnamedTask', 'UnnamedTask']s in 0.0014278888702392578fs
Testing on commit :  27be5da
Finished Tasks ['UnnamedTask', 'UnnamedTask', 'UnnamedTask', 'UnnamedTask', 'UnnamedTask']s in 0.0013408660888671875fs
Testing on commit :  c13d454
Finished Tasks ['UnnamedTask', 'UnnamedTask', 'UnnamedTask', 'UnnamedTask', 'UnnamedTask']s in 0.0014138221740722656fs
Testing on commit :  81c4d36
Finished Tasks ['UnnamedTask', 'UnnamedTask', 'UnnamedTask', 'UnnamedTask', 'UnnamedTask']s in 0.0013227462768554688fs


Can you please check if I am correct on the above approach?

Thanks

Karthikeyan, thanks for taking a look. I'm also unable to recreate with your test. I'm not sure what the difference is. I'll report back if I can figure it out.

I've narrowed this down a bit more. It appears to be caused by eventlet patching of standard library thread modules. See new attached patch bp034173-recreate.py. I'll get a bug opened against eventlet.

eventlet issue opened at: https://github.com/eventlet/eventlet/issues/508

Thanks, can confirm it on Python 3.7 . Unfortunately I couldn't test it with master since I have some problems setting up virtualenv and pip with the compiled binary. Running it on Python 3.7 hangs

(bpo34173-env) ➜  cpython git:(master) ✗ python
Python 3.7.0 (default, Jun 28 2018, 02:32:19)
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>

(bpo34173-env) ➜  cpython git:(master) ✗ python bpo34173-recreate.py
^CException in worker
Traceback (most recent call last):
  File "/usr/lib/python3.7/concurrent/futures/thread.py", line 78, in _worker
    work_item = work_queue.get(block=True)
KeyboardInterrupt
^CException in worker
Traceback (most recent call last):
  File "/usr/lib/python3.7/concurrent/futures/thread.py", line 78, in _worker
    work_item = work_queue.get(block=True)
KeyboardInterrupt
^CException in worker
Traceback (most recent call last):
  File "/usr/lib/python3.7/concurrent/futures/thread.py", line 78, in _worker
    work_item = work_queue.get(block=True)
KeyboardInterrupt
^CException in worker
Traceback (most recent call last):
  File "/usr/lib/python3.7/concurrent/futures/thread.py", line 78, in _worker
    work_item = work_queue.get(block=True)
KeyboardInterrupt
^CException in worker
Traceback (most recent call last):
  File "/usr/lib/python3.7/concurrent/futures/thread.py", line 78, in _worker
    work_item = work_queue.get(block=True)
KeyboardInterrupt
Finished Tasks ['UnnamedTask', 'UnnamedTask', 'UnnamedTask', 'UnnamedTask', 'UnnamedTask']s in 14.368564367294312fs



Thanks

Sorry for the noise about the title change. It seems you have edited it while I was typing my comment. There was a form error while I tried submit but I ignored it to try again and it had set my title. I have reverted back to your edit.

Thanks

I think we can close this issue. It was narrowed down to eventlet. Please
see: https://github.com/eventlet/eventlet/issues/508

On Tue, Jul 24, 2018 at 1:54 PM Karthikeyan Singaravelan <
report@bugs.python.org> wrote:

>
> Karthikeyan Singaravelan <tir.karthi@gmail.com> added the comment:
>
> Sorry for the noise about the title change. It seems you have edited it
> while I was typing my comment. There was a form error while I tried submit
> but I ignored it to try again and it had set my title. I have reverted back
> to your edit.
>
> Thanks
>
> ----------
> title: [3.7] possible race condition in
> /usr/lib/python3.7/concurrent/futures/thread.py -> [3.7] deadlock in
> /usr/lib/python3.7/concurrent/futures/thread.py
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <https://bugs.python.org/issue34173>
> _______________________________________
>

Thanks for the info. I am closing it as third party.

stage: resolved

messages: + msg322317

messages: + msg322110