Home Original page

Issue 19766: test_venv: test_with_pip() failed on "AMD64 Fedora without threads 3.x" buildbot: urllib3 dependency requires the threading module

Created on 2013-11-25 09:41 by vstinner, last changed 2022-04-11 14:57 by admin. This issue is now closed.

Messages (15) msg204316 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2013-11-25 09:40 
The line "from . import exceptions" of urllib3 failed:
https://github.com/shazow/urllib3/blob/master/urllib3/__init__.py#L22

It is strange because urllib3/exceptions.py is part of the urllib3 module.

http://buildbot.python.org/all/builders/AMD64%20Fedora%20without%20threads%203.x/builds/5674/steps/test/logs/stdio

======================================================================
FAIL: test_with_pip (test.test_venv.EnsurePipTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.krah-fedora/build/Lib/test/test_venv.py", line 299, in test_with_pip
    self.run_with_capture(venv.create, self.env_dir, with_pip=True)
subprocess.CalledProcessError: Command '['/tmp/tmpd8r2w88y/bin/python', '-Im', 'ensurepip', '--upgrade', '--default-pip']' returned non-zero exit status 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.krah-fedora/build/Lib/test/test_venv.py", line 305, in test_with_pip
    self.fail(msg)
AssertionError: Command '['/tmp/tmpd8r2w88y/bin/python', '-Im', 'ensurepip', '--upgrade', '--default-pip']' returned non-zero exit status 1

**Subprocess Output**
Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.krah-fedora/build/Lib/runpy.py", line 160, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/home/buildbot/buildarea/3.x.krah-fedora/build/Lib/runpy.py", line 73, in _run_code
    exec(code, run_globals)
  File "/home/buildbot/buildarea/3.x.krah-fedora/build/Lib/ensurepip/__main__.py", line 66, in <module>
    main()
  File "/home/buildbot/buildarea/3.x.krah-fedora/build/Lib/ensurepip/__main__.py", line 61, in main
    default_pip=args.default_pip,
  File "/home/buildbot/buildarea/3.x.krah-fedora/build/Lib/ensurepip/__init__.py", line 92, in bootstrap
    _run_pip(args + [p[0] for p in _PROJECTS], additional_paths)
  File "/home/buildbot/buildarea/3.x.krah-fedora/build/Lib/ensurepip/__init__.py", line 28, in _run_pip
    import pip
  File "/tmp/tmpwzs5sx88/pip-1.5.rc1-py2.py3-none-any.whl/pip/__init__.py", line 11, in <module>
  File "/tmp/tmpwzs5sx88/pip-1.5.rc1-py2.py3-none-any.whl/pip/vcs/mercurial.py", line 9, in <module>
  File "/tmp/tmpwzs5sx88/pip-1.5.rc1-py2.py3-none-any.whl/pip/download.py", line 22, in <module>
  File "/tmp/tmpwzs5sx88/pip-1.5.rc1-py2.py3-none-any.whl/pip/_vendor/requests/__init__.py", line 58, in <module>
  File "/tmp/tmpwzs5sx88/pip-1.5.rc1-py2.py3-none-any.whl/pip/_vendor/requests/utils.py", line 24, in <module>
  File "/tmp/tmpwzs5sx88/pip-1.5.rc1-py2.py3-none-any.whl/pip/_vendor/requests/compat.py", line 7, in <module>
  File "/tmp/tmpwzs5sx88/pip-1.5.rc1-py2.py3-none-any.whl/pip/_vendor/requests/packages/__init__.py", line 3, in <module>
  File "/tmp/tmpwzs5sx88/pip-1.5.rc1-py2.py3-none-any.whl/pip/_vendor/requests/packages/urllib3/__init__.py", line 22, in <module>
ImportError: cannot import name 'exceptions'
msg204413 - (view) Author: Alyssa Coghlan (ncoghlan) * (Python committer) Date: 2013-11-25 21:21 
Potentially another platform compatibility issue for pip's dependencies.
msg204475 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2013-11-26 13:30 
In fact, it's an issue in the urllib3 library which require threads.

$ cd /home/haypo/pip/PIP/pip/_vendor/requests/packages
$ /home/haypo/prog/python/default/venv/bin/python -c 'import urllib3'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/haypo/pip/PIP/pip/_vendor/requests/packages/urllib3/__init__.py", line 24, in <module>
    from .poolmanager import PoolManager, ProxyManager, proxy_from_url
  File "/home/haypo/pip/PIP/pip/_vendor/requests/packages/urllib3/poolmanager.py", line 14, in <module>
    from ._collections import RecentlyUsedContainer
  File "/home/haypo/pip/PIP/pip/_vendor/requests/packages/urllib3/_collections.py", line 8, in <module>
    from threading import RLock
  File "/home/haypo/prog/python/default/Lib/threading.py", line 4, in <module>
    import _thread
ImportError: No module named '_thread'

Here is a patch for urllib3.

I reported the issue upstream:
https://github.com/shazow/urllib3/issues/289
msg204495 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2013-11-26 15:59 
Cool, my patch was already applied upstream:

https://github.com/shazow/urllib3/commit/929f15866e62d02a0249284cf54d1b8b6441505a

@Nick: can you try to package a new wheel package for pip?
msg205111 - (view) Author: Donald Stufft (dstufft) * (Python committer) Date: 2013-12-03 12:24 
The urllib3 in requests VCS was updated, I just need to bother Kenneth to make a new release of requests or update pip to an unreleased requests.
msg205506 - (view) Author: Alyssa Coghlan (ncoghlan) * (Python committer) Date: 2013-12-08 02:24 
Issue 19924 suggests we may need a new distlib as well
msg205512 - (view) Author: Donald Stufft (dstufft) * (Python committer) Date: 2013-12-08 03:57 
Requests was released and pip updated it, I can release a new pip but it appears that perhaps distlib needs fixed before the without threads case is taken care of?
msg205535 - (view) Author: Vinay Sajip (vinay.sajip) * (Python committer) Date: 2013-12-08 10:16 
I will look at doing a distlib update shortly - but there's another issue (#19913) that might also require an update - it' not clear yet.
msg205542 - (view) Author: Vinay Sajip (vinay.sajip) * (Python committer) Date: 2013-12-08 11:06 
I've updated distlib to use dummy_threading where threading isn't available - see

https://bitbucket.org/pypa/distlib/commits/029fee573900765729402203e39b2171d7ae0784

Can someone please test with this version vendored into pip to check that the failures no longer occur in a no-thread environment, before I do a distlib release?
msg205557 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2013-12-08 13:19 
Vinay, thanks for your fast response! :) #19913 should be resolved, too. A couple of months ago several people complained about a new file that looked like a ZIP bomb. This virus warnings looks even more severe although it's probably a false positive. Could you try a new set of binaries without UPX? https://www.virustotal.com/ lets you scan the files with more than 40 programs at once and for free.
msg205590 - (view) Author: Vinay Sajip (vinay.sajip) * (Python committer) Date: 2013-12-08 18:07 
This commit in distlib uses uncompressed launcher executables which pass the virustotal.com checks:

https://bitbucket.org/pypa/distlib/commits/e23c9e4fd3125fa88063de4dec80367b1ac82aff
msg205813 - (view) Author: Donald Stufft (dstufft) * (Python committer) Date: 2013-12-10 13:46 
Vinay, I've verified that the current default branch of distlib works without threading when vendored in pip and these tests pass.
msg206225 - (view) Author: Vinay Sajip (vinay.sajip) * (Python committer) Date: 2013-12-15 11:22 
I've released distlib 0.1.5 on PyPI. This release uses dummy_threading when threading isn't available.
msg206227 - (view) Author: Alyssa Coghlan (ncoghlan) * (Python committer) Date: 2013-12-15 12:03 
Not quite fixed yet - Donald, I think all the vendored dependencies have been updated now, would it be possible to spin and incorporate a pip 1.5rc2 somewhat soonish so we know what's still left to be addressed on the ensurepip side?
msg206753 - (view) Author: Alyssa Coghlan (ncoghlan) * (Python committer) Date: 2013-12-21 15:21 
Donald updated CPython to pip 1.5rc2, so test_venv is now passing without threading support:

http://buildbot.python.org/all/builders/AMD64%20Fedora%20without%20threads%203.x/builds/5874/steps/test/logs/stdio
History Date User Action Args 2022-04-11 14:57:54adminsetgithub: 63965 2013-12-21 15:21:29ncoghlansettype: behavior
resolution: fixed
stage: resolved 2013-12-21 15:21:19ncoghlansetstatus: open -> closed

messages: + msg206753

2013-12-15 12:03:48ncoghlansetstatus: closed -> open
assignee: dstufft
resolution: fixed -> (no value)
messages: + msg206227
2013-12-15 11:22:42vinay.sajipsetstatus: open -> closed
resolution: fixed
messages: + msg206225
2013-12-10 13:46:15dstufftsetmessages: + msg205813 2013-12-08 18:07:12vinay.sajipsetmessages: + msg205590 2013-12-08 13:19:43christian.heimessetmessages: + msg205557 2013-12-08 11:06:05vinay.sajipsetmessages: + msg205542 2013-12-08 10:16:06vinay.sajipsetmessages: + msg205535 2013-12-08 03:57:02dstufftsetmessages: + msg205512 2013-12-08 02:24:39ncoghlansetnosy: + vinay.sajip, christian.heimes
messages: + msg205506
2013-12-08 02:23:47ncoghlanlinkissue19924 superseder 2013-12-03 12:24:34dstufftsetmessages: + msg205111 2013-11-28 00:56:44vstinnersettitle: test_venv: test_with_pip() failed on "AMD64 Fedora without threads 3.x" buildbot -> test_venv: test_with_pip() failed on "AMD64 Fedora without threads 3.x" buildbot: urllib3 dependency requires the threading module 2013-11-26 15:59:32vstinnersetmessages: + msg204495 2013-11-26 13:30:13vstinnersetfiles: + urllib3_no_thread.patch
keywords: + patch
messages: + msg204475
2013-11-25 21:21:55ncoghlansetnosy: + dstufft
messages: + msg204413
2013-11-25 20:06:40ned.deilysetnosy: + ncoghlan
2013-11-25 09:41:00vstinnercreate