Mac OS builds fail due to test_site in VSTS for 3.7 branches only

Ref : https://python.visualstudio.com/cpython/_build/results?buildId=17921&view=logs

2018-07-21T07:01:08.5912930Z Traceback (most recent call last):
2018-07-21T07:01:08.5927520Z   File "/Users/vsts/agent/2.136.1/work/1/s/Lib/test/test_site.py", line 199, in test_s_option
2018-07-21T07:01:08.5942120Z     self.assertIn(usersite, sys.path)
2018-07-21T07:01:08.5959870Z AssertionError: '/Users/vsts/.local/lib/python3.7/site-packages' not found in ['/Users/vsts/agent/2.136.1/work/1/s', '/usr/local/lib/python37.zip', '/Users/vsts/agent/2.136.1/work/1/s/Lib', '/Users/vsts/agent/2.136.1/work/1/s/build/lib.macosx-10.13-x86_64-3.7-pydebug', '/usr/local/lib/python3.7/site-packages']
2018-07-21T07:01:08.5971110Z 
2018-07-21T07:01:08.5985860Z ======================================================================
2018-07-21T07:01:08.6000880Z FAIL: test_startup_imports (test.test_site.StartupImportTests)
2018-07-21T07:01:08.6017050Z ----------------------------------------------------------------------
2018-07-21T07:01:08.6033630Z Traceback (most recent call last):
2018-07-21T07:01:08.6048270Z   File "/Users/vsts/agent/2.136.1/work/1/s/Lib/test/test_site.py", line 487, in test_startup_imports
2018-07-21T07:01:08.6063180Z     self.assertFalse(modules.intersection(collection_mods), stderr)
2018-07-21T07:01:08.6079990Z AssertionError: {'reprlib', 'heapq', 'collections', 'keyword', 'operator', 'functools', 'types'} is not false : import _frozen_importlib # frozen

I couldn't find any related issues in search and hence opened this. Feel free to close this if it's a duplicate or fixed in some other branch.

Thanks

It seems that the last successful build in 3.7 was for https://github.com/python/cpython/commit/339e0c1296c61c9dbc1f8f880c5c668bf4007e5e and all PRs after it had the same failure.

Sorry for not adding the relevant components during ticket creation.

Thanks

Adding @vstinner as a friendly reminder since buildbot related failures are notified. This issue causes all the PRs for 3.7 on Mac OS VSTS builds to fail since July 19 though other Mac related builds are passing.

Feel free to remove yourself or close this if it's irrelevant.

Thanks

Maybe, vsts machine installs Homebrew's Python.
Homebrew's Python installes /usr/local/lib/python3.7/sitecustomize.py

https://github.com/Homebrew/homebrew-core/blob/0ce9a5a481b3472608fe4e4188cb5d243deebb99/Formula/python.rb#L295-L329

When running tests, it is imported.
It causes test_startup_imports fail.

Maybe, we can use random prefix instead of default /usr/local to avoid this conflict.

New changeset 777cdd94b9bebd5b22df0fc293aa078d5537b988 by INADA Naoki in branch '3.6':
bpo-34177: vsts: Avoid conflict with Homebrew Python (GH-8430)
https://github.com/python/cpython/commit/777cdd94b9bebd5b22df0fc293aa078d5537b988

Thank you for the fix Naokia!

This error is happening in the x86-64 High Sierra 3.7 buildbot:

https://buildbot.python.org/all/#/builders/147/builds/174

======================================================================
FAIL: test_startup_imports (test.test_site.StartupImportTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/buildbot/buildarea/3.7.billenstein-sierra/build/Lib/test/test_site.py", line 487, in test_startup_imports
    self.assertFalse(modules.intersection(collection_mods), stderr)
AssertionError: {'collections', 'heapq', 'types', 'functools', 'keyword', 'reprlib', 'operator'} is not false : import _frozen_importlib # frozen

Same fix can be applied to buildbot.

This is caused by (a) Homebrew installs Python to /usr/local with (hacky) sitecustomize, and (b) Python's default prefix is /usr/local.

I think test should be run without conflicting with another installation.

>  Same fix can be applied to buildbot.

Sure. Go ahead :-) The buildbot configuration can be found at:
https://github.com/python/buildmaster-config/

Send a pull request, and Zachary Ware (or me) will  review it. Zach is the expert here ;-)

I think this can be closed with https://github.com/python/buildmaster-config/commit/9a456462fb891328b5b8a170522d5f56f480fdfb. The buildbots are also green now with the failure reported in https://buildbot.python.org/all/#/builders/147/builds/174 fixed with the commit.

Thanks