Thanks to issue #15425, the number of special cases in the importlib frame stripping is growing. This patch tweaks the frame stripping mechanism to make it easy to selectively strip frames in importlib._bootstrap by creating a _call_with_frames_removed helper function.

The rules in remove_importlib_frames then change to remove any sequence of importlib frames which ends with that helper and all importlib frames in the ImportError case.

In trying to find a new case that wasn't already covered by the test suite, I found an error which I'm not even sure should be an error. Doesn't 3.2 suppress failures that occur when attempting to implicitly cache the .pyc?

Anyway, given the rest of the patch, trimming this traceback will be easy if we decide that's the right thing to do.

The test cases uploaded to issue 7559 ("TestLoader.loadTestsFromName swallows import errors") a while back contain a number of distinct cases (four, I think) in which an import error can be raised.  IIRC, these include a couple recursive scenarios (cyclical import from within __init__.py, etc).

Those test cases were designed specifically to cover different scenarios in which unittest's loader was swallowing import errors.  Might be worth taking a look.

This is an elegant solution, +1 from me.

That said, given the nature of the patch (a cleanup without any functional impact), I don't think it should be a release blocker.

The "release blocker" status comes from the test case I added in order to demonstrate the ability to strip a new frame sequence without needing to modify the C code.

Currently that test (failing to write the PYC file) fails with an IsADirectory traceback that includes a lot of importlib frames.

There are two possible solutions:
- redirect the affected call through "_call_with_frames_removed" but otherwise leave the failure intact
- change it so this particular case isn't a fatal error

I need to check if failing to write the .pyc was a fatal error in 3.2 before I decide how to fix it (unless someone else remembers off the top of their head)

OK, after a bit of experimentation, it appears both 3.2 and 3.3 eventually get annoyed if you mess about too much with __pycache__.

1. They're both fine if __pycache__ is entirely unwritable (they just silently skip caching the bytecode)

2. 3.2 throws EOFError if you replace the cache entry with an empty file, 3.3 silently rewrites it with a valid cached version

3. 3.2 throws EOFError if you replace the cache entry with a directory, 3.3 throws a more accurate IsADirectory error

That means my chosen test case is a valid one, and I can just update the offending call in importlib._bootrap to use the new frame stripping hook as I originally planned.

Looking at the actual code - I don't think it actually makes sense to strip the frames in this case. Unlike the previous examples, it only comes up if someone is doing something deliberately pathological, at which point a little noise in the traceback is the least of your worries.

Updated patch just drops that test case entirely and only has the changes that simplify the frame removal code. Since Georg isn't on IRC right now, assigning him to decide if he wants this version in beta 2 rather than the current "multiple exit points" approach.

Looks good and simplifies things.  Green light :)

patch LGTM.  Nice and clean.

New changeset 62033490ca0f by Nick Coghlan in branch 'default':
Close #15486: Simplify the mechanism used to remove importlib frames from tracebacks when they just introduce irrelevant noise
http://hg.python.org/cpython/rev/62033490ca0f

nosy: + python-dev
messages: + msg166988

resolution: fixed
stage: patch review -> resolved

keywords: + patch
stage: patch review
title: Standardise the mechanisms for stripping importlib frames -> Standardised mechanism for stripping importlib frames from tracebacks
nosy: + brett.cannon, georg.brandl, amaury.forgeotdarc, eric.snow
versions: + Python 3.3
messages: + msg166748
priority: normal -> release blocker
components: + Interpreter Core, Library (Lib)
type: behavior