Messages (8) msg249323 - (view) Author: Chris Jerdonek (chris.jerdonek) * Date: 2015-08-29 16:15

unittest swallows some lines of the stack trace when raising an AssertionError using the "raise from" syntax inside a TestCase.  This marks it harder to pinpoint the source of test failures.  It is also confusing to see a stack trace like this because the error doesn't originate where the stack trace says it originates.

To reproduce:

    import unittest

    def foo():
        raise Exception("foo")

    class Test(unittest.TestCase):

        def test_not_okay(self):
            try:
                foo()
            except Exception as exc:
                raise AssertionError("bar") from exc

        def test_okay1(self):
            try:
                foo()
            except Exception as exc:
                raise ValueError("bar") from exc

        def test_okay2(self):
            try:
                foo()
            except Exception as exc:
                raise Exception("bar") from exc

The result (observe how the display for "test_not_okay" differs from the other two):

    ======================================================================
    ERROR: test_okay1 (error.Test)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "/Users/chris/dev/error.py", line 17, in test_okay1
        foo()
      File "/Users/chris/dev/error.py", line 5, in foo
        raise Exception("foo")
    Exception: foo

    The above exception was the direct cause of the following exception:

    Traceback (most recent call last):
      File "/Users/chris/dev/error.py", line 19, in test_okay1
        raise ValueError("bar") from exc
    ValueError: bar

    ======================================================================
    ERROR: test_okay2 (error.Test)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "/Users/chris/dev/error.py", line 23, in test_okay2
        foo()
      File "/Users/chris/dev/error.py", line 5, in foo
        raise Exception("foo")
    Exception: foo

    The above exception was the direct cause of the following exception:

    Traceback (most recent call last):
      File "/Users/chris/dev/error.py", line 25, in test_okay2
        raise Exception("bar") from exc
    Exception: bar

    ======================================================================
    FAIL: test_not_okay (error.Test)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "/Users/chris/dev/error.py", line 11, in test_not_okay
        foo()
    Exception: foo

    The above exception was the direct cause of the following exception:

    Traceback (most recent call last):
      File "/Users/chris/dev/error.py", line 13, in test_not_okay
        raise AssertionError("bar") from exc
    AssertionError: bar

    ----------------------------------------------------------------------
    Ran 3 tests in 0.001s

    FAILED (failures=1, errors=2)

msg249324 - (view) Author: Chris Jerdonek (chris.jerdonek) * Date: 2015-08-29 16:26

I guess this isn't limited just to the "raise from" syntax.  It also occurs if "from exc" is removed from the example above.

msg261719 - (view) Author: Robert Collins (rbcollins) * Date: 2016-03-14 03:07

Hmm, this is a little surprising, but - why are you raising AssertionError like that - thats what assertRaises is for.

msg261847 - (view) Author: Chris Jerdonek (chris.jerdonek) * Date: 2016-03-16 10:07

This is simply a reduced test case to illustrate the issue more clearly. There was more to it in how I was using it in practice.

msg261856 - (view) Author: Robert Collins (rbcollins) * Date: 2016-03-16 18:22

I'm fairly sure its eating the stack frames because the calling frames are annotated __unittest__ - its technically a feature :/.

msg261857 - (view) Author: Robert Collins (rbcollins) * Date: 2016-03-16 18:35

Yes, it is... ish.

The frame skipping code occurs when we serialise exceptions, and we pass a limit in. The limit is calculated on the main exception only. If the cause has a longer exception than the limit we calculated, you'd see this behaviour.

Probably need to make it possible to do per-exception processing of limit: I think via a callback or similar mechanism in traceback (because the cause might have thrown from some code that is also marked __unittest, so if we're honouring that, we should honour it within each exception.

msg269782 - (view) Author: Aaron Sokoloski (Aaron Sokoloski) Date: 2016-07-04 11:29

I've run into this bug too.  Took a while to track down the cause :)

msg382895 - (view) Author: Irit Katriel (iritkatriel) * Date: 2020-12-12 01:00

I think this is the same as issue42247, where I've recently attached a patch that removes __unittest frames from the traceback rather than count them and then set a limit for printing. 

I could add to that patch what Robert suggests here - to filter the __cause__ traceback as well.

History Date User Action Args 2021-06-02 14:06:41iritkatriellinkissue44280 superseder 2021-04-09 17:54:41dseomnsetnosy: + dseomn
2021-04-09 13:31:56iritkatriellinkissue42247 superseder 2021-04-09 13:31:16iritkatrielsetkeywords: + patch
stage: test needed -> patch review
pull_requests: + pull_request24039 2021-04-09 13:26:49iritkatriellinkissue37712 superseder 2020-12-12 01:00:01iritkatrielsetnosy: + iritkatriel

messages: + msg382895
versions: + Python 3.8, Python 3.9, Python 3.10, - Python 3.5, Python 3.6

2016-07-04 11:29:32Aaron Sokoloskisetnosy: + Aaron Sokoloski
messages: + msg269782
2016-03-16 18:52:42rbcollinssetstage: needs patch -> test needed 2016-03-16 18:35:16rbcollinssetmessages: + msg261857
title: unittest swallows par t of stack trace when raising AssertionError in a TestCase -> unittest swallows part of stack trace when raising AssertionError in a TestCase 2016-03-16 18:22:50rbcollinssetmessages: + msg261856
title: unittest swallows part of stack trace when raising AssertionError in a TestCase -> unittest swallows par t of stack trace when raising AssertionError in a TestCase 2016-03-16 10:07:23chris.jerdoneksetmessages: + msg261847 2016-03-14 03:07:09rbcollinssetmessages: + msg261719
stage: needs patch 2016-01-04 00:18:15ezio.melottisetnosy: + rbcollins, ezio.melotti, michael.foord

versions: + Python 3.5, Python 3.6, - Python 3.4

2015-08-29 16:26:13chris.jerdoneksetmessages: + msg249324
title: unittest swallows part of stack trace using "raise from" with AssertionError -> unittest swallows part of stack trace when raising AssertionError in a TestCase 2015-08-29 16:15:14chris.jerdonekcreate