Flaky test - teardown in test_hist_file

Flaky test - teardown in test_hist_file_config

We build ipython in Fedora Linux as an RPM package, and we've recently hit the following issue in tests:

==================================== ERRORS ====================================
__________________ ERROR at teardown of test_hist_file_config __________________

    @pytest.fixture()
    def inner():
        assert HistoryManager._max_inst == 1
        HistoryManager._max_inst = N
        lh = len(HistoryManager._instances)
        try:
            yield
            gc.collect()
>           assert len(HistoryManager._instances) == lh
E           AssertionError: assert 2 == 1
E            +  where 2 = len({<weakref at 0x7f4e89744cc0; to 'IPython.core.history.HistoryManager' at 0x7f4e8bcc96a0>, <weakref at 0x7f4e882850d0; to 'IPython.core.history.HistoryManager' at 0x7f4e88764050>})
E            +    where {<weakref at 0x7f4e89744cc0; to 'IPython.core.history.HistoryManager' at 0x7f4e8bcc96a0>, <weakref at 0x7f4e882850d0; to 'IPython.core.history.HistoryManager' at 0x7f4e88764050>} = HistoryManager._instances

../tests/test_history.py:47: AssertionError

The problem is that it's very hard to debug because it fails in our build system only in like 3 out of 10 builds. Do you have any idea what might be causing this?