Example:
https://travis-ci.org/github/python/cpython/jobs/672698615

Output:
===========================================
0:09:01 load avg: 3.96 [104/421/1] test_distutils failed -- running: test_multiprocessing_forkserver (1 min 13 sec), test_unparse (6 min 21 sec)

Warning -- files was modified by test_distutils

  Before: []

  After:  ['_configtest.gcno'] 

test test_distutils failed -- Traceback (most recent call last):

  File "/home/travis/build/python/cpython/Lib/distutils/tests/test_build_ext.py", line 112, in test_build_ext

    assert_python_ok('-c', code)

  File "/home/travis/build/python/cpython/Lib/test/support/script_helper.py", line 156, in assert_python_ok

    return _assert_python(True, *args, **env_vars)

  File "/home/travis/build/python/cpython/Lib/test/support/script_helper.py", line 142, in _assert_python

    res.fail(cmd_line)

  File "/home/travis/build/python/cpython/Lib/test/support/script_helper.py", line 70, in fail

    raise AssertionError("Process return code is %d\n"

AssertionError: Process return code is 1

command line: ['/home/travis/build/python/cpython/python', '-X', 'faulthandler', '-I', '-c', "\ntmp_dir = '/tmp/tmp3y4hpawx'\n\nimport sys\nimport unittest\nfrom test import support\n\nsys.path.insert(0, tmp_dir)\nimport xx\n\nclass Tests(unittest.TestCase):\n    def test_xx(self):\n        for attr in ('error', 'foo', 'new', 'roj'):\n            self.assertTrue(hasattr(xx, attr))\n\n        self.assertEqual(xx.foo(2, 5), 7)\n        self.assertEqual(xx.foo(13,15), 28)\n        self.assertEqual(xx.new().demo(), None)\n        if support.HAVE_DOCSTRINGS:\n            doc = 'This is a template module just for instruction.'\n            self.assertEqual(xx.__doc__, doc)\n        self.assertIsInstance(xx.Null(), xx.Null)\n        self.assertIsInstance(xx.Str(), xx.Str)\n\n\nunittest.main()\n"]

stdout:

---

---

stderr:

---

Traceback (most recent call last):

  File "<string>", line 9, in <module>

ImportError: /tmp/tmp3y4hpawx/xx.cpython-39-x86_64-linux-gnu.so: undefined symbol: __gcov_merge_add

---

===========================================

Python is built GCC with -ftest-coverage option. But it seems like this option is "leaked" to C flags used to build third party extensions in distutils.

Maybe CFLAGS_NODIST should be used instead of CFLAGS.

The Travis CI job runs these commands:
---
./configure
xvfb-run make -j4 coverage-report
make pythoninfo
bash <(curl -s https://codecov.io/bash) -y .github/codecov.yml
---

> Maybe CFLAGS_NODIST should be used instead of CFLAGS.
Looks like it worked.

> Python is built GCC with -ftest-coverage option.
More exact description is: Python is built GCC with --coverage option, --coverage option including -ftest-coverage option.

> Looks like it worked.
Oh, sorry, I checked the wrong gate.

After `CFLAGS` replcaced by `CFLAGS_NODIST`, some extension module built failed, some info like:
*** WARNING: renaming "_struct" since importing it failed: build/lib.linux-x86_64-3.9/_struct.cpython-39-x86_64-linux-gnu.so: undefined symbol: __gcov_merge_add

the possible reason:
building '_struct' extension
gcc -pthread -fPIC -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -Werror=implicit-function-declaration -fvisibility=hidden -O0 -pg --coverage -I./Include/internal -I./Include -I. -I/usr/local/include -I/temp/shihai/cpython/Include -I/temp/shihai/cpython -c /temp/shihai/cpython/Modules/_struct.c -o build/temp.linux-x86_64-3.9/temp/shihai/cpython/Modules/_struct.o
gcc -pthread -shared (xxxxxxxxxxlacking --coverage in here xxxxxxxxx) build/temp.linux-x86_64-3.9/temp/shihai/cpython/Modules/_struct.o -L/usr/local/lib -o build/lib.linux-x86_64-3.9/_struct.cpython-39-x86_64-linux-gnu.so

FWIW, gcc Instrumentation Options in https://gcc.gnu.org/onlinedocs/gcc/Instrumentation-Options.html

I understand that with PR 19460, code coverage is not longer run on C extension modules of the stdlib. I'm not using C code coverage, so I don't know how it's supposed to be used or analyzed.

Hai: if you want to move on this issue, you have to find who uses this CI and how it should behave.

The other option is to only fix test_distutils.

> Hai: if you want to move on this issue, you have to find who uses this CI and how it should behave.

Copy that. I found PR7773 updated the travis' coverage ci and a discussion issue in https://github.com/python/core-workflow/issues/2, so I add brett and ammar to the noisy list, MAYBE they know the historical background.

The historical background is code coverage of C code seemed like a good idea, so we set it up. :) Not much else to it.

> The historical background is code coverage of C code seemed like a good idea, so we set it up. :) Not much else to it.

Hm, code coverage of C code including two parts:
1) coverage of inner c code;
2) extension modules built by distutils;

If I understand correct, we force on 1) is good enough, no?

Brett:
> The historical background is code coverage of C code seemed like a good idea, so we set it up. :) Not much else to it.

If there is no user of code coverage *on pull requests*, maybe the simplest fix is to remove this job.

There is also Codecov service running somehow on Python branches through GitHub integration.

It seems like results at published at: https://codecov.io/gh/python/cpython/

It seems like there is coverage on both C and Python code. I never used this service.

> It seems like there is coverage on both C and Python code.

Great, MAYBE we can use this coverage result to improve our testcases.

See also bpo-40993: "Don't run Python and C coverage jobs of Travis CI on pull requests".

See also Coverage report in Github CI for standard library discussion:
https://discuss.python.org/t/coverage-report-in-github-ci-for-standard-library/2836

Another issue: the "Test code coverage (C)" job on Travis CI times out after 50 minutes:
https://travis-ci.org/github/python/cpython/jobs/699170857

0:44:44 load avg: 7.76 running: test_multiprocessing_spawn (11 min 38 sec), test_concurrent_futures (9 min 33 sec), test_unparse (25 min 7 sec), test_capi (1 min)
0:45:14 load avg: 7.91 running: test_multiprocessing_spawn (12 min 8 sec), test_concurrent_futures (10 min 3 sec), test_unparse (25 min 37 sec), test_capi (1 min 30 sec)
0:45:44 load avg: 8.21 running: test_multiprocessing_spawn (12 min 38 sec), test_concurrent_futures (10 min 33 sec), test_unparse (26 min 7 sec), test_capi (2 min)
0:46:14 load avg: 7.33 running: test_multiprocessing_spawn (13 min 8 sec), test_concurrent_futures (11 min 3 sec), test_unparse (26 min 37 sec), test_capi (2 min 30 sec)

(...)

The job exceeded the maximum time limit for jobs, and has been terminated.

> Test code coverage (C) job of Travis CI fails on test_distutils which creates _configtest.gcno file

This Travis CI has been removed, I close the issue.

stage: patch review -> resolved