[Python-Dev] We now have C code coverage!
On 6/22/2018 9:40 PM, Nathaniel Smith wrote:
> On Fri, Jun 22, 2018 at 6:16 PM, Terry Reedy <tjreedy at udel.edu> wrote:
>> I discovered the answer to 2. by shift-clicking on a text_x file to see
>> their coverage report for the file. The colors actually do reflect the test
>> lines executed. codecov.io excludes gui tests*, so the reported coverage
>> for tkinter, idlelib, and turtle is deceptive and bogus, and under-reports
>> the total cpython coverage by a percent or two. It would be better to
>> exclude these modules.
>> * I assume that codecov.io uses linux servers. I have read that there are
>> programs that simulate X-Windows so that gui code will execute without
>> actual terminals.
> Codecov.io doesn't run any tests itself; it's just a service for
> aggregation and reporting. The coverage information is being gathered
> while running CPython's regular CI tests, and then uploaded to
> codecov.io to view.
Thank you for the information.
> So if you want to run the gui tests -- which seems like a good idea if
> possible! -- then the way to do that would be to make them run as part
> of the regular Travis/Appveyor/VSTS checks.
I have suggested that, and before that, the same for buildbots. The
reality is that tkinter, IDLE, or turtle could be disabled on *nix by
regressions and the official testing would not notice.
Terry Jan Reedy