Python code coverage not working for multiprocessing inside site-packages

Question

I have a simple code packages in a module/folder "src" File sample.py

import multiprocessing

def f(x):
    return x*x

def big_run(n):
    with multiprocessing.Pool(5) as p:
        p.map(f, list(range(n)))
        p.close()
        p.join()

File __init__.py

from .sample import big_run

And I test it with a file test.py

from src import big_run

def test_main():
    big_run(10)

Just copy the folder to /mnt/data/global_envs/seb_py310/lib/python3.10/site-packages I use pytest with pytest.ini

[pytest]
python_files = test*.py
required_plugins = pytest-cov pytest-xdist pytest-env

[coverage:run]
concurrency = multiprocessing
branch = true
parallel = true
sigterm = true

And the command line python -m pytest . --cov=src --cov-branch --cov-report=term --cov-config=../../../pytest.ini

The code coverage report is

---------- coverage: platform linux, python 3.10.6-final-0 -----------
Name                                                                           Stmts   Miss Branch BrPart  Cover
----------------------------------------------------------------------------------------------------------------
/mnt/data/global_envs/seb_py310/lib/python3.10/site-packages/src/__init__.py       1      0      0      0   100%
/mnt/data/global_envs/seb_py310/lib/python3.10/site-packages/src/sample.py         8      1      2      0    90%
----------------------------------------------------------------------------------------------------------------
TOTAL                                                                              9      1      2      0    91%

And if I just put the folder next to the test.py (not inside site-packages) I get 100% code coverage. The code that is not covered is the line return x*x. If I don't use multiprocessing, obviously, there is no issue.

Any idea why I can't get 100% code coverage when it is inside site-packages? I care about this because I am building a python library that uses multiprocessing.pool and I can't get the right coverage level.

I tried moving the code in many other folders, the issue is only with site-packages. I also tried copying the whole content of site-packages to another folder such as /mnt/data/global_envs/seb_py310/lib/python3.10/sites and I still get 100% code coverage. I also looked into other settings for pytest.ini and using plain coverage but nothing worked. I also tried this https://stackoverflow.com/a/41365659/11598039

Ned Batchelder · Accepted Answer · 2025-05-13 19:12:37Z

1

coverage.py tries hard to not measure code in site-packages because generally that is third-party code that you don't want to measure. In addition, you are using --cov=src which means to measure the code in the src directory. Manually copying code into site-packages seems very unusual.

Perhaps you want to use pip install -e . to install your working directory as the libraries runnable code?

answered May 13 at 19:12

Ned Batchelder

378k77 gold badges583 silver badges675 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Sébastien Eskenazi May 14 at 2:09

Yes, normally, I have pyproject.toml and then I build the lib and install it. That gives the same results and would have been more complicated to describe. My point is that I want to test the installed library. --cov=src actually means that we want to cover the code for the src module (not directory) as if the module src is not imported it throws a warning saying that the module is not imported.

Sébastien Eskenazi May 14 at 2:16

Yes, normally, I have pyproject.toml and then I build the lib and install it. That gives the same results and would have been more complicated to describe. My point is that I want to test the installed library. This is why I am doing code coverage for the code in site-packages. --cov=src actually means that we want to cover the code for the src module (not directory) as if the module src is not imported pytest-cov throws a warning saying that the module is not imported.

Ned Batchelder May 14 at 10:13

It's unusual to have a module named src also, that directory usually contains the importable module, it's not importable itself. Maybe we should move this to a coverage issue: github.com/nedbat/coveragepy/issues

Sébastien Eskenazi May 15 at 8:23

created an issue here github.com/nedbat/coveragepy/issues/1968

Collectives™ on Stack Overflow

Python code coverage not working for multiprocessing inside site-packages

1 Answer 1

4 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related