SonarQube Plugins

Coverage metrics for Python code

Details

  • Type: New Feature New Feature
  • Status: Closed Closed
  • Priority: Major Major
  • Resolution: Fixed
  • Affects Version/s: None
  • Fix Version/s: PYTHON-1.1
  • Component/s: Python
  • Labels:
    None
  • Number of attachments :
    2

Description

Provide coverage metrics for python code.

Design
------
Pythons ecosystem offers a couple of tools for measuring code coverage, the major ones being coverage by Ned Batchelder and figleaf by C. Titus Brown. Both are independent of the way the code is run; there is no canonical way to 'run the code under coverage'. Therefore, a data-driven approach seems to be the right one: define some data format as the interface and provide a way to feed it into Sonar via the Python plugin. Cobertura XML seems to a good choice for such a format because its quite popular and is already supported by Ned's coverage package.

To create a cobertura report for a set of unit test do something like:
$ nosetests --with-coverage
$ coverage xml

Activity

Ascending order - Click to sort in descending order
Hide
Permalink
Waleri Enns added a comment -

implemented with: e0c6472e8855995b68ddafd0fb4fad8c858937a2

Show
Waleri Enns added a comment - implemented with: e0c6472e8855995b68ddafd0fb4fad8c858937a2
Hide
Permalink
Evgeny Mandrikov added a comment -

Reopened for verification.

Show
Evgeny Mandrikov added a comment - Reopened for verification.
Hide
Permalink
Evgeny Mandrikov added a comment -

Hi Waleri,

I reopen this ticket, because behaviour seems strange for me:
SONARPLUGINS-2154.zip contains two projects "ok" and "nok". For first one - everything is fine. But for second - I wonder why dashboard shows that there 100% coverage, whereas clearly that there is one file without coverage. Maybe I'm doing something wrong and this example is incorrect. Could you please clarify this?

Show
Evgeny Mandrikov added a comment - Hi Waleri, I reopen this ticket, because behaviour seems strange for me: SONARPLUGINS-2154.zip contains two projects "ok" and "nok". For first one - everything is fine. But for second - I wonder why dashboard shows that there 100% coverage, whereas clearly that there is one file without coverage. Maybe I'm doing something wrong and this example is incorrect. Could you please clarify this?
Hide
Permalink
Waleri Enns added a comment -

Ahh... I see. The reason for this is described in the Wiki, in short:

"""
Make sure to put all packages to measure into the --source parameter. That ensures that coverage will report zero coverage on all untouched files, as you most probably want to. To make this work, make sure to meet two prerequisites:

use coverage => 3.4
the packages should be packages, i.e. every directory on the way to the sources should contain an _init_.py file.
"""

Some background: by default, coverage tracks only files which have been touched while executing the program/tests.
To make all the untouched files count with coverage=0, as the user most probably wants to, you have to call coverage as described in the Wiki.
Moreover, AFAIK there is no other way to pass the "--source=<package>" parameter other way, e.g. via .coveragerc or nose command line parameters.
Thats cumbersome, yes, but that the only way I know of.

For the exact command line look in the modified archive (attaching now...)

Show
Waleri Enns added a comment - Ahh... I see. The reason for this is described in the Wiki, in short: """ Make sure to put all packages to measure into the --source parameter. That ensures that coverage will report zero coverage on all untouched files, as you most probably want to. To make this work, make sure to meet two prerequisites: use coverage => 3.4 the packages should be packages, i.e. every directory on the way to the sources should contain an _ init _.py file. """ Some background: by default, coverage tracks only files which have been touched while executing the program/tests. To make all the untouched files count with coverage=0, as the user most probably wants to, you have to call coverage as described in the Wiki. Moreover, AFAIK there is no other way to pass the "--source=<package>" parameter other way, e.g. via .coveragerc or nose command line parameters. Thats cumbersome, yes, but that the only way I know of. For the exact command line look in the modified archive (attaching now...)
Hide
Permalink
Waleri Enns added a comment -
  • modified run.sh
  • added src/_init_.py
Show
Waleri Enns added a comment - modified run.sh added src/_ init _.py
Hide
Permalink
Evgeny Mandrikov added a comment -

Ok, I see what you mean. Thanks for clarification.

Show
Evgeny Mandrikov added a comment - Ok, I see what you mean. Thanks for clarification.
Hide
Permalink
Freddy Mallet added a comment -

@Waleri, to be sure of my understanding, am I right by saying :

  • It's up to the Sonar user to execute unit tests and generate a coverage report on his python project
  • The coverage report must comply with the Cobertura XML format (whatever python coverage tool is used)
  • Then the property sonar.python.coverage.reportPath allows to define where this coverage report is located
Show
Freddy Mallet added a comment - @Waleri, to be sure of my understanding, am I right by saying : It's up to the Sonar user to execute unit tests and generate a coverage report on his python project The coverage report must comply with the Cobertura XML format (whatever python coverage tool is used) Then the property sonar.python.coverage.reportPath allows to define where this coverage report is located
Hide
Permalink
Waleri Enns added a comment -

Yep. You scored 3 points

Show
Waleri Enns added a comment - Yep. You scored 3 points

People

  • Assignee:
    Waleri Enns
    Reporter:
    Waleri Enns
Vote (1)
Watch (3)

Dates

  • Created:
    Updated:
    Resolved: