jira.codehaus.org

  • Log In Access more options
    • Online Help
    • Keyboard Shortcuts
    • About JIRA
    • JIRA Credits
    • What?s New
  • Dashboards Access more options (Alt+d)
  • Projects Access more options (Alt+p)
  • Issues Access more options (Alt+i)
Signup
Sonar Plugins
  • Sonar Plugins
  • SONARPLUGINS-1637

Integration of unit tests

  • Log In
  • Views
    • XML
    • Word
    • Printable

Details

  • Type: New Feature New Feature
  • Status: Closed Closed
  • Priority: Major Major
  • Resolution: Fixed
  • Affects Version/s: None
  • Fix Version/s: PYTHON-1.1
  • Component/s: Python
  • Labels:
    None
  • Number of attachments :
    0

Description

Provide metrics considering the execution of unit test, like number of errors/failures, success rate etc.

Design
------
There is a lot of variety when it comes to running automated test for Python. That variety is caused by differences in source layout, environment setup, used test framework etc.
Long story short: We are outside of the java/maven universe, we cannot make strong assumptions about projects layout etc. and thus cannot provide a fully automated solution in the plugin. Instead, we leave the aspect that varies – calling the tests and capturing the output – on the project's site, where it belongs to. The plugin is only responsible for parsing the report and feeding the data into Sonar. We choose JUnitReport XML format because of his popularity.

The format has some drawbacks though, most notably:
1. it assumes that each function(=method) belongs to a class (Its originates from a "J"-tool, after all)
2. it doesnt explicitly provide the path to the source file (That hurts in every environment where you cannot definitely map the class name to a source file)

Despite this drawbacks it should be a goal to not move away from this (quasi)standard format. The plan for dealing with the second drawback is:
a) use the knowledge of the lexer/parser to map the class names to according source files. That should work in most cases.
b) as a fallback, provide the possibility to inject the path to the source file via an OPTIONAL 'source' attribute of the 'testcase' tag.
c) If the source file cannot be found using both ways, just create a 'virtual' one with the content "sources cannot be found" or similar.
d) To be precise about our format expectations, have a reference to a grammar in the dox which can be used to verify the validity of a report.

Activity

Ascending order - Click to sort in descending order
  • All
  • Comments
  • Work Log
  • History
  • Activity
Hide
Permalink
Evgeny Mandrikov added a comment - 18/Jul/12 11:12 AM

For the record: nose is one of the tools, which is in use.

Show
Evgeny Mandrikov added a comment - 18/Jul/12 11:12 AM For the record: nose is one of the tools, which is in use .
Hide
Permalink
Evgeny Mandrikov added a comment - 06/Aug/12 10:42 AM

Also see https://github.com/cmheisel/nose-xcover which is an extension to nose and allows to produce coverage report in Cobertura style.

Show
Evgeny Mandrikov added a comment - 06/Aug/12 10:42 AM Also see https://github.com/cmheisel/nose-xcover which is an extension to nose and allows to produce coverage report in Cobertura style.
Hide
Permalink
Evgeny Mandrikov added a comment - 05/Sep/12 12:59 AM

Hi Waleri, why this issue still marked as "in progress" whereas code was committed?

Show
Evgeny Mandrikov added a comment - 05/Sep/12 12:59 AM Hi Waleri, why this issue still marked as "in progress" whereas code was committed?
Hide
Permalink
Waleri Enns added a comment - 05/Sep/12 1:11 AM

Cause its not finished yet. The part of locating the resources is still in work. Will finish it this or next week, I guess.
FYI: this is then the last peace which was on my sheet for the next version of the plugin.

Show
Waleri Enns added a comment - 05/Sep/12 1:11 AM Cause its not finished yet. The part of locating the resources is still in work. Will finish it this or next week, I guess. FYI: this is then the last peace which was on my sheet for the next version of the plugin.
Hide
Permalink
Waleri Enns added a comment - 20/Sep/12 1:59 AM

resolved with commit 774afc68c4c796ef390937740c59b770d3eba06f
The extension of the report format by the optional 'path' field has been left out: lets see if the need arises in practice.

Show
Waleri Enns added a comment - 20/Sep/12 1:59 AM resolved with commit 774afc68c4c796ef390937740c59b770d3eba06f The extension of the report format by the optional 'path' field has been left out: lets see if the need arises in practice.
Hide
Permalink
Freddy Mallet added a comment - 26/Sep/12 4:26 AM

@Waleri, just to be sure of my understanding , Am I right by saying :

  • It's up to the Sonar user to execute Python unit tests
  • It's up to the Sonar user to generate a unit test execution report complying with JUnit report XML format (whatever tool is used to do that)
  • The property "sonar.python.xunit.reportPath" must be define to provide the location of this unit test execution report
Show
Freddy Mallet added a comment - 26/Sep/12 4:26 AM @Waleri, just to be sure of my understanding , Am I right by saying : It's up to the Sonar user to execute Python unit tests It's up to the Sonar user to generate a unit test execution report complying with JUnit report XML format (whatever tool is used to do that) The property "sonar.python.xunit.reportPath" must be define to provide the location of this unit test execution report
Hide
Permalink
Waleri Enns added a comment - 26/Sep/12 4:33 AM

Yes, thats correct.

Show
Waleri Enns added a comment - 26/Sep/12 4:33 AM Yes, thats correct.

People

  • Assignee:
    Waleri Enns
    Reporter:
    Waleri Enns
Vote (2)
Watch (4)

Dates

  • Created:
    22/Jan/12 7:17 AM
    Updated:
    01/Oct/12 9:01 AM
    Resolved:
    20/Sep/12 1:59 AM
  • Atlassian JIRA (v5.2.7#850-sha1:b2af0c8)
  • Report a problem
  • Powered by a free Atlassian JIRA open source license for Codehaus. Try JIRA - bug tracking software for your team.