Cuanto
  1. Cuanto
  2. CUANTO-70

Randomly Cuanto reports fewer test results than actually ran on the "Test Run History for <group/project>" page.

    Details

    • Type: Bug Bug
    • Status: Closed Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 2.7.1
    • Fix Version/s: 2.8.0
    • Component/s: Database
    • Labels:
      None
    • Environment:
      Ubuntu 8.04.4 LTS on a P4 @ 2.80GHz w/ 3 Gigs of RAM
    • Number of attachments :
      0

      Description

      Randomly (about every week) Cuanto reports fewer test results than actually ran on the "Test Run History for <group/project>" page. The problem happens with multiple group/projects. We're submitting via the API.

        Activity

        Hide
        Keith Felton added a comment -

        Todd, we're not running 2.8.0b1 yet. We're still running 2.7.1.

        Show
        Keith Felton added a comment - Todd, we're not running 2.8.0b1 yet. We're still running 2.7.1.
        Hide
        dawn hemminger added a comment -

        We recently upgraded to 2.8b2 and we're still seeing this problem. I took some snapshots if you want to see them. I don't see a way to attach them in this interface. I also don't know how to reopen an issue, so I hope you get this comment.

        Show
        dawn hemminger added a comment - We recently upgraded to 2.8b2 and we're still seeing this problem. I took some snapshots if you want to see them. I don't see a way to attach them in this interface. I also don't know how to reopen an issue, so I hope you get this comment.
        Hide
        Todd Wells added a comment -

        Is there any matter to the incorrect totals? For instance, the total not including skipped tests or something like that? Can you tell if one subset of the results is wrong? Like passed/failed/skipped? Like if you look at the total "passed" in the summary does it match the total passed when you add them up in the analysis table? Or are all the values randomly incorrect?

        Show
        Todd Wells added a comment - Is there any matter to the incorrect totals? For instance, the total not including skipped tests or something like that? Can you tell if one subset of the results is wrong? Like passed/failed/skipped? Like if you look at the total "passed" in the summary does it match the total passed when you add them up in the analysis table? Or are all the values randomly incorrect?
        Hide
        dawn hemminger added a comment -

        In this case, there were 397 tests run. We're using the cuanto API with a wrapper we created to write the tests to Cuanto. All tests ran and all passed (no fails, no skips, etc). I manually counted the number of tests displayed under the analysis tab and it's 397 just like the analysis page states on the bottom left corner, so all tests are accounted for. It's only when you're looking at the "Most recent runs" or "Test Runs History" view, that it's showing 396 tests run and passed. In the analysis view, if I hit "recalc" and then go back to the "most recent runs" view and refresh a few times, the count eventually comes up to 397 as expected.

        Note, this is similar to the problem that still exists in this build where when you enter any analysis information, e.g. "Test Bug" under the "Reason" column, the analyzed results do not show up in the "most recent runs" or "Test Runs History" view. You have to hit "recalc" in the Analysis view first to get that analyzed data to update in those other views.

        For this particular issue, I'm running Firefox 7.0.1.

        Show
        dawn hemminger added a comment - In this case, there were 397 tests run. We're using the cuanto API with a wrapper we created to write the tests to Cuanto. All tests ran and all passed (no fails, no skips, etc). I manually counted the number of tests displayed under the analysis tab and it's 397 just like the analysis page states on the bottom left corner, so all tests are accounted for. It's only when you're looking at the "Most recent runs" or "Test Runs History" view, that it's showing 396 tests run and passed. In the analysis view, if I hit "recalc" and then go back to the "most recent runs" view and refresh a few times, the count eventually comes up to 397 as expected. Note, this is similar to the problem that still exists in this build where when you enter any analysis information, e.g. "Test Bug" under the "Reason" column, the analyzed results do not show up in the "most recent runs" or "Test Runs History" view. You have to hit "recalc" in the Analysis view first to get that analyzed data to update in those other views. For this particular issue, I'm running Firefox 7.0.1.
        Hide
        Todd Wells added a comment -

        I've disabled caching for TestRunStats in commit 937a0ec in an attempt to resolve this issue.

        Show
        Todd Wells added a comment - I've disabled caching for TestRunStats in commit 937a0ec in an attempt to resolve this issue.

          People

          • Assignee:
            Todd Wells
            Reporter:
            Keith Felton
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: