Details
-
Type:
Improvement
-
Status:
Open
-
Priority:
Major
-
Resolution: Unresolved
-
Affects Version/s: None
-
Fix Version/s: None
-
Component/s: User Interface
-
Labels:None
-
Number of attachments :
Description
I find myself desiring the ability to look at a project group and determine the overall health (with respect to test runs) of the group. For example, a group may represent a single product, with each project being test run containers for different components of the product. Being able to see, at a single glance, the historic health of the test runs in every project would be helpful. Currently, the group view displays a table of the recent runs, but that does not provide the historic view of the health of the project.
I'm envisioning the new feature as a multi-line chart that appears in the group view, in which a single line represents a single project, the y axis is the pass rate, and the x axis is time. The problem, however, is what granularity of time to choose (day, week, month).
Another interesting and potentially helpful information this new feature would provide is the frequency of test runs for every project. For example, if one project is consistently passing at a very high rate and is being regressed frequently, whereas the pass rate on another project is fluctuating significantly but is regressed rather infrequently, one may wish to adjust the frequencies of test runs on these projects.
Hi Cho: I'm curious what you would do with this information. Would you use it to help identify projects with unstable tests? Or maybe to weigh how much one should trust all-pass outcomes?
My pie-in-the-sky feature for Cuanto is for it to automate the analysis as much as possible. If there is some trending that indicates useful information, I want it to show it to me first. Otherwise I have to try and draw my own conclusions from (possibly) a multitude of data. (I did say pie-in-the-sky.)
We have also wanted more information at different levels, though not exactly what is asked for here. Whatever way it goes, more visualizations and analysis tools would help.