RVM
  1. RVM
  2. RVM-363

Improve the speed of cat track's graphs

    Details

    • Number of attachments :
      0

      Description

      Loading the performance graphs from cat track can be very slow. I believe the google chart API [1] may be faster as the rendering job can be moved closer to where the client is. I'm not sure if this is a good idea or we could solve the performance problem another way.

      [1] http://code.google.com/apis/chart/

        Issue Links

          Activity

          Hide
          Peter Donald added a comment -

          The amount of data we use to generate the graphs probably precludes using the google charts api. One of the main reasons for the slowness is that the web server is effectively single threaded due to the way it is set up but it would be easy enough to fix (see RVM-307).

          One option to make the slowness less jarring is to actually specify height-width on all images so the page does not jump about.

          Another option is to generate the images when the data is uploaded which would mean that it would be effectively apache serving everything but 3-4 pages and this would be about as fast as we could get it with the setup.

          Show
          Peter Donald added a comment - The amount of data we use to generate the graphs probably precludes using the google charts api. One of the main reasons for the slowness is that the web server is effectively single threaded due to the way it is set up but it would be easy enough to fix (see RVM-307 ). One option to make the slowness less jarring is to actually specify height-width on all images so the page does not jump about. Another option is to generate the images when the data is uploaded which would mean that it would be effectively apache serving everything but 3-4 pages and this would be about as fast as we could get it with the setup.
          Hide
          Steve Blackburn added a comment -

          THanks for the pointer, Ian. It looks like a nice API indeed. We (at ANU) have been searching for decent graphing APIs.

          I agree with Peter's analysis.

          However, I think the real issue is the lack of a summary page, which we've always wanted.

          We cannot expect Daniel to do that at the moment---he's at crunch (do-or-die) point for his PhD. Any one of us could have a crack at it though.

          The main issue is how to meaningfully aggregate the vast pool of information. I'd love to have a single sparkline summarizing overall performance. This could be embedded in a 24hrly email, for example. Or even if there were 3 sparklines, that would be great. For example, a line which moved up and down according to the overall number of benchmarks which improved or degraded, or perhaps the number of benchmarks which were within 1 std dev of their best. A dip in this line would indicate an overall dip in performance. These are just off the top of my head. I'm sure there are better metrics, but the truth is that providing a meaningful aggregation of 30 or so time series results is not trivial.

          Show
          Steve Blackburn added a comment - THanks for the pointer, Ian. It looks like a nice API indeed. We (at ANU) have been searching for decent graphing APIs. I agree with Peter's analysis. However, I think the real issue is the lack of a summary page, which we've always wanted. We cannot expect Daniel to do that at the moment---he's at crunch (do-or-die) point for his PhD. Any one of us could have a crack at it though. The main issue is how to meaningfully aggregate the vast pool of information. I'd love to have a single sparkline summarizing overall performance. This could be embedded in a 24hrly email, for example. Or even if there were 3 sparklines, that would be great. For example, a line which moved up and down according to the overall number of benchmarks which improved or degraded, or perhaps the number of benchmarks which were within 1 std dev of their best. A dip in this line would indicate an overall dip in performance. These are just off the top of my head. I'm sure there are better metrics, but the truth is that providing a meaningful aggregation of 30 or so time series results is not trivial.
          Hide
          Steve Blackburn added a comment -

          Looking again right now, I think there must be something badly wrong with the set-up.

          Right now it is taking me in the order of 10 minutes to view a single perf page. Network latency is hardly the issue; it's plugged into the same gigabit switch as my laptop.

          When I run top on the machine I see that mongrel_rails is hogging most of virtual memory and plenty of CPU. postgress is also busy. Is there a problem with our database not scaling very well as our volume of data increases?

          Peter, if you have the chance, could you please check it out? I know virtually nothing about the setup, but from my naive POV, the patient seems unwell!

          Show
          Steve Blackburn added a comment - Looking again right now, I think there must be something badly wrong with the set-up. Right now it is taking me in the order of 10 minutes to view a single perf page. Network latency is hardly the issue; it's plugged into the same gigabit switch as my laptop. When I run top on the machine I see that mongrel_rails is hogging most of virtual memory and plenty of CPU. postgress is also busy. Is there a problem with our database not scaling very well as our volume of data increases? Peter, if you have the chance, could you please check it out? I know virtually nothing about the setup, but from my naive POV, the patient seems unwell!
          Hide
          Peter Donald added a comment -

          It looks like the time is being spent in the image rendering code. There is also significant memory leaks (not in our code it seems?) which is causing gc grinding. I think that we could probably cleanup the rendering code so less inefficient but we could also tune postgres much better. For the moment I have set it to restart mongrel once a day which should reduce some of the performance problems due to mem exhaustion.

          However I will try to look at cleaning up the image code within the next week or so.

          Show
          Peter Donald added a comment - It looks like the time is being spent in the image rendering code. There is also significant memory leaks (not in our code it seems?) which is causing gc grinding. I think that we could probably cleanup the rendering code so less inefficient but we could also tune postgres much better. For the moment I have set it to restart mongrel once a day which should reduce some of the performance problems due to mem exhaustion. However I will try to look at cleaning up the image code within the next week or so.
          Hide
          Erik Brangs added a comment -

          Removing assignees of all issues where the assignee has not contributed to Jikes RVM for quite some time or where the assignee is no longer a member of the core team.

          If you are affected and would like to work on one of the issues, please drop me a note (or change the assignee yourself if you have the necessary permissions).

          Show
          Erik Brangs added a comment - Removing assignees of all issues where the assignee has not contributed to Jikes RVM for quite some time or where the assignee is no longer a member of the core team. If you are affected and would like to work on one of the issues, please drop me a note (or change the assignee yourself if you have the necessary permissions).

            People

            • Assignee:
              Unassigned
              Reporter:
              Ian Rogers
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated: