So I think what we want is:
for the speed up rates:
run BaseBase, production_Opt_0, production_Opt_1, production_Opt_2 on 20 iterations of the entire SPEC JVM and measure the total execution time
(production_Opt_X implies the boot image is compiled at level X and we disable adaptive recompilation and use an IRC of Opt X)
we calculate the speed up rates as the total execution time for each run divided by the total execution time for the BaseBase run.
for the compilation rates:
run a production image with no recompilation the IRC as base, O0, O1 and O2 on the entire SPEC JVM but for just 1 iteration as we don't actually care about the execution time
the compilation rate is the bcb/ms reported for base as the baseline rate, and for O0..O2 as the opt compiler bcb/ms in the -X:vm:measureCompilation report.
Given that I think this is what we want, I'm unclear as to:
why in the current compiler-dna we run SPEC JVM benchmarks individually?
why the current compiler-dna doesn't do BaseBase runs?
what we can do to avoid the floating point remainder bug in _200_check (RVM-352) ?
what we can do to avoid the GC crash when running 20 iterations of SPEC JVM (RVM-355) ?
I've scripted extracting the compilation rates to appear in reports. I believe the next steps are to make the runs as I describe them above and possibly write something that will extract the statistics from the report(s?). I think in all cases we need to document somewhere the compiler DNA gathering process.
We have in the tools directory the perl scripts for computeCompilerDNA and in the build/test-runs we have a compiler-dna properties which states:
When running this properties file the some what surprising message is shown during build-bootimage:
[java] BootImageWriter: compiler arg: O1
so not only is the test measuring the performance of the compiler running at say O1, its also boot image compiling itself with O1. This doesn't make sense to me as we want to know the performance of the O1 compiler when it itself has been compiled as O2. Is the test run wrong?
- The compiler-dna test-run is used to measure data on how effective
- the various levels of optimization in the opt compiler in terms of speedup
- and how much they cost in terms of compile time.
- Although this whole process could perhaps be done as an ant task for now
- I'm just defining an ant task to build/run the images to get the raw
- data and then will manurally run the perl script in
- RVM_ROOT/tools/computeCompilerDNA over the output files to get the
- actual DNA.
When running this properties file the some what surprising message is shown during build-bootimage: [java] BootImageWriter: compiler arg: O1 so not only is the test measuring the performance of the compiler running at say O1, its also boot image compiling itself with O1. This doesn't make sense to me as we want to know the performance of the O1 compiler when it itself has been compiled as O2. Is the test run wrong?