GRECLIPSE

performance poor for large projects

Details

  • Number of attachments :
    2

Description

Raised by Marcel on the mailing list, with some figures:
The project consists of:

3718 java files
1100 groovy files

Test case is a clean rebuild of the project.

  • Eclipse is 3.5.1, win32, V2 plugin 20091016-1800-e35
    compile time: 2 hours 35 minutes (yes TWO hours and 35 minutes!!!!!)

Debug output:
>FULL BUILD STATS for: server
> compiled 679098 lines in 9191856ms:73.8lines/s
> parse: 16936 ms (0.1%), resolve: 38667 ms (0.4%), analyze: 9649 ms (0.1%), generate: 9139251 ms (99.4%)
Recording new state : State for server (#0 @ Sat Oct 17 15:46:10 CEST 2009)
Finished build of server @ Sat Oct 17 18:20:06 CEST 2009

  • Eclipse 3.5.1, win32, V1 plugin 1.6.3.200907102326NGT
    compile time: 5 minutes (also slow compared to a java build)
  • command line, ant javac/groovyc, 1.7-beta-2:
    compile time: 2 minutes (this speed would be a dream in eclipse)
  • command line, ant javac/groovyc, 1.6.5:
    compile time: 2 minutes

I think there is something fundamentally broken in the V2 plugin ;-(. When compiling, the CPU usage is about 30% with heavy disk IO. The V1 plugin has much lower disk activity when compiling. The groovyc ant task has the lowest disk activity.

Perhaps the OSGI integration is broken. I have seen this in other projects. There are perhaps problems with classloading.

Hope you could improve something,

Marcel

—
and a profile stack trace:

99,9% - 232s - 1 inv. org.eclipse.jdt.internal.compile.ProcessTaskManager.run
86,5% - 201 s - 25 inv. org.codehaus.jdt.goovy.internal.compiler.ast.GroovyCompilationUnitDeclaration.generateCode
85,6% - 199 s - 25 inv. org.codehaus.jdt.goovy.internal.compiler.ast.GroovyCompilationUnitDeclaration.processToPhase
85,6% - 199 s - 25 inv. org.codehaus.goovy.control.CompilationUnit.compile
62,3% - 145 s - 264 inv. org.codehaus.goovy.control.CompilationUnit.applyToPrimaryClassNodes
59,7% - 139 s - 264 inv. org.codehaus.goovy.control.CompilationUnit.getPrimaryClassNodes
50,3% - 117 s - 48 inv. org.codehaus.goovy.control.CompilationUnit.getSorted
25,1% - 58 s - 29225160 inv. java.util.List.size

Activity

Ascending order - Click to sort in descending order
Hide
Permalink
Andy Clement added a comment -

After thinking about this, I've always known that the stack Marcel included is quite expensive - however, it may be obscenely expensive because of how we are driving CompilationUnits under eclipse.

I'll slap a patch in here that shortcuts a piece of function - if that helps we can look to properly solve the issue.

Show
Andy Clement added a comment - After thinking about this, I've always known that the stack Marcel included is quite expensive - however, it may be obscenely expensive because of how we are driving CompilationUnits under eclipse. I'll slap a patch in here that shortcuts a piece of function - if that helps we can look to properly solve the issue.
Hide
Permalink
Andy Clement added a comment -

This patch contains a few classes to replace those in the org.codehaus.groovy plugin. To apply it, go into your most recent version of the plugin under your eclipse/plugins folder. for example mine is

eclipse\plugins\org.codehaus.groovy_1.7.0.xx-20091002-1800-e35

once in that folder:

0) shutdown eclipse
1) backup the existing groovy-eclipse.jar
2) unpack the patch: jar -xvf patch.zip
3) apply the patch: jar -uvf groovy-eclipse.jar org
4) restart eclipse - full build - and see if there is any difference.

Show
Andy Clement added a comment - This patch contains a few classes to replace those in the org.codehaus.groovy plugin. To apply it, go into your most recent version of the plugin under your eclipse/plugins folder. for example mine is eclipse\plugins\org.codehaus.groovy_1.7.0.xx-20091002-1800-e35 once in that folder: 0) shutdown eclipse 1) backup the existing groovy-eclipse.jar 2) unpack the patch: jar -xvf patch.zip 3) apply the patch: jar -uvf groovy-eclipse.jar org 4) restart eclipse - full build - and see if there is any difference.
Hide
Permalink
Andy Clement added a comment -

New patch - can be applied over the old one.

Show
Andy Clement added a comment - New patch - can be applied over the old one.
Hide
Permalink
Marcel Wagner added a comment -

The new patch works for me, the compile speed is now normal.

Show
Marcel Wagner added a comment - The new patch works for me, the compile speed is now normal.
Hide
Permalink
Andy Clement added a comment -

All changes now committed (will be in a dev build later today).

Key changes were:

  • removal of the output phase was failing so the same classes were being written out multiple times
  • sorting of classes was slow, but worse than that it was run far too frequently.

If you have any time Marcel... I'd love to know the precise timing differences now between inside eclipse and outside of eclipse?

Show
Andy Clement added a comment - All changes now committed (will be in a dev build later today). Key changes were: removal of the output phase was failing so the same classes were being written out multiple times sorting of classes was slow, but worse than that it was run far too frequently. If you have any time Marcel... I'd love to know the precise timing differences now between inside eclipse and outside of eclipse?
Hide
Permalink
Predrag Knezevic added a comment -

After upgrading to 2.0.0.xx-20091108 I have not recognized any improvements regarding the compilation speed. I am working currently on a mid-size Grails project (118 Groovy classes, 222 Java classes). When building the workspace - I easily reach only 10%, and the processor load is constantly high (approx 80% on a dual core machine).

Any suggestions?

Show
Predrag Knezevic added a comment - After upgrading to 2.0.0.xx-20091108 I have not recognized any improvements regarding the compilation speed. I am working currently on a mid-size Grails project (118 Groovy classes, 222 Java classes). When building the workspace - I easily reach only 10%, and the processor load is constantly high (approx 80% on a dual core machine). Any suggestions?
Hide
Permalink
Andy Clement added a comment -

By 'reach 10%' - do you mean the progress bar? Currently the progress bar does not accurately reflect the progress through the build as all the groovy code is treated as 'one thing' - so it will jump from the point it hits the groovy code to 100%.

The aim of this bug to fix compile speed is to make compilation speed inside eclipse as close to compile speed outside eclipse as we can - are you not seeing that?

Show
Andy Clement added a comment - By 'reach 10%' - do you mean the progress bar? Currently the progress bar does not accurately reflect the progress through the build as all the groovy code is treated as 'one thing' - so it will jump from the point it hits the groovy code to 100%. The aim of this bug to fix compile speed is to make compilation speed inside eclipse as close to compile speed outside eclipse as we can - are you not seeing that?
Hide
Permalink
Predrag Knezevic added a comment -

I have tried again with the latest available version (2.0.0.xx-20091120) and I must say that compiling is fast! It takes only a few seconds on my project.

Happy again

Show
Predrag Knezevic added a comment - I have tried again with the latest available version (2.0.0.xx-20091120) and I must say that compiling is fast! It takes only a few seconds on my project. Happy again
Hide
Permalink
Marcel Wagner added a comment -

Nice job, my performance data:

Project size:
3735 java files
1103 groovy files

ANT build with javac/groovyc: compile time 1 min 37s
groovy eclipse (2.0.0.xx-20091203-1300-e35): compile time 1 min 24s

The plugin seems now 10% faster then the command line tools

Show
Marcel Wagner added a comment - Nice job, my performance data: Project size: 3735 java files 1103 groovy files ANT build with javac/groovyc: compile time 1 min 37s groovy eclipse (2.0.0.xx-20091203-1300-e35): compile time 1 min 24s The plugin seems now 10% faster then the command line tools

People

  • Assignee:
    Andy Clement
    Reporter:
    Andy Clement
Vote (2)
Watch (2)

Dates

  • Created:
    Updated:
    Resolved: