RVM

Reducing file copying when running SPECjvm98

Details

  • Type: Improvement Improvement
  • Status: Closed Closed
  • Priority: Minor Minor
  • Resolution: Won't Fix
  • Affects Version/s: None
  • Fix Version/s: None
  • Component/s: Infrastructure: Test
  • Labels:
    None
  • Number of attachments :
    0

Description

The current test system copies large portions of the SPECjvm98 test suite from the external lib to the testing directory.

My experience has been that this copying action can take multiple minutes, and in many cases actually takes longer than running the test. Can we avoid doing much of the copying by doing symlinks or some other cheaper operation?

fileset: Setup scanner in dir /home/dgrove/jikesrvm-benchmarks/SPECjvm98 with patternSet

{ includes: [spec/**, props/**] excludes: [] }

Activity

Ascending order - Click to sort in descending order
Hide
Permalink
Peter Donald added a comment -

I am not seeing that behviour here which leads me to believe that it is copying from a remote filesystem? It should be possible to avoid but is there any way you can get the benchmark code on a local filesystem as there could be other tests which have similar behaviour.

Show
Peter Donald added a comment - I am not seeing that behviour here which leads me to believe that it is copying from a remote filesystem? It should be possible to avoid but is there any way you can get the benchmark code on a local filesystem as there could be other tests which have similar behaviour.
Hide
Permalink
David Grove added a comment -

It's much worse when copying from a network file system. But even the benchmark dir is on local disk it takes several minutes on excalibur (AIX). From the output I see from ant, the time seems to be spent in actually building up the fileset, not the actual copy. It's the setup scanner line that is what's sitting there for several minutes when I run ant with -v -d. Maybe it's the stat or whatever other processing any is doing to figure out which files to copy? (or gack, is ant using an O(N^2) algorithm to build up a list of files to copy?)

Show
David Grove added a comment - It's much worse when copying from a network file system. But even the benchmark dir is on local disk it takes several minutes on excalibur (AIX). From the output I see from ant, the time seems to be spent in actually building up the fileset, not the actual copy. It's the setup scanner line that is what's sitting there for several minutes when I run ant with -v -d. Maybe it's the stat or whatever other processing any is doing to figure out which files to copy? (or gack, is ant using an O(N^2) algorithm to build up a list of files to copy?)
Hide
Permalink
Peter Donald added a comment -

yes - ant does ant not have the best design in that area (But I deny all responsibility! ;]). You should just be able to <exec/> an ln to achieve what we want.

Show
Peter Donald added a comment - yes - ant does ant not have the best design in that area (But I deny all responsibility! ;]). You should just be able to <exec/> an ln to achieve what we want.
Hide
Permalink
David Grove added a comment -

Not worth bothering with; we'll be switching to SPECjvm2008 instead of SPECjvm98 at some point anyways and if external lib is on local disk it doesn't more than a few seconds to do the copy.

Show
David Grove added a comment - Not worth bothering with; we'll be switching to SPECjvm2008 instead of SPECjvm98 at some point anyways and if external lib is on local disk it doesn't more than a few seconds to do the copy.

People

  • Assignee:
    Unassigned
    Reporter:
    David Grove
Vote (0)
Watch (0)

Dates

  • Created:
    Updated:
    Resolved: