Usage

Our suite will evolve to maintain its relevance. It is therefore essential that the version number associated with the release be cited in any use of the benchmark, and we ask that you please cite the 2006 OOPSLA paper describing the suite:

Running benchmarks

Execute the jar without any arguments to get usage information:

java -jar dacapo-9.12-bach.jar

To run one of the benchmarks, use the command followed by the benchmark name, for example avrora:

java -jar dacapo-9.12-bach.jar avrora

Guidance

When reporting results, be sure to report the precise version of the suite, as well as any command line arguments used.

The benchmark harness allows control over the number of iterations executed (allowing observation of the benchmark warm-up).

The harness also allows the number of client (external) threads to be controlled for many benchmarks (some benchmarks are intentionally single-threaded). Note that we sharply differentiate internal and external concurrency. The former (internal) is an intrinsic property of the benchmark and is not user-controlable. For example the avrora benchmark uses threads to represent simulated entities. The number of threads used is a property of the system being simulated and is not user controllable. This is an example of internal concurrency. On the other hand, some benchmarks, such as lusearch exhibit external concurrency which allows the user to define the number of threads across which the task (or part of the task) is partitioned. The tradebeans and tradesoap benchmarks exhibit both forms of concurrency, with their server components using thread pools to manage requests (internal concurrency, not user controlable), and while the harness specifies the number of clients which concurrently make requests to the server (external concurrency, user controlable). For all benchmarks that exhibit external concurrency, the level of concurrency is by default set at the number of available processors. This may be overriden at the command-line, either with an explicit number of threads or with a user-specified multiplier against the number of available processors. It is important that you explicitly report the number of threads used when you override the harness defaults.

Callbacks

The harness allows for pluggable callbacks which are executed before and after benchmark execution. These allow (for example) JVM-specific statistics collection to be started and stopped.

Here is an example benchmark callback class.

The distributed jar file contains one pre-built callback, MMTkCallback, which brackets the timing run with calls to MMTk's benchmark harness.