Usage

The DaCapo Benchmark Maven Plugin is used to bundle an individual DaCapo benchmark. To this effect, it follows a POM‐first approach; all the benchmark’s configuration is done in the POM.

The preferred way to use the DaCapo Benchmark Maven Plugin is to set your project’s package type to dacapo-benchmark. This will bind the dacapo-benchmark:configuration, dacapo-benchmark:package-data, dacapo-benchmark:package-libraries, dacapo-benchmark:integration-test, and dacapo-benchmark:verify goals to the appropriate lifecycle phases. If desired, however, goals can be bound to phases manually, e.g., to use a hand‐crafted benchmark configuration file instead of the one generated by the dacapo-benchmark:configuration goal.

How to Use the Package Type

To use the dacapo-benchmark project package type and thus benefit from the plugin’s default lifecycle bindings, you must load the build extensions of the DaCapo Benchmark Maven Plugin.

<project>
  <packaging>dacapo-benchmark</packaging>
  ...
  <build>
    <plugins>
      <plugin>
        <groupId>org.scalabench.plugins</groupId>
        <artifactId>dacapo-benchmark-maven-plugin</artifactId>
        <version>0.1.0-SNAPSHOT</version>
        <!-- Load build extensions -->
        <extension>true</extension>
      </plugin>
      ...
    </plugins>
    ...
  </build>
  ...
</project>

How to Declare the Benchmark’s Dependencies

An individual DaCapo benchmark declares its dependencies in an accompanying .cnf file. When using the dacapo-benchmark:configuration goal, these dependency declarations are derived from the POM; all your provided‐scoped dependencies are assumed to be dependencies of the benchmark itself (as opposed to dependencies of the harness).

<project>
  ...
  <dependencies>
    <!-- Dependency of the harness (compiled‐scoped) -->
    <dependency>
      <groupId>org.dacapobench</groupId>
      <artifactId>dacapo-benchmark-suite</artifactId>
      <version>9.12</version>
    </dependency>
    <!-- Dependency of the benchmark (provided‐scoped) -->
    <dependency>
      <groupId>...</groupId>
      <artifactId>...</artifactId>
      <version>...</version>
      <scope>provided</scope>
    </dependency>
    ...
  </dependencies>
  ...
</project>

Using dependencies and dependency scopes thus makes it easy for the DaCapo Benchmark Maven Plugin to retrieve transitive dependencies, while at the same time keeping separate the dependencies of the benchmark and its harness.

Where to Place the Benchmark’s Inputs

By default, the dacapo-benchmark:package-data goal expects the benchmark’s input data in ${basedir}/src/main/data. You can configure the goal, however, to look for input data in a different directory.

<project>
  ...
  <build>
    <plugins>
      <plugin>
        <groupId>org.scalabench.plugins</groupId>
        <artifactId>dacapo-benchmark-maven-plugin</artifactId>
        <version>0.1.0-SNAPSHOT</version>
        <configuration>
          <dataSourceDirectory>...</dataSourceDirectory>
        </configuration>
      </plugin>
      ...
    </plugins>
    ...
  </build>
  ...
</project>

It is furthermore possible to configure any number of additional data source directories.

How to Declare the Benchmark’s Expected Outputs

The DaCapo benchmark suite’s harness can validate the benchmark’s output to determine whether an iteration is deemed a pass or failure. In case the dacapo-benchmark:configuration mojo is used to automatically generate the benchmark’s configuration file, the DaCapo Benchmark Maven Plugin’s configuration needs to describe the expected outputs of the benchmark.

<project>
  ...
  <build>
    <plugins>
      <plugin>
        <groupId>org.scalabench.plugins</groupId>
        <artifactId>dacapo-benchmark-maven-plugin</artifactId>
        <version>0.1.0-SNAPSHOT</version>
        <configuration>
          <output>
            <name>stdout.log</name>
            <text>true</text>
            <lines>42</lines>
          </output>
          <output>
            <name>$${SCRATCH}/example/file.dat</name>
            <checksum>0x0123456789ABCDEF0123456789ABCDEF01234567</checksum>
          </output>
          ...
        </configuration>
      </plugin>
      ...
    </plugins>
    ...
  </build>
  ...
</project>

Each <output> element can place zero or more constraints on the named output file. These constraints can prescribe the number of lines or bytes (using the <lines> or <bytes> element, respectively) in the output file, or they can prescribe a SHA‐1 digest (using the <checksum> element). If any of the output files’ constraints are not fulfilled, the benchmark’s iteration is deemed a failure.

Since the benchmark’s scratch directory, set by the harness’s --scratch-directory command line option, may not reside below the current directory, the $${SCRATCH} property can be used to refer to the scratch directory in a portable manner. Note the two dollar signs; the former acts as an escape for the latter within the POM, as Maven itself also makes use of such property references.