Run JMH benchmark under Eclipse
Asked Answered
D

2

13

I'm trying to get started with JMH under Eclipse. I can build a jar to execute from the command line but would also like me be able to run it directly within Eclipse for ease of development.

Currently I'm getting:

java.lang.RuntimeException: ERROR: Unable to find the resource: /META-INF/BenchmarkList

I'm using the simple starter case from http://nitschinger.at/Using-JMH-for-Java-Microbenchmarking/:

public class MyBenchmark {

    @Benchmark
    public void testMethod() {
        // This is a demo/sample template for building your JMH benchmarks. Edit as needed.
        // Put your benchmark code here.
    }
    
    public static void main(String... args) throws Exception {
          Options opts = new OptionsBuilder()
              .include(".*")
              .warmupIterations(10)
              .measurementIterations(10)
//            .jvmArgs("-server")
              .forks(1)
//            .outputFormat(OutputFormatType.TextReport)
              .build();
          
          new Runner(opts).run();
        }    
}

I generated the POM as specified in the JMH docs and added the exec-maven-plugin as specified in JMH Unable to find the resource: /META-INF/BenchmarkList:

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>org.sample</groupId>
    <artifactId>test</artifactId>
    <version>1.0</version>
    <packaging>jar</packaging>

    <name>JMH benchmark sample: Java</name>

    <!-- This is the demo/sample template build script for building Java benchmarks 
        with JMH. Edit as needed. -->

    <prerequisites>
        <maven>3.0</maven>
    </prerequisites>

    <dependencies>
        <dependency>
            <groupId>org.openjdk.jmh</groupId>
            <artifactId>jmh-core</artifactId>
            <version>${jmh.version}</version>
        </dependency>
        <dependency>
            <groupId>org.openjdk.jmh</groupId>
            <artifactId>jmh-generator-annprocess</artifactId>
            <version>${jmh.version}</version>
            <scope>provided</scope>
        </dependency>
    </dependencies>

    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>

        <!-- JMH version to use with this project. -->
        <jmh.version>1.15</jmh.version>

        <!-- Java source/target to use for compilation. -->
        <javac.target>1.8</javac.target>

        <!-- Name of the benchmark Uber-JAR to generate. -->
        <uberjar.name>benchmarks</uberjar.name>
    </properties>

    <build>
        <plugins>
<!--            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>3.1</version>
                <configuration>
                    <compilerVersion>${javac.target}</compilerVersion>
                    <source>${javac.target}</source>
                    <target>${javac.target}</target>
                </configuration>
            </plugin> -->
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-shade-plugin</artifactId>
                <version>2.2</version>
                <executions>
                    <execution>
                        <phase>package</phase>
                        <goals>
                            <goal>shade</goal>
                        </goals>
                        <configuration>
                            <finalName>${uberjar.name}</finalName>
                            <transformers>
                                <transformer
                                    implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
                                    <mainClass>org.openjdk.jmh.Main</mainClass>
                                </transformer>
                            </transformers>
                            <filters>
                                <filter>
                                    <!-- Shading signed JARs will fail without this. https://mcmap.net/q/45819/-quot-invalid-signature-file-quot-when-attempting-to-run-a-jar -->
                                    <artifact>*:*</artifact>
                                    <excludes>
                                        <exclude>META-INF/*.SF</exclude>
                                        <exclude>META-INF/*.DSA</exclude>
                                        <exclude>META-INF/*.RSA</exclude>
                                    </excludes>
                                </filter>
                            </filters>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
            
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>3.3</version>
                <configuration>
                    <compilerVersion>${javac.target}</compilerVersion>
                    <source>${javac.target}</source>
                    <target>${javac.target}</target>
                </configuration>
            </plugin>

<!--            <plugin>
                <groupId>org.codehaus.mojo</groupId>
                <artifactId>exec-maven-plugin</artifactId>
                <version>1.4.0</version>
                <configuration>
                    <executable>java</executable>
                    <arguments>
                        <argument>-classpath</argument>
                        <classpath />
                        <argument>com.javapapers.java.benchmark.jmh.JMHHelloWorld</argument>
                    </arguments>
                </configuration>
            </plugin> -->

            <plugin>
                <groupId>org.codehaus.mojo</groupId>
                <artifactId>exec-maven-plugin</artifactId>
                <executions>
                    <execution>
                        <id>run-benchmarks</id>
                        <phase>integration-test</phase>
                        <goals>
                            <goal>exec</goal>
                        </goals>
                        <configuration>
                            <classpathScope>test</classpathScope>
                            <executable>java</executable>
                            <arguments>
                                <argument>-classpath</argument>
                                <classpath />
                                <argument>org.openjdk.jmh.Main</argument>
                                <argument>.*</argument>
                            </arguments>
                        </configuration>
                    </execution>
                </executions>
            </plugin>

        </plugins>
        <pluginManagement>
            <plugins>
                <plugin>
                    <artifactId>maven-clean-plugin</artifactId>
                    <version>2.5</version>
                </plugin>
                <plugin>
                    <artifactId>maven-deploy-plugin</artifactId>
                    <version>2.8.1</version>
                </plugin>
                <plugin>
                    <artifactId>maven-install-plugin</artifactId>
                    <version>2.5.1</version>
                </plugin>
                <plugin>
                    <artifactId>maven-jar-plugin</artifactId>
                    <version>2.4</version>
                </plugin>
                <plugin>
                    <artifactId>maven-javadoc-plugin</artifactId>
                    <version>2.9.1</version>
                </plugin>
                <plugin>
                    <artifactId>maven-resources-plugin</artifactId>
                    <version>2.6</version>
                </plugin>
                <plugin>
                    <artifactId>maven-site-plugin</artifactId>
                    <version>3.3</version>
                </plugin>
                <plugin>
                    <artifactId>maven-source-plugin</artifactId>
                    <version>2.2.1</version>
                </plugin>
                <plugin>
                    <artifactId>maven-surefire-plugin</artifactId>
                    <version>2.17</version>
                </plugin>
            </plugins>
        </pluginManagement>
    </build>  
</project>

I'm launching with an Eclipse Run Configuration as follows:

enter image description here

I can see that the missing resource /META-INF/BenchmarkList /is/ in the jar but /not/ in the /target directory.

Is it possible to get this executing under Eclipse?

Dominance answered 28/10, 2016 at 19:27 Comment(5)
running benchmark in IDE will falsify the benchmark result. Run you benchmark via command line.Altar
I understand there may be some small differences. But, for ease of development of the benchmarks I'd like to be able to do this.Dominance
it is not easy, thats why I created a start .cmd file. So I just double click it to run the benchmark. github.com/dit-j/template-jmh-benchmarkAltar
@dit I'm not sure that "falsify" is the case for JMH, because anyway it will fork new JVM for the benchmark in the same way as for command-line execution.Estremadura
@lan this file supposed to be generated by JMH annotation processor, so maybe you can check that Eclipse executes it? I'm guessing that one in JAR is product of Maven execution, but Eclipse has "target" directory in classpath and not JAR.Estremadura
G
14

I run JMH tests in Eclipse all the time (for debug purposes), but I run maven build separately before running tests, while tests are executed as a Java application:

Step 1: build Maven project

  1. Right click on project,
  2. Select Run As,
  3. Select Maven Build and specify goals as clean install

(this step can also be done in command line by running the same goals)

Step 2: run tests

  1. Right click on project,
  2. Select Run As,
  3. Select Java Application and choose either Main - org.openjdk.jmh or the main you created
Glister answered 22/11, 2016 at 18:2 Comment(2)
This works for me in the (Eclipse) IDE but when I go to debug it, my breakpoints aren't catching. I'm working if it has something to do with the -server VM arg.Unequaled
Ah, I had to set the forks to 0 (which I thought was frowned upon but perhaps ok for development). github.com/artyushov/idea-jmh-plugin/issues/…Unequaled
J
0

I had the same problem, so i removed the <scope> from the dependency and that was the trick!

Jemena answered 15/6, 2022 at 4:50 Comment(0)

© 2022 - 2024 — McMap. All rights reserved.