How can I tell which signed jar is causing maven-shade-plugin to fail?

Question

To run maven-shade-plugin, I have to use to method described here due to signed dependencies, as shown here:

<plugin>
    <artifactId>maven-shade-plugin</artifactId>
    <executions>
        <execution>
            <phase>package</phase>
            <goals><goal>shade</goal></goals>
            <configuration>
                <filters>
                    <filter>
                        <!-- filter out signature files from signed dependencies, else repackaging fails with security ex -->
                        <artifact>*:*</artifact>
                        <excludes>
                            <exclude>META-INF/*.SF</exclude>
                            <exclude>META-INF/*.DSA</exclude>
                            <exclude>META-INF/*.RSA</exclude>
                        </excludes>
                    </filter>
                </filters>
            </configuration>
        </execution>
    </executions>
</plugin>

Question: How can I tell which JAR caused the plugin to fail? It doesn't seem to be the most recently mentioned one.

Tail of debug output:

[DEBUG] We have a duplicate org/jdom2/xpath/util/AbstractXPathCompiled.class in C:\Users\me\aaaa-1.11.0-SNAPSHOT.jar
[DEBUG] We have a duplicate org/jdom2/xpath/util/XPathDiagnosticImpl.class in C:\Users\me\aaaa-1.11.0-SNAPSHOT.jar
[DEBUG] Processing JAR C:\Users\me\bbbb-1.11.0-SNAPSHOT.jar
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] 
[INFO] Retrievers JAR ................................ FAILURE [  9.581 s]
[INFO] Retrievers .................................... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 10.208 s
[INFO] Finished at: 2015-05-12T14:20:54-05:00
[INFO] Final Memory: 100M/726M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-shade-plugin:2.3:shade (default) on project aaa-retrievers-jar: Error creating shaded jar: Invalid signature file digest for Manifest main attributes -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-shade-plugin:2.3:shade (default) on project aaa-retrievers-jar: Error creating shaded jar: Invalid signature file digest for Manifest main attributes
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:216)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:108)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:76)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:116)
    at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:361)
    at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:155)
    at org.apache.maven.cli.MavenCli.execute(MavenCli.java:584)
    at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:213)
    at org.apache.maven.cli.MavenCli.main(MavenCli.java:157)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:483)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
Caused by: org.apache.maven.plugin.MojoExecutionException: Error creating shaded jar: Invalid signature file digest for Manifest main attributes
    at org.apache.maven.plugins.shade.mojo.ShadeMojo.execute(ShadeMojo.java:566)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:133)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:208)
    ... 19 more
Caused by: java.lang.SecurityException: Invalid signature file digest for Manifest main attributes
    at sun.security.util.SignatureFileVerifier.processImpl(SignatureFileVerifier.java:284)
    at sun.security.util.SignatureFileVerifier.process(SignatureFileVerifier.java:238)
    at java.util.jar.JarVerifier.processEntry(JarVerifier.java:316)
    at java.util.jar.JarVerifier.update(JarVerifier.java:228)
    at java.util.jar.JarFile.initializeVerifier(JarFile.java:383)
    at java.util.jar.JarFile.getInputStream(JarFile.java:450)
    at org.apache.maven.plugins.shade.DefaultShader.shade(DefaultShader.java:147)
    at org.apache.maven.plugins.shade.mojo.ShadeMojo.execute(ShadeMojo.java:471)
    ... 21 more
[ERROR] 
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException

Leinster

did you try building with the -X option? – Kahler

Yes; that's what produces the debug output shown. – Leinster

Answer 1

The shade plugin is unpacking all of the jars for the dependencies you have included and stuffing their contents into a single jar file. Sort of as if you had written all of it yourself.

The configuration is telling the shade plugin not to move any files which end in .SF, .DSA or .RSA if they are included in a directory called META-INF.

So all you need to do is figure out which jar has those files.

First thing I would do is comment out the filter section and re-build. Then grep your shaded jar for those extensions. It might give you a clue to the package.

The -t option on the jar command will list all of the files in the archive without extracting them. In general jar syntax is pretty similar to tar.

jar -tvf target/myapp-1.0.3-SNAPSHOT.jar | grep -i dsa

META-INF/BCKEY.DSA

In my case it was pretty obvious. I had recently added Bouncy Castle as a dependency. BCKEY.DSA seems like it might be the Bouncy Castle Key.

To confirm I just performed the same action on the bouncy castle jar. Since I built this with maven the jar is in my local repository:

tar -tvf .m2/repository/org/bouncycastle/bcprov-jdk15on/1.48/bcprov-jdk15on-1.48.jar | grep -i dsa
-rwxrwxrwx  0 0      0           0 Feb  9  2013 META-INF/BCKEY.DSA

Project

"Then grep your shaded jar for those extensions. It might give you a clue to the package." ... unfortunately it is completely empty – Boethius

Same here - shaded jar is empty, no difference whether or not the filter section is active. Unshaded jar does not contain any signature files. Besides that, this answer leaves me without any idea about how to correct the problem. – Tague

Answer 2

to get list of signed JARs with Maven and bash+awk+sed, one can try something like this:

#!/bin/bash
mvn_classpath=`mvn dependency:build-classpath -B | awk '/Dependencies classpath:/{getline; print}' | sed -e s/:/\\\\n/g`

for jar in $mvn_classpath; do 
        echo -n `jarsigner -verify $jar | grep verified | wc -l`; echo " $jar"; 
done

This will list JAR files used by your project - those that are signed and verified are preceded by 1, the unsigned by 0. I had no signed JAR that would not be possible to verify, so I'm not sure how the logic should look in this case.

Thar