Maven assembly plugin redownloading dependencies in jenkins

Question

For some reason, maven seems to be attempting to download my project's dependencies every time I try to build it with Jenkins. There are a few problems. The first one is it shouldn't be doing this at all, and the other problem is it's not even trying to download them from the correct repo, which means it has to wait for every one of them to time out.

I can run mvn clean package myself and it builds it in ~4 seconds because all of the dependencies are in my local .m2 repo. Jenkins is installed locally and is sharing this maven repo so I don't know why it's behaving differently. It appears that jenkins is issuing the maven command with -B and -f flags and is using install instead of package but even when I do this myself it does it in a few seconds whereas Jenkins takes about 8 minutes due to all of the timing out it has to wait for.

I tagged the Slick2d and lwjgl libraries because I think it has something to do with them as they aren't in a public maven repo and I have to add their repo's to the pom manually.

It's also probably worth noting that the builds finish and are successful, the only difference is Jenkins spends 8 minutes on timeouts while doing it myself happens in seconds.

Also, when I was trying to figure this out originally it would also happen when I tried to do it manually. It stopped doing this after I added the Maven Natives Repository but I don't know why Jenkins is still taking its time, as they share the same local repo and

Update: After trying to find out what's going on I seem to have narrowed it down to being related to the maven assembly plugin because if I take that part out of my pom it works fine.

Here is the relevant pieces of my pom file:

<repositories>
    <repository>
        <id>slick</id>
        <name>slick</name>
        <url>http://slick.cokeandcode.com/mavenrepo</url>
    </repository>
    <repository>
        <id>mavenNatives</id>
        <name>Maven Natives Repository</name>
        <url>http://mavennatives.googlecode.com/svn/repo</url>
    </repository>
    <repository>
        <id>nolat.org</id>
        <name>Nolat</name>
        <url>https://raw.github.com/Talon876/repo/master/releases</url>
    </repository>
</repositories>
<dependencies>
    <dependency>
        <groupId>junit</groupId>
        <artifactId>junit</artifactId>
        <version>4.8.2</version>
        <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>log4j</groupId>
        <artifactId>log4j</artifactId>
        <version>1.2.16</version>
    </dependency>
    <dependency>
        <groupId>org.nolat</groupId>
        <artifactId>toolkit</artifactId>
        <version>1.0</version>
    </dependency>
    <dependency>
        <groupId>org.lwjgl.lwjgl</groupId>
        <artifactId>lwjgl</artifactId>
        <version>2.8.3</version>
    </dependency>
    <dependency>
        <groupId>slick</groupId>
        <artifactId>slick</artifactId>
        <version>274</version>
    </dependency>
</dependencies>
<build>
    <plugins>
        [...]
        <plugin>
            <artifactId>maven-assembly-plugin</artifactId>
            <version>2.3</version>
            <executions>
                <execution>
                    <id>jar-with-dependencies</id>
                    <phase>package</phase>
                    <goals>
                        <goal>single</goal>
                    </goals>
                    <configuration>
                        <descriptorRefs>
                            <descriptorRef>jar-with-dependencies</descriptorRef>
                        </descriptorRefs>
                        <archive>
                            <manifest>
                                <mainClass>${main}</mainClass>
                                <addClasspath>true</addClasspath>
                            </manifest>
                        </archive>
                        <appendAssemblyId>false</appendAssemblyId>
                    </configuration>
                </execution>
                <execution>
                    <id>dist</id>
                    <phase>package</phase>
                    <goals>
                        <goal>single</goal>
                    </goals>
                    <configuration>
                        <descriptors>
                            <descriptor>src/META-INF/dist.xml</descriptor>
                        </descriptors>
                    </configuration>
                </execution>
            </executions>
        </plugin>
    </plugins>
 </build>

And here is the relevant pieces of the Jenkins console (Each of the downloading lines is where it sits for awhile before giving up and continuing):

[JENKINS] Recording test results
mojoStarted org.apache.maven.plugins:maven-jar-plugin:2.3.2(default-jar)
[INFO] 
[INFO] --- maven-jar-plugin:2.3.2:jar (default-jar) @ Falldown ---
mojoSucceeded org.apache.maven.plugins:maven-jar-plugin:2.3.2(default-jar)
mojoStarted org.apache.maven.plugins:maven-assembly-plugin:2.3(jar-with-dependencies)
[INFO] 
[INFO] --- maven-assembly-plugin:2.3:single (jar-with-dependencies) @ Falldown ---
Downloading: http://slick.cokeandcode.com/mavenrepo/phys2d/phys2d/060408/phys2d-    060408.pom
Downloading: http://mavennatives.googlecode.com/svn/repo/phys2d/phys2d/060408/phys2d-060408.pom
Downloading:     https://raw.github.com/Talon876/repo/master/releases/phys2d/phys2d/060408/phys2d-060408.pom
Downloading: http://repo.maven.apache.org/maven2/phys2d/phys2d/060408/phys2d-060408.pom
Downloading: http://slick.cokeandcode.com/mavenrepo/org/lwjgl/lwjgl/2.1.0/lwjgl-2.1.0.pom
Downloading: http://mavennatives.googlecode.com/svn/repo/org/lwjgl/lwjgl/2.1.0/lwjgl-  2.1.0.pom
Downloading:    https://raw.github.com/Talon876/repo/master/releases/org/lwjgl/lwjgl/2.1.0/lwjgl-2.1.0.pom
Downloading: http://repo.maven.apache.org/maven2/org/lwjgl/lwjgl/2.1.0/lwjgl-2.1.0.pom
Downloading: http://slick.cokeandcode.com/mavenrepo/org/lwjgl/lwjgl-jinput/2.1.0/lwjgl-jinput-2.1.0.pom
Downloading: http://mavennatives.googlecode.com/svn/repo/org/lwjgl/lwjgl-   jinput/2.1.0/lwjgl-jinput-2.1.0.pom
Downloading: https://raw.github.com/Talon876/repo/master/releases/org/lwjgl/lwjgl- jinput/2.1.0/lwjgl-jinput-2.1.0.pom
Downloading: http://repo.maven.apache.org/maven2/org/lwjgl/lwjgl-jinput/2.1.0/lwjgl-jinput-2.1.0.pom
Downloading: http://slick.cokeandcode.com/mavenrepo/org/lwjgl/lwjgl-native/2.1.0/lwjgl-native-2.1.0.pom
Downloading: http://mavennatives.googlecode.com/svn/repo/org/lwjgl/lwjgl-native/2.1.0/lwjgl-native-2.1.0.pom
Downloading: https://raw.github.com/Talon876/repo/master/releases/org/lwjgl/lwjgl-native/2.1.0/lwjgl-native-2.1.0.pom
Downloading: http://repo.maven.apache.org/maven2/org/lwjgl/lwjgl-native/2.1.0/lwjgl-native-2.1.0.pom
Downloading: http://slick.cokeandcode.com/mavenrepo/javax/jnlp/jnlp/1.2/jnlp-1.2.pom
Downloading: http://mavennatives.googlecode.com/svn/repo/javax/jnlp/jnlp/1.2/jnlp-1.2.pom
Downloading: https://raw.github.com/Talon876/repo/master/releases/javax/jnlp/jnlp/1.2/jnlp-1.2.pom
Downloading: http://repo.maven.apache.org/maven2/javax/jnlp/jnlp/1.2/jnlp-1.2.pom
Downloading: http://slick.cokeandcode.com/mavenrepo/ibxm/ibxm/unknown/ibxm-unknown.pom
Downloading: http://mavennatives.googlecode.com/svn/repo/ibxm/ibxm/unknown/ibxm-unknown.pom
Downloading: https://raw.github.com/Talon876/repo/master/releases/ibxm/ibxm/unknown/ibxm-unknown.pom
Downloading: http://repo.maven.apache.org/maven2/ibxm/ibxm/unknown/ibxm-unknown.pom
Downloading: http://slick.cokeandcode.com/mavenrepo/jcraft/jogg/0.0.7/jogg-0.0.7.pom
Downloading: http://mavennatives.googlecode.com/svn/repo/jcraft/jogg/0.0.7/jogg-0.0.7.pom
Downloading: https://raw.github.com/Talon876/repo/master/releases/jcraft/jogg/0.0.7/jogg-0.0.7.pom
Downloading: http://repo.maven.apache.org/maven2/jcraft/jogg/0.0.7/jogg-0.0.7.pom
Downloading: http://slick.cokeandcode.com/mavenrepo/jcraft/jorbis/0.0.15/jorbis-0.0.15.pom
Downloading: http://mavennatives.googlecode.com/svn/repo/jcraft/jorbis/0.0.15/jorbis-0.0.15.pom
Downloading: https://raw.github.com/Talon876/repo/master/releases/jcraft/jorbis/0.0.15/jorbis-0.0.15.pom
Downloading: http://repo.maven.apache.org/maven2/jcraft/jorbis/0.0.15/jorbis-0.0.15.pom
[WARNING] Artifact: org.nolat:Falldown:jar:1.0-SNAPSHOT references the same file as the assembly destination file. Moving it to a temporary location for inclusion.
[INFO] META-INF/ already added, skipping
[INFO] META-INF/MANIFEST.MF already added, skipping
[INFO] org/ already added, skipping
...and so on

Fakir

Did you tried -U in maven parameter fields on Jenkins configuration in order to force update all dependencies? – Abuttal

When clicking 'Advanced' button under 'Build', what are MAVEN_OPTS and what boxes are checked? – Unceasing

The only one checked is "Send e-mail for each failed module" I'm trying a build with -U now but it's still taking a long time. I'll wait until it's finished and build another without -U and post a comment with the results. – Fakir

Forcing a -U seemed to get it stuck at Downloading: raw.github.com/Talon876/repo/master/releases/org/lwjgl/lwjgl/… I had to abort the build. – Fakir

@Talon876, you need to specify who are you talking to in the comment with @ - otherwise they won't be notified (unless it's the OP, or the user who posted the answer on which you are commenting). – Unceasing

Answer 1

The first step to get stable Maven builds is to use a Maven Repository Manager such as Nexus.

• Install the Maven Repoitory Manager (like Jenkins you can install Nexus locally if you are the only developer)
• Configure Maven to use a single Nexus group. You have to do this for every user, also for the Jenkins user if you have a dedicated system user to run Jenkins (recommended approach). If you're using a different Maven Repository Manager than Nexus, check the documentation.
• Only define repositories in your Maven Repository Manager. No repositories sections in your POMs!

Once you've done this check the behaviour and tell us if you have any issues.

Goethe

Answer 2

It seemed like the maven assembly plugin was the cause of trying to download things so I replaced it with the maven shade plugin which lets me exclude some files I was wanting to exclude anyway. Not sure if this is necessarily the best solution but it works in my case. Ended up dropping my build time from 8 minutes to 25 seconds.

This is what I added to my pom after taking out the jar-with-dependencies part of the assembly plugin.

        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-shade-plugin</artifactId>
            <version>1.6</version>
            <configuration>
                <createDependencyReducedPom>false</createDependencyReducedPom>
            </configuration>
            <executions>
                <execution>
                    <phase>package</phase>
                    <goals>
                        <goal>shade</goal>
                    </goals>
                    <configuration>
                        <transformers>
                            <transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
                                <mainClass>${main}</mainClass>
                            </transformer>
                        </transformers>
                        <filters>
                            <filter>
                                <artifact>*:*</artifact>
                                <excludes>
                                    <exclude>*.dll</exclude>
                                    <exclude>*.so</exclude>
                                    <exclude>*.jnilib</exclude>
                                </excludes>
                            </filter>
                        </filters>
                    </configuration>
                </execution>
            </executions>
        </plugin>

Fakir