Why is Maven downloading the maven-metadata.xml every time?

Question

Below is the error I usually get when my internet connection is flanky when trying to build a web application with maven.

My question is that, why does maven always have to download every time when the same app has been built earlier.

What could be wrong in my configuration that makes maven to download every time?

Below is an error I get when I try to build offline:

[INFO] ------------------------------------------------------------------------
[INFO] Building mywebapp 1.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
Downloading: https://raw.github.com/pagecrumb/mungo/mvn-repo/com/pagecrumb/mungo/0.0.1-SNAPSHOT/maven-metadata.xml

[WARNING] Could not transfer metadata com.mywebapp:mungo:0.0.1-SNAPSHOT/maven-metadata.xml 
from/to mungo-mvn-repo (https://raw.github.com/pagecrumb/mungo/mvn-repo/): raw.github.com
[INFO] 
[INFO] --- maven-war-plugin:2.1.1:war (default-cli) @ mywebapp ---
[INFO] Packaging webapp
[INFO] Assembling webapp [mywebapp] in [D:\workspace\web\target\mywebapp-1.0-SNAPSHOT]
[INFO] Processing war project
[INFO] Copying webapp resources [D:\workspace\web\src\main\webapp]
[INFO] Webapp assembled in [1237 msecs]
[INFO] Building war: D:\workspace\web\target\mywebapp-1.0-SNAPSHOT.war
[WARNING] Warning: selected war files include a WEB-INF/web.xml which will be ignored 
(webxml attribute is missing from war task, 
or ignoreWebxml attribute is specified as 'true')
[INFO]                                                                         
[INFO] ------------------------------------------------------------------------
[INFO] Building com.mywebapp [com.mywebapp] 0.0.1-SNAPSHOT
[INFO] ------------------------------------------------------------------------
Downloading: http://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-release-plugin/2.1/maven-release-plugin-2.1.pom

[WARNING] Failed to retrieve plugin descriptor for org.apache.maven.plugins:maven-release-plugin:2.1: Plugin org.apache.maven.plugins:maven-release-plugin:2.1 or one of its dependencies could not be resolved: Failed to read artifact descriptor for org.apache.maven.plugins:maven-release-plugin:jar:2.1
Downloading: http://download.java.net/maven/2/org/apache/maven/plugins/maven-metadata.xml
Downloading: http://download.java.net/maven/2/org/codehaus/mojo/maven-metadata.xml

397/397 B   

Downloaded: http://download.java.net/maven/2/org/codehaus/mojo/maven-metadata.xml (397 B at 0.0 KB/sec)
[WARNING] Failure to transfer org.apache.maven.plugins:maven-war-plugin/maven-metadata.xml from http://download.java.net/maven/2 was cached in the local repository, resolution will not be reattempted until the update interval of maven2-repository.dev.java.net has elapsed or updates are forced. Original error: Could not transfer metadata org.apache.maven.plugins:maven-war-plugin/maven-metadata.xml from/to maven2-repository.dev.java.net (http://download.java.net/maven/2): download.java.net
[INFO] 
[INFO] --- maven-war-plugin:2.3:war (default-cli) @ mywebapp-build ---
[INFO] Packaging webapp
[INFO] Assembling webapp [mywebapp-build] in [D:\workspace\target\mywebapp-build-0.0.1-SNAPSHOT]
[INFO] Processing war project
[INFO] Webapp assembled in [15 msecs]
[INFO] Building war: D:\workspace\target\mywebapp-build-0.0.1-SNAPSHOT.war
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] 
[INFO] mywebapp ..................................... SUCCESS [27.999s]
[INFO] com.mywebapp [com.mywebapp] ..................... FAILURE [1:00.406s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1:41.409s
[INFO] Finished at: Tue May 07 22:13:38 SGT 2013
[INFO] Final Memory: 11M/28M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-war-plugin:2.3:war 
(default-cli) on project mywebapp-build: Error assembling WAR: webxml attribute is required (or pre-existing WEB-INF/web.xml if executing in update mode)

Maroc

The access to metadata in case of SNAPSHOT's necessary to get Maven informed about newly created SNAPSHOT's etc. – Solarism

I don't know why but you can avoid this by using -o option like mvn clean install -o – Permute

Actually, the error that the build fails on is "Error assembling WAR: webxml attribute is required (or pre-existing WEB-INF/web.xml if executing in update mode)". So you should fix that. I can't image it being related to your internet connection. The dependency resolution warnings are just that: warnings. They're not the ultimate cause of your build failure. – Warpath

Perhaps you can clarify your question as you seem to have two questions: 1) Why is my build failing? 2) Why is maven trying to download metadata? user944849's answer goes a long way to answering 2). If that answers your question, you should accept it. – Warpath

Snapshot metadata update can be avoided using -nsu, --no-snapshot-updates option with mvn – Asthenopia

Answer 1

Look in your settings.xml (or, possibly your project's parent or corporate parent POM) for the <repositories> element. It will look something like the below.

<repositories>
    <repository>
        <id>central</id>
        <url>http://gotoNexus</url>
        <snapshots>
            <enabled>true</enabled>
            <updatePolicy>always</updatePolicy>
        </snapshots>
        <releases>
            <enabled>true</enabled>
            <updatePolicy>daily</updatePolicy>
        </releases>
    </repository>
</repositories>

Note the <updatePolicy> element. The example tells Maven to contact the remote repo (Nexus in my case, Maven Central if you're not using your own remote repo) any time Maven needs to retrieve a snapshot artifact during a build, checking to see if there's a newer copy. The metadata is required for this. If there is a newer copy Maven downloads it to your local repo.

In the example, for releases, the policy is daily so it will check during your first build of the day. never is also a valid option, as described in Maven settings docs.

Plugins are resolved separately. You may have repositories configured for those as well, with different update policies if desired.

<pluginRepositories>
    <pluginRepository>
        <id>central</id>
        <url>http://gotoNexus</url>
        <snapshots>
            <enabled>true</enabled>
            <updatePolicy>daily</updatePolicy>
        </snapshots>
        <releases>
            <enabled>true</enabled>
            <updatePolicy>never</updatePolicy>
        </releases>
    </pluginRepository>
</pluginRepositories>

Someone else mentioned the -o option. If you use that, Maven runs in "offline" mode. It knows it has a local repo only, and it won't contact the remote repo to refresh the artifacts no matter what update policies you use.

Farly

The question is: why is it not always "never" (which I think it should be)? Why do you need update daily or always? – Footgear

@Footgear During mid-development cycle your project might have '-SNAPSHOT' dependencies for which you might want to always pick the latest built version - at least, on a CI system you probably would, a developer might have different preference - trading build stability vs getting latest changes. What the maven docs (characteristically, unfortunately) fail to make clear is the default values for these if nothing is set. – Quieten

Best practice is that once released an artifact never changes, so <updatePolicy>never</updatePolicy> should be appropriate for them. – Quieten

But what if you update your POM dependencies to a new release? Will it never be updated? – Ayres

@PhilipRego - the updatePolicy applies per-artifact. If you change a version number or group/artifact ID, that's a different artifact. If the updatePolicy is 'never' then the artifact will be downloaded once, unless refresh is forced with -U or the artifact is removed from local repo and thus needs to be redownloaded. – Farly

Answer 2

It is possibly to use the flag -o,--offline "Work offline" to prevent that.

Like this:

maven compile -o

Roa

I believe this should be the correct answer, because you can simply call this command when your internet connection is flanky. And this was what was asked ... – Mcmahan

FWIW, the question was "why does maven always have to download every time when the same app has been built earlier" and that's what was explained above. – Farly

Answer 3

I suppose because you didn't specify plugin version so it triggers the download of associated metadata in order to get the last one.

Otherwise did you try to force local repo usage using -o ?

Mammal

So how do you specify the plugin version? I'm sure i have version set in the POM... can you be more specific? – Maroc

well if you have a version element inside the plugin element then you've indeed configured it, if so i'am out of ideas... Good luck – Mammal

in my case the version was a range [12.1 12.2) and metadata cache was set to 24 hrs so it would check for a newer version at first build each day – Eggbeater

What about default plugins? such as maven-surefire-common, which I haven't specified? – Roxanneroxburgh

their version are fixed for a given maven release but you may force a different one using pluginManagement section – Mammal

Answer 4

I haven't studied yet, when Maven does which look-up, but to get stable and reproducible builds, I strongly recommend not to access Maven Respositories directly but to use a Maven Repository Manager such as Nexus.

Here is the tutorial how to set up your settings file:

http://books.sonatype.com/nexus-book/reference/maven-sect-single-group.html

http://maven.apache.org/repository-management.html

Mabe

@acdcjunior as I said, I haven't studied that yet. But using a local Maven Repository Manager will solve most of these issues as well (if Maven checks for metadata it will only access your Maven Repository Manager) – Mabe

IMHO a repo manager is usefull only inside an organisation to avoid redundant download and particularly to deploy artifacts specific to this organization (like parent poms and internal modules). Otherwise why adding an additional mirror as you already have the local one ? – Mammal

@Mabe I don't see how that would solve the issue. If you don't want maven to check for metadata across the network at all, it doesn't matter if it checks from the internet or from intranet repo manager. – Austinaustina

@Austinaustina it should help if the "internet connection is flanky" or if one of the repository server is currently not available, as you're only accessing the maven repository manager in your LAN. – Mabe

@Gap while repo managers are particularly useful in an organisation for the reasons you mentioned, a repo manager is also important to get stable and reproducible builds, which is something I usually aim for even if I'm currently the only developer on the project. It assures that I can wipe my local repo and still am able to reproduce all builds and that other developers could easily join the project. I assure this with Jenkins, which sometimes also runs locally, accesses the repo manager as well and also helps to release my projects -> 2 users accessing the repo manager: Jenkins and myself – Mabe

Answer 5

Maven does this because your dependency is in a SNAPSHOT version and maven has no way to detect any changes made to that snapshot version in the repository. Release your artifact and change the version in pom.xml to that version and maven will no longer fetch the metadata file.

Lecia

Answer 6

In my case I use private repository mananger. While some other maven repository manangers are deprecated. It looks like gradle looks up my private artifacts from deprecated repository managers. Update or remove deprecated repository manangers fix the issue for me.

Occlude

Answer 7

I encountered a similar issue, and I was able to resolve it by removing this line from my build.gradle file:

implementation 'com.onesignal:OneSignal:[4.0.0, 4.99.99]'

Note: Kindly check all the dependencies in build.gradle. It is when one of the dependencies is depreciated or has an issue will downloading that dependency file from maven. Thanks alot!

Daric

How is build.gradle relevant to a question about Maven behaviour? – Rosamariarosamond