How to get Ivy to download sources from Nexus repo

Question

I have been searching around for ages trying to find a solution to my problem, but all of the other posts I have come across either don't work or don't match my situation, so hopefully someone can give me some insight.

I have a library that I am build with ant, which outputs 2 jar files, one as normal containing the compiled version of the library, and another containing just the source files.

This library is uploaded to our Nexus repository with packaging set to jar. For the actual jar file I set extension to jar and leave classifier blank. For the source jar I set classifier to sources and extension to jar.

No matter what I do with my ivy.xml file I can't get it to download the sources jar. Based on posts I have read elsewhere I tried adding:

conf="*->default,sources"

to my dependency, but then I get an error:

configuration not found in org#name;version: 'sources'

So this is basically expecting my library to define sources as a configuration somewhere? I thought it would just pick up the fact that there is another artefact with the classifier set to sources in the repository.

Can anyone give me other suggestions of what I might be doing wrong either with the way I have published my library to Nexus, or with how I have set up my dependency declaration in my Ivy file.

This is the basic Ivy.xml file where I am defining my dependency on the library that I have put in Nexus.

<ivy-module version="2.0">

    <info organisation="${ivy.organisation}" module="moduleName" />

    <configurations>
        <conf name="pda" description="moduleName for PDA"/>
        <conf name="server" description="moduleName for server"/>
    </configurations>

    <dependencies>
        <!-- Internal -->
        <dependency name="utility" org="${ivy.organisation}" rev="latest.integration" conf="${ivy.configuration}"/>
        <dependency name="myLib" org="my.org" rev="0.0.+"/>
    </dependencies>

</ivy-module>

In response to comments below, the ivy.configuration property is set to pda within my build.properties for Ant. For Eclipse I have created a properties file and also set ivy.configuration to pda (this properties file is then referenced in the Ivy settings within Eclipse).

The error message I get is:

Some projects fail to be resolved
    Impossible to resolve dependencies of ${ivy.organisation}#moduleName;working@host
        unresolved dependency: my.org#myLib;0.0.+: configuration not found in my.org#myLib;0.0.0: 'sources'. It was required from ${ivy.organisation}#moduleName;working@host pda

UPDATE Here is the resolved ivy.xml file:

<?xml version="1.0" encoding="UTF-8"?>
<ivy-module version="2.0">
    <info organisation="my.org"
        module="myLib"
        revision="0.0.0"
        status="release"
        publication="20120419131909"
        default="true"
    />
    <configurations>
        <conf name="default" visibility="public"/>
    </configurations>
    <publications>
        <artifact name="myLib" type="jar" ext="jar" conf="default"/>
    </publications>
</ivy-module>

This obviously explains why it fails to find the 'sources' conf. But I was under the impression that the sources conf should be added automatically because I have added the sources to Nexus.

Henryhenryetta

For the above ivy file I specify which configuration to run either in my ant properties file, or using IvyDE in Eclipse (and in my case it is the PDA config I am running) – Henryhenryetta

@Henryhenryetta did I read that correctly? you don't use the publish task of ivy, but manually copy the jars and the above ivy.xml to the repo? If that is the case your ivy.xml is missing the publication tags, that are need for retrieval. – Wagers

The above ivy.xml file is not copied to the repo, it is the Ivy file for the project that has the dependency on the library in the repo. There is no publication tags in that ivy file because there is only one output and that is the jar file with the same name as the module. – Henryhenryetta

can you get/post the actual ivy.xml from the repo? – Wagers

There is no ivy.xml file in the repo, it is a maven repo, so there is a pom file. – Henryhenryetta

And the pom file was generated automatically when I uploaded the artefacts to Nexus. – Henryhenryetta

ah okay, can you get the ivy.xml from the ivy cache in USER_HOME/.ivy2/cache/... ? This should be the resolved ivy.xml from the pom. And maybe this will give us the clue we need. – Wagers

I've added the resolved ivy file and some more info. – Henryhenryetta

and what do you see when you browse in Nexus to your artifact? It really seems that your repo is not handling that well. Maybe you should try Marks approach and publish with ant. – Wagers

Answer 1

So after a lot of playing around I seem to finally have a solution to this. If I declare my dependency in the following way:

<dependency name="myLib" org="my.org" rev="0.0.+">
    <artifact type="jar"/>
    <artifact type="source" m:classifier="sources" conf="pdaDev->sources"/>
</dependency>

Then I can add a new config (e.g. pdaDev above) which links to the sources config of the dependency. When using Eclipse I can tell it to use the pdaDev config and it does pull in the source jar too (if it exists).

It seems strange that Nexus doesn't seem to include the sources config in the resolved Ivy file unless it is explicitly asked for like I have done above.

Henryhenryetta

Answer 2

You should tell Ivy about configurations when you publish artifacts. For example:

<ivy-module version="2.0">
    <info organisation="apache" module="commons-lang" revision="2.5" status="release"/>
    <configurations>
        <conf name="binary" description="provide only binary files"/>
        <conf name="development" extends="binary" description="provide binary files with javadoc and sources"/>
    </configurations>
    <publications>
        <artifact name="commons-lang" ext="jar" conf="binary" type="jar"/>
        <artifact name="commons-lang-javadoc" ext="jar" conf="development" type="javadoc"/>
        <artifact name="commons-lang-sources" ext="jar" conf="development" type="source"/>
    </publications>
</ivy-module>

As you can see, commons-lang.jar published under binary configuration. But commons-lang-sources and commons-lang-javadoc published under development configuration.

When you retrieve dependencies you should use the following ivy.xml:

<ivy-module version="2.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="http://ant.apache.org/ivy/schemas/ivy.xsd">
    <info organisation="bitrete" module="strongpoint" status="development"/>
    <configurations>
        <conf name="binary" description="provide only binary files"/>
        <conf name="development" extends="binary" description="provide binary files with javadoc and sources"/>
    </configurations>
    <publications>
        <artifact name="strongpoint" ext="jar" type="jar"/>
    </publications>
    <dependencies>
        <dependency org="apache" name="commons-lang" rev="2.5"/>
    </dependencies>
</ivy-module>

and provide desired configuration when resolving with Ant:

<property name="ivy.conf" value="binary"/>
...
<property name="ivy.conf" value="development"/>
...
<ivy:resolve conf="${ivy.conf}"/>

Mayworm

But how do I publish this configuration information to a Nexus repo? So far I have published the artefacts using Ant, then manually uploading them to Nexus. This involves selecting which files belong to the library and assigning a classifier (e.g. sources). So where is the information about the configuration of the library stored in Nexus? – Henryhenryetta

Answer 3

Create two configurations in your ivy file. One for the binaries that should be on your classpath the other for the source jars.

Secondly, create two configuration mappings on the dependency:

<ivy-module version="2.0">
    <info organisation="org.demo" module="demo"/>
    <configurations>
        <conf name="compile" description="jars used for compilation classpath"/>
        <conf name="sources" description="Source jars"/>
    </configurations>
    <dependencies>
        <dependency org="org.slf4j" name="slf4j-simple" rev="1.6.4" conf="compile->default;sources->sources"/>
    </dependencies>
</ivy-module>

NOTE:

• Maven modules are translated by ivy to have a standard set of configurations. For more detail see How are maven scopes mapped to ivy configurations by ivy

Within your ANT build you can use the configurations as follows:

<target name="init">
    <ivy:resolve/>

    <!-- Populate a compile classpath for use with javac task -->
    <ivy:cachepath pathid="compile.path" conf="compile"/>

    <!-- Put source jars in the src directory -->
    <ivy:retrieve pattern="src/[artifact]-[revision](-[classifier]).[ext]" conf="sources"/>

</target>

Arthropod

That's roughly what I already tried, but it complains that the conf sources is not found in my dependency. Is there something I have to do to my dependency when adding it to nexus to add the sources configuration? I thought it would do this automatically when I add an artefact with the classifier sources. – Henryhenryetta

The "sources" configuration should be available in all remote Maven modules (see the link in my answer above). Could you update your post with the error you're getting? Additionally you need to tell me your configuration mapping (value of the ivy.configuration property).... Basically, I currently don't know how to reproduce your problem... – Funeral

I have edited my question with the error message that I get and the configuration values. – Henryhenryetta

You did not provide a value for the property "ivy.configuration". Secondly, from the error message it's clear your build is not providing a value for the property "ivy.organisation".... So, let me guess that "ivy.configuration" is something like "pda->sources"... So ivy is looking for a sources configuration in the remote module "my.org#myLib"... Look at the resolve ivy file. No sources present! – Funeral