Using closed-source dependencies with Maven

Question

I have a closed-source project that I would like to build using Maven. It has a dependency on two java libraries which are not available in any public repository that I've been able to find (libGoogleAnalytics.jar and FlurryAgent.jar in this case, but the question applies to any closed-source dependency).

I would like anyone in my organization to be able to build the application using the exact same versions of the dependencies that I use to build the application. This includes my colleagues and our build-server.

How do I manage closed-source dependencies that maven doesn't know how to resolve?

Obviously, I could go to each person's machine and manually execute "mvn install:install-file" to get the binary into their maven repository, but manually managing dependencies like that defeats the purpose of a dependency manager.

As per maven's Internal Repositories documentation, I could set up a repository server somewhere and put the binaries there, which all the developers would then access. But that means I have a new server to maintain (or at least a new website on an existing server). It also means I have to worry about permissions to ensure that outside parties can't access the repository. It also means I have to worry about backups and availability now so that developers don't run into hiccoughs if the repository is unavailable.

All of these problems would go away for me if I could somehow use our existing scm (hg in this case, but it could be git or svn or whatever) to store the dependencies. Our source control repository is backed up already, it will basically always be available to developers doing builds, and its permissions have already been dealt with.

But I haven't been able to figure out how to manage maven dependencies using hg yet, if this is even possible.

Ammoniac

Check this question: #2589002 I had a similar issue. – Absorptance

If you are building a commercial product, you should have an internal repo with all of your dependencies anyway, so that your build is not dependent on external sources - and you can still create a build if your internet connection goes down. – Chalybite

Answer 1

It turns out that Manfred's answer didn't quite work for me. The app compiled, but it did not run on my Android device because the required google analytics classes were missing.

Following the links he supplied, I discovered this solution which is actually a little cleaner and worked properly.

In summary, I added the following dependencies to my pom.xml. The groupId, artifactId, and version were all made up by me using reasonable values:

<dependencies>
    ...
    <dependency>
        <groupId>com.google.android.apps.analytics</groupId>
        <artifactId>libGoogleAnalytics</artifactId>
        <version>1.1</version>
    </dependency>
    <dependency>
        <groupId>com.flurry</groupId>
        <artifactId>FlurryAgent</artifactId>
        <version>1.24</version>
    </dependency>
</dependencies>

I then added a repository definition for where I'm storing the third party dependencies in my project's source tree:

    <repository>
        <id>third.party.closed.source.repo</id>
        <url>file://${basedir}/../maven_repo_3rd_party</url>
    </repository>

I then moved the jar files to the following location:

./maven_repo_3rd_party/com/google/android/apps/analytics/libGoogleAnalytics/1.1/libGoogleAnalytics-1.1.jar
./maven_repo_3rd_party/com/flurry/FlurryAgent/1.24/FlurryAgent-1.24.jar

Once I did that, my project compiled and ran exactly as if the third party dependencies were resolved from an official maven repo.

Ammoniac

I wonder if this works if you add modules deeper in the directory structure. The relative path ../maven_repo_3rd_party seems fragile. – Paintbrush

It works fine for most maven projects. You just need to adjust the relative path for the module. – Ammoniac

Answer 2

While I really think you should use a dedicated repository server and Sean Patrick is totally right about it here is a hack to get it to work.

Put the jar file in a libs folder just like you did in the days gone by (remember Ant.. ouch) .. and then declare a dependency to each jar using the scope system and a path.

An example can I did this for is described here

http://www.simpligility.com/2010/01/how-to-mavenize-a-typical-web-application-build-jasperserver-3-7-sample-webapp/

Specifically a dependency would e.g. look like this

<dependency>
  <groupId>jasperreports</groupId>
  <artifactId>jasperreports-chart-themes</artifactId>
  <version>3.7.0</version>
  <scope>system</scope>
  <systemPath>${project.basedir}/src/main/webapp/WEB-INF/lib/jasperreports-chart-themes-3.7.0.jar</systemPath>
</dependency

Oh and now that I told you how to do it keep in mind that this is BAD practice and has a bunch of issues but it will work...

Ravens

Sean Patrick is totally right Being "totally right" bought me a downvote. I guess that's the way Obama must feel :-) – Cursive

But +1 for your solution which is probably better than misusing an SCM as a maven repo – Cursive

Sorry sean! But this really was more what I was looking for. – Ammoniac

@Ammoniac I am not complaining about not being upvoted. I am complaining about being downvoted. – Cursive

yeah... sometimes I get downvoted when I know I am right and others agree and there is nothing I can do.. – Ravens

I'm glad manfred voted you back up, Sean. And I agree that for some deployments, your answer is a good best practices answer. However, for my case, and for others like me, the best practices answer you gave is a rather heavy-weight solution to a common problem. I already knew how to implement that solution, and the question was to explore what OTHER solutions were available. I appreciate your desire to encourage me to do the right thing, but you did explicitly ignore the intent of the question, which is why I downvoted you. – Ammoniac

@Ammoniac fair enough. about explicitly ignoring things: I am a consultant, and I would be a bad consultant if I didn't know that the customer sometimes thinks he wants a, while he really wants to be persuaded to do b. Not always, and not in your case, but it's a valid scenario. If all questions on this site were answered literally only, this site would stink. What annoys me about this question now is that the "proper" answer (or at least one that gets close) has zero rep, while a brutal hack is upvoted. If I were a newbie coming from Google that would imply to me that this answer is a best ... – Cursive

... practice. This site is not only about your question and my answer, it's supposed to be a valid reference to users seeking for advice. If someone else asks a similar question, it will be closed as a duplicate and merged with this one. This is not the kind of scenario where I would go ahead and downvote best practices, just because they don't fit my personal situation. But as you can see I've got a weird idealistic attitude about this. – Cursive

Answer 3

Use A dedicated Repository Server

As per maven's Internal Repositories documentation, I could set up a repository server somewhere and put the binaries there, which all the developers would then access.

Exactly. Set up a maven repository server with several repositories, e.g. these:

• internal-releases
• internal-snapshots
• external-opensource
• external-closedsource (this is where the lib we are talking about goes)

But that means I have a new server to maintain (or at least a new website on an existing server). It also means I have to worry about permissions to ensure that outside parties can't access the repository.

Yes, but a company that does serious software development should have an infrastructure like that. But if your company is serious about using Maven, there should probably also be a dedicated position for configuration management, and that person should administer this server.

It also means I have to worry about backups and availability now so that developers don't run into hiccoughs if the repository is unavailable.

The standard repository servers (e.g. Sonatype Nexus) are rock solid. If it ever hangs, just restart the app server / servlet container it's running on. Also, once developers have downloaded a library from the repo, it remains in the local repo, so even if the repo is down, there shouldn't be a problem (but you can't reference a new dependency when the server is down).

---

Use your existing SCM as a maven repository

OK, if you really want to use your SCM as a maven repo, here's how to do it:

http://maven-svn-wagon.googlecode.com/svn/site/index.html

This article describes how to setup an SVN-based maven repository for your own project. But if you want to deploy a third-party to the repo, just create a pom with the config mentioned here and use that pom to deploy:deploy-file your library.

(There are other wagon / scm implementations also, and the configuration is slightly different, but the solution remains the same: create a pom according to the wagon implementation you are using and then do deploy:deploy-file (see more info on the usage page)

Cursive

Thank you, but as I said in my description, I'm explicitly looking to avoid this solution. My company is not "serious about maven". I'm trying to use maven on a project without significant support from other company employees, and as such cannot afford to take on the maintenance overhead of another server. – Ammoniac

@Ammoniac sorry, this is the only way to do it that has anything to do with how maven works. If you don't like how maven works, don't use maven. Sure, you could also put the jar on a file server together with a shell script that installs it into the user's local server, but that's not the way software should be developed these days. Builds should be reproducible and workplace-independent. Take the Joel Test ferchrissakes, if you don't do it as above (or similarly) you are at least violating items 2 and 3. – Cursive

Thanks for the update, Sean. I've had extensive problems getting your suggestion to work using mercurial with maven-scm-provider-hg, but luckily Manfred's come up with a solution that works for me. See these open issues: jira.codehaus.org/browse/MNG-2227 and maven.40175.n5.nabble.com/… – Ammoniac