How to access common resource files from multiple projects

Question

In one of my projects I have resources stored in /src/test/resources (typical maven directory structure). Those resources are being used by an utility class stored in the project.

The utility class itself is being used from other projects (other projects depend on this one). I would access the resource like that:

final InputStream inputStreamDobs = 
    ClassLoader.class.getResourceAsStream("/dbunit/clear_db.xml");

but since I use it from different projects the path is not correct - it is relative to current project that is being built/tested, not the one where utility class and resources are.

Any thought how to approach this?

I need to avoid absolute paths - would like to have a way of defining relative path to the utility class.

I also don't want to replicate resources over multiple projects. Cheers.

EDIT: To give a context I have a definition of tables in XML file that needs to be cleared after Integration Tests (clear whole DB schema). Integration Tests sits in multiple project, but the clear script and resource file is the same for all of them and sits in common parent project.

EDIT2:

Bonus question: I would like to access common DTD file (let's call it tables.dtd) that need to be accessed from XML files from multiple other projects. (it will sit in common parent project).

Currently I have it duplicated over multiple project, and I refer to it from XML using directive:

<!DOCTYPE dataset SYSTEM "src/test/resources/dbunit/dobs.dtd">

How to point it to a file in different project?

Preciousprecipice

Answer 1

You wrote

... but since I use it from different projects the path is not correct - it is relative to current project that is being built/tested, not the one where utility class and resources are ...

Relative paths are not the problem here. You use an absolute path in your example, but even if you would use a relative one, this would refer to the package or directory structure. getResourceAsStream would pick them up as long as the classpath is correct. The real problem is that you are referring to test resources in another project. But test resources and classes are not contained in the project artifact, so they are not accessible from modules that include this as a dependency. If you need these resources for tests in several projects, I would suggest that you create a new project (let's say "projectxyz-testresources") with these resouces contained in src/main/resources and add this as a dependency with scope "test" where relevant.

EDITED TO ADD:

If you don't want to use a separate project for test resources, you can create a test-jar containing test classes and resources using goal jar:test-jar and include this as a test dependency. You may want to configure the jar plugin in your pom to execute this goal on regular builds.

Schwerin

How come? I use relative path.. it is relative to project root. (?) – Preciousprecipice

getResourceAsStream tries to load the Resource from classpath. It is absolute because it starts searching at the root of each classpath entry. A relative path would not Start with a slash and would be searched relative to the current package. So your path is not absolute regarding you filesystem, but absolute in Java sense. – Schwerin

Answer 2

Your runtime classpath shouldn't reference src/test/resources. Maven will copy everything over to target so you should be able to get it with "/dbunit/clear_db.xml"

Drawer

That's right, my mistake in copy-pasting. Question is still valid though. – Preciousprecipice

That absolute path is "absolute" against a classpath root. so if you put it in a jar or even an exploded directory, it will be found via the classpath not any particular location on disk. So as long as your classpath is correct, that path will be portable across every installation. – Drawer

But classpaths are different for different projects, right? (I have rich maven modules hierarchy) – Preciousprecipice

The classpath will vary in that, e.g., paths to jars on a windows file system will differ from those on linux. But to your app, it's just a classpath and so your code is shielded from such variations and can be Written Once and Run Anywhere. – Drawer

But those projects (modules) are being build separately. I'm most interested in maven's integration-test phase for each of them as this Util class is used for testing (with dbunit). – Preciousprecipice

maven puts src/test/resources/* in to target/test-classes which is on the classpath so "/dbunit/..." will still work. – Drawer

Yes, for each project separately. You still can use a class from other project somehow, but it will access resources not from own project. – Preciousprecipice

Answer 3

Have you tried using "classpath:" prefix? You shouldn't have to specify full paths to resources if they are available on the classpath. For example:

<mvc:resources location="classpath:/META-INF/web-resources/" mapping="/resources/**" />

/META-INF/web-resources/ comes from Spring MVC's JAR which is one of the project's dependencies. The project doesn't need to know how to get to that resource "directly" instead it uses the classpath.

J

Answer 4

I think you can use 2 below plugins for this purpose. if your project is web project, you can create 2 web projects, one of them contains config files and another project is your main project, now on you can overlay projects with maven-war-plugin

There are many types for overlay, you can visit this page for more information maven-war-plugin overlay

for jar files you can merge them to single one with maven-assembly-plugin

maven-assembly-plugin

Kweisui

Answer 5

We had a similar instance in which, we had some configuration files that were used across multiple modules in big multi module maven project. So what we did was split the conf files into a separate module, that just jars up the config files. And then whatever module needed those files could declare the dependency to the config files modules and just use the jar files with the config files and unzip them to use them. You could do something similar for the test resources. Create a module with just the resources and then put them into the relevant path.

Thema

Answer 6

In this case, I would suggest to program a maven plugin for this task, so that this so variable path may be configured via properties. You will also benefit from the Project object.

/**
 * Location of the file.
 */
@Parameter(defaultValue = "${basedir}/src/main/java", property = "sourceFolder", required = true)
private File sourceFolder;

@Parameter(defaultValue = "${basedir}/src/test/java", property = "testFolder", required = true)
private File testFolder;

@Parameter(defaultValue = "${project}", property = "project", readonly = true, required = true)
private MavenProject project;

Wynellwynn

I will try this (after the weekend probably) and will give you an update, but I think it may work. Thanks! – Preciousprecipice

@Wynellwynn - your application shouldn't know about the build tool used to build it or the folder structure expected by that tool. What if you want to use a resource from another project that is not part of your code base or doesn't use maven's layout? Java's classpath does this for you without having to know project layouts. – J

@ike_love Pardon? I can't follow you thoughts. – Wynellwynn

@Wynellwynn - your solution uses absolute paths and only works for resources that are part of a JAR exploded during the build. The standard Java way of using a resource from a JAR is to use classpath, not to explode the jar and refer to that resource directly. – J

@ike_love Are you sure you comment the right post? This is a rather typical setup for building a maven plugin. At this phase sources-generate we are not realy talking about jar but folders with sources and resources. And of course the annotated paths are just defaults which can be overriten being the configuration you set. – Wynellwynn

@ike_love Further more the example uses the base path environment with is the root of the current project to be build. If you look at the jaxb plugin, they use the same mechanism. – Wynellwynn

@Wynellwynn - there is no need to write a plugin just to be able to share a resource between JARs, that mechanism already exists - classpath. – J

Answer 7

Can you try using Maven Overlays? We had faced similar situation in the past, and very much resolved using overlays. Usually, overlays are used to share common resources across multiple web applications.

Squall

Answer 8

Maybe one of these approaches:

Either: You outsource the xml files AND the java classes (like the utility class) that need access to them into a seperate library which you can then include into your several other projects (conveniently as maven dependencies).

Or: You put these xml files into a seperate VCS-tree or repository and you include them in your project tree (e.g. as an "external" in SVN). So you can update this reference seperately in your projects, but you only have to maintain one source file.

Jenny