Neo4j embedded throws IllegalArgumentException: Config has no association with setting: 'unsupported.dbms.lucene.ephemeral'

Question

I am currently developing a java application (with maven) with an embedded neo4j database. I followed the official guide and also looked at the examples on github.

My code looks like this now:

import org.neo4j.dbms.api.DatabaseManagementService;
import org.neo4j.dbms.api.DatabaseManagementServiceBuilder;
import org.neo4j.graphdb.*;

import static org.neo4j.configuration.GraphDatabaseSettings.DEFAULT_DATABASE_NAME;

public class GraphDatabase {

    private DatabaseManagementService managementService;
    private GraphDatabaseService databaseService;

    public void start(String dir) {
        managementService = new DatabaseManagementServiceBuilder(new File(dir)).build();

        databaseService = managementService.database(DEFAULT_DATABASE_NAME);
    }
...
}

In the pom.xml i included:

<dependency>
   <groupId>org.neo4j</groupId>
   <artifactId>neo4j</artifactId>
   <version>4.0.0</version>
</dependency>
...
<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-assembly-plugin</artifactId>
    <version>3.2.0</version>
    <executions>
        <execution>
            <phase>package</phase>
            <goals>
                <goal>single</goal>
            </goals>
        </execution>
    </executions>
    <configuration>
        <archive>
            <manifest>
                <addClasspath>true</addClasspath>
                <mainClass>{myPackageHere}.App</mainClass>
            </manifest>
        </archive>
        <descriptorRefs>
            <descriptorRef>jar-with-dependencies</descriptorRef>
        </descriptorRefs>
    </configuration>
</plugin>

When running in Intellij IDE there are no problems, but when i compile with mvn package and execute with java -jar <name>.jar I get the following exception:

java.lang.IllegalArgumentException: Config has no association with setting: 'unsupported.dbms.lucene.ephemeral'
        at org.neo4j.configuration.Config.getObserver(Config.java:635)
        at org.neo4j.configuration.Config.set(Config.java:653)
        at org.neo4j.dbms.api.DatabaseManagementServiceBuilder.newDatabaseManagementService(DatabaseManagementServiceBuilder.java:83)
        at org.neo4j.dbms.api.DatabaseManagementServiceBuilder.build(DatabaseManagementServiceBuilder.java:78)

I tried this on both linux and windows with the same exception.

Anyone has an idea what my problem might be?

Update:

I removed the maven-assembly-plugin. If I compile and run the projekt like this, it works:

mvn compile
mvn exec:java -Dexec.mainClass={myPackageHere}.App

But I would still prefere a jar...

Nymphet

Answer 1

I ran into the same exception when I tried to create an uber-jar for a project that included Neo4j as an embedded database. At that point I used the maven-assembly-plugin as well.

The solution I found was to use the maven-shade-plugin instead and to include a ServicesResourceTransformer in the configuration of the plugin.

The entry of the maven-shade-plugin in my pom.xml looks like this:

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-shade-plugin</artifactId>
  <version>3.2.2</version>
  <executions>
    <execution>
      <phase>package</phase>
      <goals>
        <goal>shade</goal>
      </goals>
      <configuration>
        <transformers>
          <transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
            <manifestEntries>
              <mainClass>{myPackageHere}.App</mainClass>
            </manifestEntries>
          </transformer>
          <transformer implementation="org.apache.maven.plugins.shade.resource.ServicesResourceTransformer"/>
        </transformers>
        <filters>
          ...
        </filters>
      </configuration>
    </execution>
  </executions>
</plugin>

I build the jar file with the mvn clean package command. Executing the jar via java -jar <name>.jar gave me the exact same result as running the app from my IDE and did not throw the exception above.

Some additional context: The output of the maven-shade-plugin clearly indicates that multiple Neo4j dependencies intend to include overlapping services in the META-INF/services directory of the jar, e.g.

[WARNING] neo4j-configuration-4.0.2.jar, neo4j-dbms-4.0.2.jar, neo4j-fulltext-index-4.0.2.jar, neo4j-kernel-4.0.2.jar, neo4j-ssl-4.0.2.jar define 1 overlapping resources: 
[WARNING]   - META-INF/services/org.neo4j.configuration.SettingsDeclaration

It is my understanding that the maven-shade-plugin will only keep the last instance of the respective resource it sees during its execution. The ServicesResourceTransformer will aggregate the resources by appending their content.

Vincevincelette

Answer 2

I had a very similar bug, and just found a fix after hours of debugging. I suspect it'll be the same thing for you.

Problem

The problem is related to service loading, a Java feature I previously knew nothing about. In order to remain "pluggable", neo4j loads a lot of its services dynamically, by declaring an interface that describes a "service" and implementing it with several "service providers" that are identified at runtime. This is a fairly common pattern in Java projects.

One example is the interface org.neo4j.configuration.SettingsDeclaration, which has implementations in various packages in the neo4j project, including org.neo4j.configuration.GraphDatabaseSettings, which is needed in order for any graph databases to be configured. When run from a local build (in your case, your IDE) these are all found in the classpath and everything works fine. But when run from an executable jar, we need to declare them all explicitly in a file called META-INF/services/org.neo4j.configuration.SettingsDeclaration, on one line each.

The problem arises because maven-assembly-plugin copies these services/ files fairly dumbly, and the declared services from one package can overwrite those from another. For me this meant that only one implementation of SettingsDeclaration was being loaded, where there should have been 15. This causes things to fail in mysterious ways, with unhelpful error messages appearing when you actually try to create a database.

Solution

If two services/* files exist with the same name, in different imported packages, we need to merge them instead of overwriting. Fortunately, maven-assembly-plugin has a feature just for this: the metaInf-services handler. We need to configure the plugin to use this handler.

Your pom.xml contains the following element in the configuration of the plugin:

        <descriptorRefs>
            <descriptorRef>jar-with-dependencies</descriptorRef>
        </descriptorRefs>

This is telling the plugin to use the pre-defined jar-with-dependencies descriptor, which does most of what we want. But to get the behaviour we want, we'll need to create our own descriptor.

Replace the above descriptorRefs element with the following:

                    <descriptors>
                        <descriptor>src/assembly/my-jar-descriptor.xml</descriptor>
                    </descriptors>

This is telling the plugin to look in src/assembly/my-jar-descriptor.xml for a descriptor that it can use to build the jar. You should create such a file, with the following contents:

<!-- Based on jar-with-dependencies, one of the defaults available in Maven -->
<assembly xmlns="http://maven.apache.org/ASSEMBLY/2.1.1"
          xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
          xsi:schemaLocation="http://maven.apache.org/ASSEMBLY/2.1.1 https://maven.apache.org/xsd/assembly-2.1.1.xsd">
    <id>complete</id>
    <formats>
        <format>jar</format>
    </formats>
    <includeBaseDirectory>false</includeBaseDirectory>
    <dependencySets>
        <dependencySet>
            <outputDirectory>/</outputDirectory>
            <useProjectArtifact>true</useProjectArtifact>
            <unpack>true</unpack>
            <scope>runtime</scope>
        </dependencySet>
    </dependencySets>

    <!-- This element added to make sure files in META-INF/services are merged from all packages. -->
    <containerDescriptorHandlers>
        <containerDescriptorHandler>
            <handlerName>metaInf-services</handlerName>
        </containerDescriptorHandler>
    </containerDescriptorHandlers>
</assembly>

This is identical to the definition of jar-with-dependencies which you can find on the Maven website. Except that we've added that one containerDescriptorHandlers element at the bottom, which tells the plugin to use the metaInf-services handler. This means that the plugin will merge the services files instead of overwriting them, and so neo4j should have access to all the configuration stuff it needs, even when loaded from a jar.

I hope this works for you. It felt like a miracle to me (a Maven newbie) after countless hours of debugging and questioning my life choices.

I wonder why this behaviour isn't included in jar-with-dependencies by default. It seems harmless, and would've saved me a lot of trouble.

Tenaille

Answer 3

As the previous answers correctly pointed out the problem is creating a fat jar without merging the service descriptors. Since both answers cover Maven, here's the Gradle solution.

Use Shadow to create your fat jar and make sure to enable merging:

shadowJar {
    mergeServiceFiles()
}

Almire