NoClassDefFoundError: org/apache/hadoop/conf/Configuration

Question

I am trying to install oozie and getting this error.I have hadoop 2.7.1, maven 3.3.3 .Any suggestion on this?

huseyin@ubuntu:~$ '/usr/local/oozie/oozie/Oozie/oozie-4.3.0-SNAPSHOT/bin/oozie-setup.sh' sharelib create -fs hdfs://hadoopcluster:10000 setting CATALINA_OPTS="$CATALINA_OPTS -Xmx1024m"

Error: A JNI error has occurred, please check your installation and try again Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/conf/Configuration at java.lang.Class.getDeclaredMethods0(Native Method) at java.lang.Class.privateGetDeclaredMethods(Class.java:2701) at java.lang.Class.privateGetMethodRecursive(Class.java:3048) at java.lang.Class.getMethod0(Class.java:3018) at java.lang.Class.getMethod(Class.java:1784) at sun.launcher.LauncherHelper.validateMainClass(LauncherHelper.java:544) at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:526) Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.conf.Configuration at java.net.URLClassLoader.findClass(URLClassLoader.java:381) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ... 7 more

Fermanagh

Why focus blindly on the JNI keyword? the NoClassDefFoundError is quite explicit: check the CLASSPATH to see if the Hadoop libs are in there... – Slither

Answer 1

You need to add all the needed jars for oozie to a new folder in your working directory (oozie-4.3.0-SNAPSHOT) called libext , please check the following paragraph from oozie docs

By default, oozie war will not contain hadoop and hcatalog libraries, however they are required for oozie to work.

There are 2 options to add these libraries:

1. At install time, copy the hadoop and hcatalog libraries to libext and run oozie-setup.sh to setup oozie war.

2. Build with -Puber which will bundle the required libraries in the oozie war.

so to solve the problem you need to :

1- Create a libext/ directory in the directory where Oozie was expanded.

2- If using the ExtJS library copy the ZIP file to the libext/ directory.

3- If hadoop and hcatalog libraries are not already included in the war, add the corresponding libraries to libext/ directory. (In your case ) , you can just copy all the jars from your hadoop project project to the libext folder.

Important Note : the versions of the jars inside the libext folder should match the versions of those installed during the build process.

You can find the versions of the ones downloaded in the pom file you used during the build process.

Kassel

Answer 2

cp  $HADOOP_HOME/share/hadoop/**/*.jar libext/
cp  $HADOOP_HOME/share/hadoop/common/*.jar libext/
cp  $HADOOP_HOME/share/hadoop/common/lib/*.jar libext/
cp  $HADOOP_HOME/share/hadoop/hdfs/lib/*.jar libext/ 
cp  $HADOOP_HOME/share/hadoop/hdfs/*.jar libext/
cp  $HADOOP_HOME/share/hadoop/mapreduce/*.jar libext/
cp  $HADOOP_HOME/share/hadoop/mapreduce/lib/*.jar libext/
cp  $HADOOP_HOME/share/hadoop/yarn/lib/*.jar libext/
cp  $HADOOP_HOME/share/hadoop/yarn/*.jar libext/

Nomenclature