kafka NoClassDefFoundError kafka/Kafka

Question

Regarding Apache-Kafka messaging queue.

I have downloaded Apache Kafka from the Kafka download page. I've extracted it to /opt/apache/installed/kafka-0.7.0-incubating-src.

The quickstart page says you need to start zookeeper and then start Kafka by running:
>bin/kafka-server-start.sh config/server.properties

I'm using a separate Zookeeper server, so i edited config/server.properties to point to that Zookeeper instance.

When i run Kafka, as instructed in the quickstart page, I get the following error:

Exception in thread "main" java.lang.NoClassDefFoundError: kafka/Kafka
Caused by: java.lang.ClassNotFoundException: kafka.Kafka
        at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:252)
        at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:320)
Could not find the main class: kafka.Kafka.  Program will exit.

I used telnet to make sure the Zookeeper instance is accessible from the machine that Kafka runs on. Everything is OK.

Why am i getting this error?

Tufa

Answer 1

You must first build Kafka by running the following commands:

> ./sbt update
> ./sbt package 

Only then will Kafka be ready for use.

Tufa

Answer 2

You should know that

./sbt update
./sbt package

will produce Kafka binaries for Scala 2.8.0 by default. If you need it for a different version, you need to do

./sbt "++2.9.2 update"
./sbt "++2.9.2 package"

replacing 2.9.2 with the desired version number. This will make the appropriate binaries. In general, when you switch versions, you should run

./sbt clean

to clean up the binaries from previous versions.

Actually, in addition, you might also need to perform this command

./sbt "++2.9.2 assembly-package-dependency"

This command resolves all the dependencies for running Kafka, and creates a jar that contains just these. Then the start scripts would add this to the class path and you should have all your desired classes.

Fachanan

Answer 3

It seems that without the SCALA_VERSION environment variable, the executable doesn't know how to load the libraries necessary. Try the following from the Kafka installation directory:

SCALA_VERSION=2.9.3 bin/kafka-server-start.sh config/server.properties

See http://kafka.apache.org/documentation.html#quickstart.

Kalil

Answer 4

You can find following two options on Kafka downloads page

https://kafka.apache.org/downloads.html

Source download:
Binary downloads

You have downloaded "kafka-0.7.0-incubating-src" it' source code

Download the binary package of Kafka Scala 2.10 - kafka_2.10-0.10.1.1.tgz (asc, md5)

Bismuthinite

Answer 5

Just to add to the previous answer, if you're running IntelliJ, and want to run Kafka inside IntelliJ and/or step through it, make sure to run

> ./sbt idea

I spent easily half a day trying to create the IntelliJ project from scratch, and it turns out that single command was all I needed to get it working. Also, make sure you have the Scala plugin for IntelliJ installed.

Conjunctive

Answer 6

You can also use the binary downloads provided by Apache.

For example download kafka version - 0.9.0.1 from this link.

For other version download from link2 and download the binary versions instead. These are already built version. Not need to build again using Scala.

Use are using the source download instead.

Pichardo

Answer 7

You've downloaded the source version. Download the binary package of Kafka and proceed with your testing.

Normally

This question was asked 4 years ago. At that time, the source code option was the only option. – Tufa