Starting Kafka Server Permanently

Question

I have setup Kafka on Amazon EC2 instance.

I have done the following in below order: (1) SSH into the Instance (2) Start Zookeper (3) Start Kafka (4) Execute Producer and Consumer Programs.

Everything is working fine till here. However once I close the SSH window on which I have started Kafka, the Kafka service stops. I can no longer execute Producer and Consumer programs.

How can I have the Kafka Server permanently up for all requests, even after I close the SSH window.

Thank You.

Ochlocracy

Answer 1

nohup is required at the beginning of command, so that output is not on screen, but in file. Also & is required at the end of command to start the server in background:

bin/zookeeper-server-start.sh config/zookeeper.properties

bin/kafka-server-start.sh config/server.properties

will change to:

nohup bin/zookeeper-server-start.sh config/zookeeper.properties &

nohup bin/kafka-server-start.sh config/server.properties &

Ochlocracy

Using -daemon switch is the better option now – Querist

@Querist would you elaborate why that is the case? – Castoff

@RichardSteinbrecht nohup .. & is technically doing what -daemon supports already. To run the kafka server headless as a daemon – Querist

Answer 2

This is now officially supported in kafka and zookeeper startup scripts. So if you are on latest (Since Aug 2015) kafka you can use -daemon as follows.

# ./kafka-server-start.sh
USAGE: ./kafka-server-start.sh [-daemon] server.properties

# ./zookeeper-server-start.sh
USAGE: ./zookeeper-server-start.sh [-daemon] zookeeper.properties

Posehn

@NikeshDevaki, -daemon is what you should use now since that's supported by kafka directly. – Posehn

This does not work on Windows (starting ZK, Kafka using .bat files) – Handgrip

Is there a way to get the logs to be displayed as it initializes? It seems like the -daemon flag hides all of those so it becomes harder to troubleshoot potential startup exceptions. – Pyx

Answer 3

Try bin/kafka-server-start.sh -daemon config/server.properties.

OR:

Try upstart script here:upstart script for kafka

Zachariahzacharias

no idea why so few people upvoted my answer, but the following one.... No much difference – Zachariahzacharias

after executing above shell, how to check if it already running? – Kaleidoscopic

how to stop zookeeper and Kafka if it's unnecessary ? – Tate