How to enable remote JMX on Kafka brokers (for JmxTool)?

Asked 19/4, 2016 at 4:8 Answered 15/2 at 8:11

I enabled JMX on Kafka brokers by adding

KAFKA_JMX_OPTS="-Dcom.sun.management.jmxremote=true
                -Dcom.sun.management.jmxremote.authenticate=false
                -Dcom.sun.management.jmxremote.ssl=false
                -Djava.rmi.server.hostname=<server_IP>
                -Djava.net.preferIPv4Stack=true"

However, when I use kafka.tools.JmxTool to get the JMX metrics, it outputs Unix timestamps only. Why?

./bin/kafka-run-class.sh kafka.tools.JmxTool \
  --object-name 'kafka.server:type=BrokerTopicMetrics,name=AllTopicsMessagesInPerSec' \
  --jmx-url "service:jmx:rmi:///jndi/rmi://<server_IP>:9111/jmxrmi"

How can I have it print out the metrics?

Aerify answered 19/4, 2016 at 4:8 Comment(1)

editing the sh files is not a good idea. why is everyone recommending it? – Bloodcurdling 31/5, 2022 at 13:20

Edit bin/kafka-run-class.sh and set KAFKA_JMX_OPTS variable

KAFKA_JMX_OPTS="-Dcom.sun.management.jmxremote=true -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Djava.rmi.server.hostname=your.kafka.broker.hostname -Djava.net.preferIPv4Stack=true"

Update bin/kafka-server-start.sh add the below line

export JMX_PORT=PORT

Landin answered 2/8, 2017 at 5:53 Comment(4)

Only export JMX_PORT=PORT before start of Kafka service is enough to enable JMX on my mac. I tested this with jconsole. And if you want to connect producer and consumer, set/export different jmx port before start of each. – Lightyear 20/4, 2018 at 13:2

is it necessary to restart kafka ? – Lamellibranch 23/9, 2018 at 16:50

@DivyanshuJimmy yeap :) – Bennettbenni 16/11, 2018 at 13:43

You can export outside of the script. Don't modify it – Cropdusting 17/2, 2021 at 15:23

Setting JMX_PORT inside bin/kafka-run-class.sh will clash with Zookeeper, if you are running Zookeeper on the same node. Best is to set JMX port individually inside corresponding server-start scripts:

Insert line “export JMX_PORT=${JMX_PORT:-9998}” before last line in $KAFKA_HOME/bin/zookeeper-server-start.sh file.
Restart the Zookeeper server.
Repeat steps 1 and 2 for all zookeeper nodes in the cluster.
Insert line “export JMX_PORT=${JMX_PORT:-9999}” before last line in $KAFKA_HOME/bin/kafka-server-start.sh file.
Restart the Kafka Broker.
Repeat steps 4 and 5 for all brokers in the cluster.

Vestigial answered 21/2, 2018 at 23:23 Comment(4)

This answer is perfectly worked for my problem. I'm using kafka_2.11-2.0.0 & zookeeper-3.4.12 together as 3 node cluster. This solution worked for Kafka-Manager's JMX based metrics display charts as well as kafka-topics.sh --list --zookeeper & Kafka Console Producer (kafka-console-producer.sh --broker-list ) & Console Consumer (kafka-console-consumer.sh --bootstrap-server) & kafka-topics.sh --describe. Zookeeper & Kafka both are working on different JMX ports. Zookeeper working on port 9998 and Kafka & Kafka-Manager working on port 9999. Thank you @Rahul Singhai – Scleroprotein 20/10, 2018 at 18:16

Every Kafka Broker using same JMX port? – Grata 1/11, 2019 at 0:35

No, if running multiple Kafka Brokers or ZKs on same node, then you need to specify different JMX ports for each of them. – Vestigial 6/11, 2019 at 16:52

I suggest not modifying the shell scripts because those shouldn't be kept across server upgrades – Cropdusting 17/2, 2021 at 15:22

If you're running via systemd:

edit /etc/systemd/system/multi-user.target.wants/kafka.service
in the "[service]" section add a line:
- Environment=JMX_PORT=9989
reload: systemctl daemon-reload
restart: systemctl restart kafka
enjoy the beans: echo 'beans' | java -jar jmxterm-1.0-alpha-4-uber.jar -l localhost:9989 -n 2>&1

Unfrock answered 24/8, 2018 at 11:31 Comment(0)

You must set 'JMX_PORT' variable, or add the following line to bin/kafka-server-start.sh.

export JMX_PORT=${JMX_PORT:-9999}

then you will be able to connect to Kafka JMX metrics. I use jconsole tool and 'localhost:9999' address.

Monaxial answered 21/2, 2017 at 16:42 Comment(0)

This is Kafka 2.3.0.

Using jconsole For Available MBeans

You should use jconsole first to know the names of the MBeans available.

The proper name of the MBean you wanted to query metrics of is kafka.server:type=BrokerTopicMetrics,name=MessagesInPerSec (the AllTopics prefix was used in older verions). Thanks AndyTheEntity.

Enabling Remote JMX (with no authentication or SSL)

As described in Monitoring and Management Using JMX Technology you should set certain system properties when you start the Java VM of a Kafka broker.

Kafka's bin/kafka-run-class.sh shell script makes the configuration painless as it does the basics for you and sets KAFKA_JMX_OPTS.

# JMX settings
if [ -z "$KAFKA_JMX_OPTS" ]; then
  KAFKA_JMX_OPTS="-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.authenticate=false  -Dcom.sun.management.jmxremote.ssl=false "
fi

For remote JMX you should set com.sun.management.jmxremote.port that Kafka's bin/kafka-run-class.sh shell script sets using JMX_PORT environment variable.

# JMX port to use
if [  $JMX_PORT ]; then
  KAFKA_JMX_OPTS="$KAFKA_JMX_OPTS -Dcom.sun.management.jmxremote.port=$JMX_PORT "
fi

With that, enabling remote JMX is as simple as the following command:

JMX_PORT=9999 ./bin/kafka-server-start.sh config/server.properties

Using JmxTool

With the above, run the JmxTool:

$ ./bin/kafka-run-class.sh kafka.tools.JmxTool \
  --object-name 'kafka.server:type=BrokerTopicMetrics,name=MessagesInPerSec'
Trying to connect to JMX url: service:jmx:rmi:///jndi/rmi://:9999/jmxrmi.
"time","kafka.server:type=BrokerTopicMetrics,name=MessagesInPerSec:Count","kafka.server:type=BrokerTopicMetrics,name=MessagesInPerSec:EventType","kafka.server:type=BrokerTopicMetrics,name=MessagesInPerSec:FifteenMinuteRate","kafka.server:type=BrokerTopicMetrics,name=MessagesInPerSec:FiveMinuteRate","kafka.server:type=BrokerTopicMetrics,name=MessagesInPerSec:MeanRate","kafka.server:type=BrokerTopicMetrics,name=MessagesInPerSec:OneMinuteRate","kafka.server:type=BrokerTopicMetrics,name=MessagesInPerSec:RateUnit"
1567586728595,0,messages,0.0,0.0,0.0,0.0,SECONDS
1567586730597,0,messages,0.0,0.0,0.0,0.0,SECONDS
...

You could use --one-time option to print the JMX metrics just once.

$ ./bin/kafka-run-class.sh kafka.tools.JmxTool \
    --object-name 'kafka.server:type=BrokerTopicMetrics,name=MessagesInPerSec' \
    --one-time true
Trying to connect to JMX url: service:jmx:rmi:///jndi/rmi://:9999/jmxrmi.
"time","kafka.server:type=BrokerTopicMetrics,name=MessagesInPerSec:Count","kafka.server:type=BrokerTopicMetrics,name=MessagesInPerSec:EventType","kafka.server:type=BrokerTopicMetrics,name=MessagesInPerSec:FifteenMinuteRate","kafka.server:type=BrokerTopicMetrics,name=MessagesInPerSec:FiveMinuteRate","kafka.server:type=BrokerTopicMetrics,name=MessagesInPerSec:MeanRate","kafka.server:type=BrokerTopicMetrics,name=MessagesInPerSec:OneMinuteRate","kafka.server:type=BrokerTopicMetrics,name=MessagesInPerSec:RateUnit"
1567586898459,0,messages,0.0,0.0,0.0,0.0,SECONDS

Essy answered 4/9, 2019 at 8:50 Comment(0)

vim kafka_2.11-0.10.1.1/bin/kafka-run-class.sh

and then add the first two lines and comment as I have done for other lines, (Note : after doing this Kafka scripts cannot be used for client operations for listing topics.. for your client operations you need to use a separate scripts , download again in different locations and use)

export JMX_PORT=9096
KAFKA_JMX_OPTS="-Dcom.sun.management.jmxremote=true -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Djava.rmi.server.hostname=<ipaddress> -Dcom.sun.management.jmxremote.port=$JMX_PORT -Dcom.sun.management.jmxremote.rmi.port=$JMX_PORT"



# JMX settings
#if [ -z "$KAFKA_JMX_OPTS" ]; then
# KAFKA_JMX_OPTS="-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.authenticate=false  -Dcom.sun.management.jmxremote.ssl=false "

#fi

# JMX port to use
#if [  $JMX_PORT ]; then
#  KAFKA_JMX_OPTS="$KAFKA_JMX_OPTS -Dcom.sun.management.jmxremote.port=$JMX_PORT "
#fi

Desimone answered 8/5, 2017 at 17:35 Comment(1)

Right idea, wrong implementation. You're right that you may need to set other JMX JVM args, but you should set those variables before invoking the script, don't modify the script itself. You're setting yourself up for a maintenance headache. – Loom 2/6, 2017 at 19:34

This is standard Kafka start procedure:

bin/kafka-server-start.sh config/server.properties

This is Kafka start procedure with JMX:

JMX_PORT=8004 bin/kafka-server-start.sh config/server.properties

Interleave answered 24/8, 2021 at 17:22 Comment(0)

Use kafka.server:type=BrokerTopicMetrics,name=MessagesInPerSec

The AllTopics prefix was used in older verions. You can specify topic using kafka.server:type=BrokerTopicMetrics,name=MessagesInPerSec,topic=<topic-name>

src: http://grokbase.com/t/kafka/users/164ksnhff0/enable-jmx-on-kafka-brokers

Talbott answered 12/10, 2016 at 9:36 Comment(0)

Kafka has provided all you need. When your start your server, activate the KAFKA_JMX_OPTS arg by using these command:

$KAFKA_JMX_OPTS JMX_PORT=[your_port_number] ./kafka-server-start.sh -daemon ../config/server.properties

Using those command, you activated JMX Remote and related port. Then you can connect your JConsole or another monitoring tools.

Grata answered 1/11, 2019 at 3:57 Comment(0)

Just before calling kafka-server-start.sh add following exports. It worked like a charm for my case. You can set desired port for JMX_PORT and you should set broker for $BROKER_IP part.

   export JMX_PORT=9900
   export KAFKA_JMX_OPTS="-Dcom.sun.management.jmxremote=true -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Djava.rmi.server.hostname=$BROKER_IP -Djava.net.preferIPv4Stack=true"

Maricamarice answered 17/12, 2020 at 18:27 Comment(0)

I just want to share my experience related to JMX_PORT

I'm using windows OS. and I configure my JMX_PORT under \bin\windows\kafka-run-class.bat.

Since I'm using zookeeper and kafka server on the same node, and I start the zookeeper first (successfully start). And then I start the kafka server (failed start). And the reason it failed is because the script to run both zookeeper and kafka server refer to the same script that set the JMX_PORT(kafka-run-class.bat).

What I do is modify the kafka-run-class.bat by adding parameter checking to decide the JMX_PORT to be use by zookeeper and kafka server start script.

IF %1 == kafka.Kafka (
    rem using port 8005 for the Kafka server 
    set JMX_PORT=8005
)
IF %1 == org.apache.zookeeper.server.quorum.QuorumPeerMain (
    rem using port 8004 for the ZooKeeper
    set JMX_PORT=8004
)

Gautious answered 15/2 at 8:11 Comment(0)

Hot tags

Godot Unity Godot Help Programming Godot 4.X GUI GDScript 3D 2D Physics CSharp Godot 3.X VR XR Projects C++

Using jconsole For Available MBeans

Enabling Remote JMX (with no authentication or SSL)

Using JmxTool

Recommended topics

Hot tags