Error connecting to remote JMX agent! while starting Nodetool

Question

I am trying to use nodetool against our Cassandra production database. But I am getting exception as soon as I try to start nodetool against Cassandra production cluster from my local machine.

S:\Apache Cassandra\apache-cassandra-1.2.3\bin>nodetool -h pdb.vip.phx.host.com compactionstats
Starting NodeTool
Error connecting to remote JMX agent!
java.io.IOException: Failed to retrieve RMIServer stub: javax.naming.CommunicationException [Root exception is java.rmi.ConnectIOException: error duri
ng JRMP connection establishment; nested exception is:
        java.net.SocketTimeoutException: Read timed out]
        at javax.management.remote.rmi.RMIConnector.connect(Unknown Source)
        at javax.management.remote.JMXConnectorFactory.connect(Unknown Source)
        at org.apache.cassandra.tools.NodeProbe.connect(NodeProbe.java:145)
        at org.apache.cassandra.tools.NodeProbe.<init>(NodeProbe.java:115)
        at org.apache.cassandra.tools.NodeCmd.main(NodeCmd.java:966)
Caused by: javax.naming.CommunicationException [Root exception is java.rmi.ConnectIOException: error during JRMP connection establishment; nested exce
ption is:
        java.net.SocketTimeoutException: Read timed out]
        at com.sun.jndi.rmi.registry.RegistryContext.lookup(Unknown Source)
        at com.sun.jndi.toolkit.url.GenericURLContext.lookup(Unknown Source)
        at javax.naming.InitialContext.lookup(Unknown Source)
        at javax.management.remote.rmi.RMIConnector.findRMIServerJNDI(Unknown Source)
        at javax.management.remote.rmi.RMIConnector.findRMIServer(Unknown Source)
        ... 5 more
Caused by: java.rmi.ConnectIOException: error during JRMP connection establishment; nested exception is:
        java.net.SocketTimeoutException: Read timed out
        at sun.rmi.transport.tcp.TCPChannel.createConnection(Unknown Source)
        at sun.rmi.transport.tcp.TCPChannel.newConnection(Unknown Source)
        at sun.rmi.server.UnicastRef.newCall(Unknown Source)
        at sun.rmi.registry.RegistryImpl_Stub.lookup(Unknown Source)
        ... 10 more
Caused by: java.net.SocketTimeoutException: Read timed out
        at java.net.SocketInputStream.socketRead0(Native Method)
        at java.net.SocketInputStream.read(Unknown Source)
        at java.io.BufferedInputStream.fill(Unknown Source)
        at java.io.BufferedInputStream.read(Unknown Source)
        at java.io.DataInputStream.readByte(Unknown Source)
        ... 14 more

Can anyone provide some thoughts why I am seeing this exception? Any help will be appreciated.

Secretory

Make sure that the JMX port listens on that interface – Supererogatory

Read wiki.apache.org/cassandra/JmxGotchas – Supererogatory

Answer 1

"Use JVM option '-Djava.rmi.server.hostname=$IP' (where $IP is the ip address of the interface you want to reach from the remote machine)."

From: wiki.apache.org/cassandra/JmxGotchas

Supererogatory

Thanks for suggestion. In my local dev box from where I am trying to start the nodetool to see the stats of my production db cluster, I uncommented out the line that you mentioned above like this JVM_OPTS="$JVM_OPTS -Djava.rmi.server.hostname=10.43.96.33" and then I am trying to connect to that IP Address using nodetool, I am still getting the same exception. Is there anything else I am missing? – Secretory