Tomcat 6, JMX and the dynamic port problem

Question

after reading and trying a lot around, I have to ask if anyone has a solution for my problem.

I am trying to set up some Tomcats (V6) behind a firewall. This is no big deal - but I want to monitor them via JMX.

I read the TC docu and came across the JMXRemoteLifecycleListener. My test TC installation is set up exactly as stated in the above link. Thus I don't get a connection from one host in our network to another. Additionally a third, random port is open every time I start TC.

In my server.xml the listener is activated

<Listener className="org.apache.catalina.mbeans.JmxRemoteLifecycleListener"
rmiRegistryPortPlatform="8050" rmiServerPortPlatform="8060" />

catalina.out says that everything is okay.

2011-06-14 16:46:48,819 [main] INFO org.apache.catalina.mbeans.JmxRemoteLifecycleListener-
The JMX Remote Listener has configured the registry on port 8050 and the server on port 8060 for the Platform server

The ports are open, I can connect to them via telnet from any other host. I am able to connect to to the vm locally with (service:jmx:rmi://<hostname>:8xxx/jndi/rmi://<hostname>:8xxxx/jmxrmi)

Netstats output is as follows:

tcp6       0      0 :::8080                 :::*                    LISTEN      11291/java
tcp6       0      0 :::8050                 :::*                    LISTEN      11291/java
tcp6       0      0 :::8060                 :::*                    LISTEN      11291/java
tcp6       0      0 127.0.0.1:8005          :::*                    LISTEN      11291/java
tcp6       0      0 :::60901                :::*                    LISTEN      11291/java
tcp6       0      0 127.0.0.1:8009          :::*                    LISTEN      11291/java

Tomcat is even started with all sufficient VM options

CATALINA_OPTS="-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.ssl=false
-Dcom.sun.management.jmxremote.authenticate=TRUE 
-Dcom.sun.management.jmxremote.password.file=$CATALINA_HOME/conf/jmxremote.password
-Dcom.sun.management.jmxremote.access.file=$CATALINA_HOME/conf/jmxremote.access"

Does anyone has a hint why I am stuck here? Thanks in advance!

Leannleanna

try setting the port in the VM options It looks like you are missing that in your first line of the Cat opts -Dcom.sun.management.jmxremote.port=8086 – Magellan

@Magellan Regarding the documentation this Option should be a avoided if you are using the TC listener. And even with this option it does not work. – Subprincipal

Gotcha, glossed over too quick the first time. If its running locally without a problem on those ports, You could be hitting this issue. olegz.wordpress.com/2009/03/23/… halfway down in the article he discusses the private address being pushed to the client and not the public one. – Magellan

Thanks a lot, with putting -Djava.rmi.server.hostname=IP in my startup-script it worked :) But is it possible to have this done automatically out of TC? Its somewhat annoying to do this for a bunch of instances... – Subprincipal

Answer 1

The answer is putting -Djava.rmi.server.hostname=xxx.xxx.xxx.xxx to the general JMX options.

An example to automatically set the hostname:

IP=`ifconfig eth0  | grep 'inet '| grep -v '127.0.0.1' | cut -d: -f2 | awk '{ print $1}'``;

CATALINA_OPTS="-Dcom.sun.management.jmxremote
-Dcom.sun.management.jmxremote.ssl=false
-Dcom.sun.management.jmxremote.authenticate=TRUE
-Dcom.sun.management.jmxremote.password.file=$CATALINA_HOME/conf/jmxremote.password
-Dcom.sun.management.jmxremote.access.file=$CATALINA_HOME/conf/jmxremote.access
-Djava.rmi.server.hostname=$IP"

Leannleanna

Answer 2

Problem maybe the second random port opened by java itself when enabling jmx monitoring. Starting from Java 7 this port can be set, too:

-Dcom.sun.management.jmxremote.rmi.port=7091

So in combination - same port can be used:

-Dcom.sun.management.jmxremote.port=7091 -Dcom.sun.management.jmxremote.rmi.port=7091

That was the solution for my problem. Once the port is set to a fixed number, it can be easily set in the firewall.

Hohenstaufen