Access tomcat running in docker container by JMX

Question

I want to get tomcat's performance by JMX. Tomcat version is 7 and Java version is 1.7.0_79.

Tomcat runs on the docker's container. The port mapping has set up by the following.

docker run -itd -p 18080:8080 -p 19998:19998 -p 62911:62911 sad_jang2 bash

In the tomcat container, setevn.sh was created in the path, /tomcat_home/bin/ with the following content.

export JAVA_OPTS="
-Xdebug -Xrunjdwp:transport=dt_socket,address=62911,server=y,suspend=n
-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=19998
-Dcom.sun.management.jmxremote.authenticate=false
-Dcom.sun.management.jmxremote.ssl=false
-Djava.rmi.server.hostname=172.17.0.16"

I write a simple java code as

JMXServiceURL serviceURL = new JMXServiceURL("service:jmx:rmi:///jndi/rmi://" + "172.17.0.16"+ ":" + "19998" + "/jmxrmi");
JMXConnector connector = JMXConnectorFactory.connect(serviceURL, null);
MBeanServerConnection mbsc = connector.getMBeanServerConnection();

and can get related values by JMX.

However, I run this code in my desktop with the docker host's address and 19998 port such as

JMXServiceURL serviceURL = new JMXServiceURL("service:jmx:rmi:///jndi/rmi://" + "10.70.1.5"+ ":" + "19998" + "/jmxrmi");

The console displayed the following message

java.rmi.ConnectException: Connection refused to host: 172.17.0.16; nested exception is: 
java.net.ConnectException: Connection timed out: connect
at sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:619)
at sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:216)
at sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:202)
at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:130)
at javax.management.remote.rmi.RMIServerImpl_Stub.newClient(Unknown Source)
at javax.management.remote.rmi.RMIConnector.getConnection(RMIConnector.java:2432)
at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:308)
at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:270)
at utility.GetStatistic.getTomcatStats(GetStatistic.java:84)
at exec.Main.main(Main.java:53)

Caused by: java.net.ConnectException: Connection timed out: connect
at java.net.DualStackPlainSocketImpl.connect0(Native Method)
at java.net.DualStackPlainSocketImpl.socketConnect(DualStackPlainSocketImpl.java:79)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:345)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:172)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at java.net.Socket.connect(Socket.java:538)
at java.net.Socket.<init>(Socket.java:434)
at java.net.Socket.<init>(Socket.java:211)
at sun.rmi.transport.proxy.RMIDirectSocketFactory.createSocket(RMIDirectSocketFactory.java:40)
at sun.rmi.transport.proxy.RMIMasterSocketFactory.createSocket(RMIMasterSocketFactory.java:148)
at sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:613)
... 9 more

Using the browser to connect the URL http://10.70.1.5:18080 can show the tomcat root's page.

Can anyone give any suggestions? Thanks.

Chiefly

What happens if you leave out the -Djava.rmi.server.hostname=172.17.0.16 option? This option should offer a default value, Cf. #10174334 – Lyallpur

Answer 1

Here's the flags I've successfully used for monitoring my ZooKeeper container via JMX:

-Dcom.sun.management.jmxremote
-Dcom.sun.management.jmxremote.local.only=false
-Dcom.sun.management.jmxremote.authenticate=false
-Dcom.sun.management.jmxremote.port=9010
-Dcom.sun.management.jmxremote.rmi.port=9010
-Djava.rmi.server.hostname=0.0.0.0
-Dcom.sun.management.jmxremote.ssl=false

So the difference would be this entry -Dcom.sun.management.jmxremote.rmi.port which is fix the RMI port to be the same as JMX port.

With this I can just docker run -p 9010:9010 ... and everything is fine for me.

Cletis

Thanks. I use the following configuration and can monitor Tomcat via JMX. the field of rmi.port you suggested was added and hostname was set with external address, docker host's address. CATALINA_OPTS="${CATALINA_OPTS} -Xdebug -Xrunjdwp:transport=dt_socket,address=62911,server=y,suspend=n -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.local.only=false -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.port=9998 -Dcom.sun.management.jmxremote.rmi.port=9998 -Djava.rmi.server.hostname=10.70.1.5 -Dcom.sun.management.jmxremote.ssl=false" – Chiefly

This does not work when you run that container in Docker machine - in VirtualBox. I am not sure what kind of address I should add to java.rmi.server.hostname because when I add there external IP of the Docker container I am connecting from my host, it does not work. So basically situation is like HOST ---> (MACHINE --- CONTAINER) and I can not reach it from HOST to CONTAINER. – Courier

Answer 2

This does not work when you run that container in Docker machine - in VirtualBox. I am not sure what kind of address I should add to java.rmi.server.hostname because when I add there external IP of the Docker container I am connecting from my host, it does not work. So basically situation is like HOST ---> (MACHINE --- CONTAINER) and I can not reach it from HOST to CONTAINER.

Courier

i reply to myself :) you have to set these properties: -Djava.rmi.server.hostname=$JAVA_RMI_SERVER_HOSTNAME -Dactivemq.jmx.url=service:jmx:rmi:///jndi/rmi://$JAVA_RMI_SERVER_HOSTNAME:$ACTIVEMQ_JMX_PORT/jmxrmi in such way that JAVA_RMI_SERVER_HOSTNAME address has to be set to external address of the Docker machine in case running in machine but it has to be set to external IP of the container itself when running in the native mode. Keep in mind that you have to export and map these ports from container to your local host. – Courier

Answer 3

I'm not sure if it's necessary but I would recommend adding this to your additional options -Djgroups.bind_addr=172.17.0.16. Also I'm pretty sure that this kind of options should probably go to CATALINA_OPTS and not JAVA_OPTS but that shouldn't make a difference. Another thing that I noticed is that the rmi server host name you defined is different than the one in the JMXServiceURL constructor. I'm no docker expert but shouldn't this be identical?

Hendecagon

172.17.0.16 is internal address for docker's container. If a container doesn't have any settings, any clients outside the container can't connect to it. But, docker provides the functions that containers can expose some port and set a port mapping such as -p 18080:8080. Thus, I can use the URL 10.70.1.5:18080 to connect to the tomcat's root page running in the container. – Chiefly

I get this but I would expect you would use 127.0.0.1 cause as far as tomcat and rmi jmx are concerned, they are running on localhost. Which port they will be exposed on on the host machine is another story. – Hendecagon

Answer 4

This config worked for me:

-Dcom.sun.management.jmxremote
-Dcom.sun.management.jmxremote.local.only=false 
-Dcom.sun.management.jmxremote.authenticate=false 
-Dcom.sun.management.jmxremote.port=10005 
-Dcom.sun.management.jmxremote.rmi.port=10005 
-Djava.rmi.server.hostname=127.0.0.1 
-Dcom.sun.management.jmxremote.ssl=false

The important part was to to configure the rmi server hostname with the local loopback address. If i used 0.0.0.0 i wasn't able to reach the exposed port localy via an SSH Tunnel, with 127.0.0.1 everything worked fine.

Note: if you're running docker inside virtualbox things are different... you have to use the ip which virtualbox gives you here.

Lochia

Answer 5

Need to expose both connector port and jmx port while running a docker image which is built by docker file.

docker run -it -p 8080:8080 -p 9997:9997 <build_tag> image_name

Where, 8080 is connector port and 9997 is jmx port which is being exposed as 9997 so that jmx client can connect to Tomcat mbeans.

Plush