Monitor JVM in AWS Fargate

Question

I have been currently trying to connect VisualVM (A program which monitors the JVM, heap and memory usage etc) to a Spring Boot Application (Java App) running on AWS Fargate in Docker containers.

I have been exposing the JMX ports accordingly and I am able to connect through the JMX ports when running the Docker container locally. However, when running the Java App on Fargate, I have not found a way to connect to the Container through JMX. I have tried setting the VM argument -Djava.rmi.server.hostname to the IP Address of the container, but when I try to connect through JMX it still fails to do so. Has anyone had any experience with this?

JMX commands for reference:

-Dcom.sun.management.jmxremote \
-Dcom.sun.management.jmxremote.local.only=false \
-Dcom.sun.management.jmxremote.authenticate=false \
-Dcom.sun.management.jmxremote.ssl=false \
-Djava.rmi.server.hostname=172.17.0.2 \
-Dcom.sun.management.jmxremote.port=9090\
-Dcom.sun.management.jmxremote.rmi.port=9090\
-jar java-api.jar server

Ocker

Which JVM version are you using? If it is 8 then it seems to be impossible to get the JMX server running because of an issue with resolving localhost. I've succesfully setup a remote JMX connection in a Fargate container using JVM 11. Make sure the port is exposed in the task definition and that the security groups allow inbound connection on this port. Also in order to connect your container needs to have a public IP and you need to use the public IP to connect to the container (it can be found in the task detail of the container). – Kowalewski

I am using Java 11 and the JMX ports are exposed accordingly. However the container only has a private IP due to security reasons, I guess that would be the reason why it is failing. – Ocker

Answer 1

Following changes worked for me in connecting Visual VM to Spring Boot Application deployed in AWS Fargate (private VPC)

• jvm parameters

-Dcom.sun.management.jmxremote \
-Dcom.sun.management.jmxremote.port=1099 \
-Dcom.sun.management.jmxremote.rmi.port=1099 \
-Dcom.sun.management.jmxremote.ssl=false \
-Dcom.sun.management.jmxremote.authenticate=false \
-Dcom.sun.management.jmxremote.local.only=false \
-Djava.rmi.server.hostname=127.0.0.1

• export port 1099 (both in dockerfile and cloudformation template - PortMappings->ContainerPort)

• Container security group to accept incoming traffic on 1099 (tcp and udp) from one of the existing EC2 in vpc (jump server)

• ssh port forwarding by using EC2 (jump server) to task running in fargate (use private ip of task running in fargate)

run following command on local

ssh -l <user> -L 127.0.0.1:1099:<task-private-ip-in-fargate>:1099 <ec2-ip(jump server)>

• Connect VisualVM using JMX Connection on 127.0.0.1:1099

Calorifacient