Profiling tomcat application with VisualVM

Question

I am trying to run visualvm under the username tomcat6 because apparently visualvm can only find applications running under its username. So by default it is only finding applications running under my username. I have been able to connect visualvm with tomcat6 through jmx but that lacks the fine granularity of instrumented profiling.

I tried the following to run visualvm under the username tomcat6 but got the following error that I don't understand.

$ sudo -u tomcat6 jvisualvm
No protocol specified
Exception in thread "main" java.awt.AWTError: Can't connect to X11 window server using ':0' as the value of the DISPLAY variable.
    at sun.awt.X11GraphicsEnvironment.initDisplay(Native Method)
    at sun.awt.X11GraphicsEnvironment.access$200(X11GraphicsEnvironment.java:65)
    at sun.awt.X11GraphicsEnvironment$1.run(X11GraphicsEnvironment.java:115)
...

Poky

#24153916 – Actinology

Answer 1

Unfortunately only sampling is available in remote mode so JMX will lack instrumentation tools.

Actually your approach to running visualvm under tomcat6 user is correct. You should take a look at this question on how to run X11 applications under sudo.

The easiest way to pass DISPLAY and XAUTHORITY environment variables is to use sudo -E command to preserve current user environment.

Also if you can't see your process under tomcat6 user you should check if CATALINA_TMPDIR is pointing to /tmp. Otherwise you should pass it to visualvm

jvisualvm -J-Djava.io.tmpdir="${CATALINA_TMPDIR}"

---

Actually there is a lot of alternatives like yourkit or jprofiler shipped with java agents which allows remote instrumentation profiling.

Flavorous

echo ${DISPLAY} return :0 as my account and gui programs of course work fine logged in as me. So what does this mean in terms of trying to open a gui program as the tomcat6 user. – Poky

@Poky could you please try to launch it with sudo -E ? also you may try to pass an XAUTHORITY environment variable. I'll extend my answer. – Flavorous

I just tried sudo -E -u tomcat6 jvisualvm and things work after having to chmod a bunch of stuff to keep various files readable. – Poky

Answer 2

If the computer running your application is remote -- like a server -- then you can't run GUI applications without some work. It's probably going to be easier to enable remote access to VisualVM.

You can use two techniques to attach to a remote JVM: using jstatd or using JMX. I'm not sure what you think you are losing by using JMX, but evidently jstatd doesn't give you access to profiling tools, CPU monitor, etc.).

You need to configure your JVM and Tomcat to allow for remote access. That requires 3 steps:

1. Enable remote JMX. Turns out, there's a guide for that.

2. Fix the "wandering port" used for RMI. There's a guide for that, too.

3. (Optional) Arrange for secure remote-access to the server. The easiest way to do that would be to use ssh -Lport:localhost:port with a series of -L arguments to forward multiple ports from your workstation to your server. Map all the ports you had to configure in steps #1 and #2. If you don't do this, you'll need to have non-firewalled access to all the aforementioned ports.

4. Restart your JVM and connect with JVisualVM.

Update 2022-06-01

Note that the "wandering port" problem has been fixed at the JVM level, so there is no need for application (i.e. Tomcat) support for that. Item #1 for Tomcat 8.5 and later contains updated instructions making item #2 unnecessary with a recent JVM.

Jackinthebox

Finally a working solution! This should be the accepted answer. – Obstruct

Answer 3

The easiest is to open a remote JXM port on Tomcat in order to be able to remotely (from your desktop computer) connect to your remote Tomcat (on your server) with jvisualvm.

You need to pass the following system properties to your JVM :

-Dcom.sun.management.jmxremote 
-Dcom.sun.management.jmxremote.port=<whatever_port_you_want>
-Dcom.sun.management.jmxremote.ssl=false

Then open jvisualvm on your local computer (JVM version needs to be the same or newer), File -> Add Remote Host -> Enter the name on the Host. It will create an entry for this host. Right lick on this Entry -> Add JMX connection -> Enter the port -> OK

jvisualvm will then be able to access remotely to your application.

You can also secure the connection if needed by using the following system properties (you need to create the files and locate them where you want :

-Dcom.sun.management.jmxremote.password.file=jmxremote.password 
-Dcom.sun.management.jmxremote.access.file=jmxremote.access 

These properties needs to be added to the CATALINA_OPTS environment variable. Fr exemple :

export CATALINA_OPTS = "$CATALINA_OPTS -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=8888 "

More info on JMX lies here : https://docs.oracle.com/javase/8/docs/technotes/guides/management/agent.html

Sarsenet

Answer 4

I tried to do something similar but I was not allowed to install JVisualVM on the server. Having JVisualVM connect to the remote machine never seemed to work correctly. I suspect firewall rules were blocking part of the the network connections. The only way I found to remotely profile the server was via an ssh tunnel.

1. Set the JMX port in CATALINA_OPTS on the server

   CATALINA_OPTS="$CATALINA_OPTS -Dcom.sun.management.jmxremote.port=13333 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false; export CATALINA_OPTS

2. On your desktop machine open an ssh connection to the server

   ssh -D 61444 [email protected]

3. Add a flag to JVisualVM so that it will proxy its network connection

   "C:\Program Files\Java\jdk1.7.0_79\bin\jvisualvm.exe" -J-Dnetbeans.system_socks_proxy=localhost:61444 -J-Djava.net.useSystemProxies=true

4. Have JVisualVM connect to the jmxport and the network traffic is tunneled via ssh.

   Good luck.

Meghan