Unable to view Visual GC in VisualVM

Question

I am using Java VisualVM on my local windows machine to monitor my remote server(Linux). I am able to get all stats in Bisual VM except "Visual GC". It shows "Not Supported for this JVM". I have googled it and find that it might be because jstatd is not running and connected. So executed "jstatd -J-Djava.security.policy=/tmp/tools.policy". It shows a ip "175.41.139.225". I can't understand why it is trying to connect to this ip. It is not the ip of my remote machine nor of my local machine.

[root@shipping_pt mail]# jstatd -J-Djava.security.policy=/tmp/tools.policy
Could not contact registry
Connection refused to host: 175.41.139.225; nested exception is:
        java.net.ConnectException: Connection refused
java.rmi.ConnectException: Connection refused to host: 175.41.139.225; nested exception is:
        java.net.ConnectException: Connection refused
        at sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:619)
        at sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:216)
        at sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:202)
        at sun.rmi.server.UnicastRef.newCall(UnicastRef.java:340)
        at sun.rmi.registry.RegistryImpl_Stub.rebind(Unknown Source)
        at java.rmi.Naming.rebind(Naming.java:177)
        at sun.tools.jstatd.Jstatd.bind(Jstatd.java:57)
        at sun.tools.jstatd.Jstatd.bind(Jstatd.java:66)
        at sun.tools.jstatd.Jstatd.main(Jstatd.java:143)
Caused by: java.net.ConnectException: Connection refused
        at java.net.PlainSocketImpl.socketConnect(Native Method)
        at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:327)
        at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:193)
        at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:180)
        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:384)
        at java.net.Socket.connect(Socket.java:546)
        at java.net.Socket.connect(Socket.java:495)
        at java.net.Socket.<init>(Socket.java:392)
        at java.net.Socket.<init>(Socket.java:206)
        at sun.rmi.transport.proxy.RMIDirectSocketFactory.createSocket(RMIDirectSocketFactory.java:40)
        at sun.rmi.transport.proxy.RMIMasterSocketFactory.createSocket(RMIMasterSocketFactory.java:146)
        at sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:613)
        ... 8 more

Synchrotron

Answer 1

On remote server,

vim /tmp/tools.policy

Add,

grant codebase "file:${java.home}/../lib/tools.jar" {
   permission java.security.AllPermission;
};

And start jstatsd

jstatd -J-Djava.security.policy=/tmp/tools.policy -J-Djava.rmi.server.hostname=<public_ip>

and you ready are with Visual GC inside VisualVM on local machine.

Cookie

Answer 2

You are right that you need to run jstatd to be able to use VisualGC plugin, because VisualGC uses jvmstat API. It looks like you are running jstatd as root. Is you monitored application running as 'root'? How does /tmp/tools.policy file looks like? Try to run jstatd with the following args:

jstatd -J-Djava.rmi.server.hostname=<IP address of shipping_pt machine> -J-Djava.security.policy=/tmp/tools.policy

Carmelacarmelia

Hi Tomas, my application runs from a user 'tomcat'. And tools.policy is having "grant codebase "file:${java.home}/../lib/tools.jar" { permission java.security.AllPermission; };" – Synchrotron

After executing jstatd -J-Djava.rmi.server.hostname=<IP address of shipping_pt> -J-Djava.security.policy=/tmp/tools.policy I am still getting the same exception as above. Just the IP is different and it is nor the ip which I have specified neither of any of my machine – Synchrotron

Every time I execute the same above specified command it shows same exception but with different random IP's Connection refused to host: <some random IP>; nested exception is: – Synchrotron

Can you try to run jstatd as tomcat user? It is strange that -J-Djava.rmi.server.hostname= did not help you. Maybe this thread tinyurl.com/c3q4sj3 will help you. In general the problem seems to be caused by the fact that jstatd cannot figure out IP address of your machine. – Carmelacarmelia

permissions!! yes, that was my problem: I was running eclipse as administrator, but not jvisualvm. Thanks, Tomas. – Security

Answer 3

VsiualVM is very version dependant. If you can connect but some functionality is not available, I would make sure you are running the exactly the same version of the Java.

If you can't connect you need to change the permissions of your JVM. Note: you can only see your processes by default.

Vaunt

VisualVM is definitely not 'very version dependant'. It can monitor JVM from 1.4.2 up to JDK 8 and you can still run VisualVM on the same JDK. Moreover the error above has nothing to do with VisualVM - jstatd does not contain any VisualVM code - VisualVM just uses jvmstat API, which is park of JRE. jstatd is a daemon, which allows jvmstat to work with remote hosts. – Carmelacarmelia

@TomasHurka You can run it, but some version of VisualVM don't have all the features enabled in you have a different JDK. Perhaps that has been fixed in the latest versions. I take you point about jstat – Vaunt

@Peter - Thanks for suggestion but I am getting above exception while executing jstat on my remote machine. So I think it is definitely nothing to do with the Visual VM version. – Synchrotron

@Peter It would be useful if you can provide some example. Note that features are available if monitored JVM supports it. There is also difference in feature set for local/remote monitoring. VisualVM uses several technologies (jvmstat, JMX, Attach API, SA) to access monitored JVM). Some of those are not available in certain versions of JVM or in remote case. For example JMX is available in JDK 5, some features relying on JMX are not available when monitoring older JDK. Similarly Attach API works only for local applications and therefor feature relying on it does not work in remote case. – Carmelacarmelia