jvisualvm connect to remote jstatd not showing applications

Question

I started a jstatd on the remote server (Ubuntu Server 14.04):

jstatd -J-Djava.security.policy=.jstatd.all.policy -J-Djava.rmi.server.logCalltrue -p 9099

and try to connect to it with jvisualvm on windows. I checked netstat, the connection is established, and on the remote it logs the call:

Sep 11, 2015 12:48:51 PM sun.rmi.server.UnicastServerRef logCall
FINER: RMI TCP Connection(4)-10.82.199.0: [10.82.199.0: sun.rmi.registry.RegistryImpl[0:0:0, 0]: java.rmi.Remote lookup(java.lang.String)]
Sep 11, 2015 12:48:55 PM sun.rmi.server.UnicastServerRef logCall
FINER: RMI TCP Connection(4)-10.82.199.0: [10.82.199.0: sun.rmi.registry.RegistryImpl[0:0:0, 0]: java.rmi.Remote lookup(java.lang.String)]
Sep 11, 2015 12:48:59 PM sun.rmi.server.UnicastServerRef logCall
FINER: RMI TCP Connection(4)-10.82.199.0: [10.82.199.0: sun.rmi.registry.RegistryImpl[0:0:0, 0]: java.rmi.Remote lookup(java.lang.String)]

All signs are saying that it's working. but however no applications is showing in jvisualvm:

Quanta

Answer 1

Apparently VisualVM expects a consistent DNS name for the server you're trying to connect to remotely (the Ubuntu Server 14.04 in your case). Hence, if you're specifying an IP address instead of a DNS name to VisualVM you should add the following to your jstatd startup line:

-J-Djava.rmi.server.hostname=<the IP address to your Ubuntu server here>

Additionally, I found out that specifying the port option (-p 9099 in your case) is not supported in some VisualVM releases:

Known limitation: In this VisualVM release the jstatd's default port and rminame must be used when starting the jstatd utility, i.e. the use of the -p and -n options is not supported. VisualVM Troubleshooting Guide

All in all, you should try running the following jstatd line on your Ubuntu Server:

jstatd -J-Djava.security.policy=.jstatd.all.policy -J-Djava.rmi.server.hostname=10.82.83.117 -J-Djava.rmi.server.logCalltrue

Sources:

1. http://www.catify.com/2012/09/26/remote-monitoring-with-visualvm/
2. It worked for me :)

Sandrocottus

adding the IP address as the hostname option fixed the issue for me. Thanks for the answer. – Exhilarative

Just in case that someone is having problems with the command above, the parameter "-J-Djava.rmi.server.logCalltrue" should be "-J-Djava.rmi.server.logCalls=true". – Exalt

Answer 2

jstatd -p 1099 -J-Djava.rmi.sver.hostname=10.250.105.112 -J-Djava.security.policy=<(echo 'grant codebase "file:${java.home}/../lib/tools.jar" {permission java.security.AllPermission;};')      

Works for Me Perfectly

Delvalle

In JDK 11 tools.jar is replaced with modules. You'll want to explore jrt:/jdk.jstatd. – Advection

Should java.rmi.sver.hostname be java.rmi.server.hostname ? – Serving

Answer 3

In case this helps someone else...

I was running into problems where neither jstatd nor adding a plain JMX connection in VisualVM worked. The former would not give any error messages, it just wouldn't list any apps. The latter would give me an error saying "Cannot connect to some-server:30648 using service:jmx:rmi:///jndi/rmi://some-server:30648/jmxrmi.

Trying to use the excellent sjk-plus tool to manually connect to the JMX service gave the following error:

$ java --add-opens java.base/jdk.internal.perf=ALL-UNNAMED \
       --add-opens jdk.attach/sun.tools.attach=ALL-UNNAMED \
       -Dsjk.breakCage=false \
       -jar scripts/sjk-plus-0.14.jar 
       mx --get --allMatched -b com.acme.some.package:name=* -f Count \
      -s some-server:30648
JMX Connection failed: java.rmi.ConnectException: Connection refused to host: 127.0.1.1; nested exception is: 
    java.net.ConnectException: Connection refused (Connection refused)

Do you see it? 127.0.1.1, what is that weird IP address doing there?

This was caused by a particular entry in the /etc/hosts file on the server:

user@some-server:~$ cat /etc/hosts
127.0.0.1   localhost
127.0.1.1   some-server

# The following lines are desirable for IPv6 capable hosts
::1     localhost ip6-localhost ip6-loopback
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters

Changing the some-server entry in the hosts file and restarting the process made it work with sjk-plus, and made it discoverable with jstatd as well.

Behr

Answer 4

in java 11+

jstatd -J-Djava.rmi.server.logCalls=true \
    -J-Djava.security.policy=.jstatd.all.policy \
    -J-Djava.net.preferIPv4Stack=true \
    -J-Djava.security.policy=<(echo 'grant codebase "jrt:/jdk.jstatd" {permission java.security.AllPermission;}; grant codebase "jrt:/jdk.internal.jvmstat" {permission java.security.AllPermission;};')

Chism