Has anyone ever got a remote JMX JConsole to work?
Asked Answered
C

20

130

It seems that I've never got this to work in the past. Currently, I KNOW it doesn't work.

But we start up our Java process:

-Dcom.sun.management.jmxremote
-Dcom.sun.management.jmxremote.port=6002
-Dcom.sun.management.jmxremote.authenticate=false
-Dcom.sun.management.jmxremote.ssl=false

I can telnet to the port, and "something is there" (that is, if I don't start the process, nothing answers, but if I do, it does), but I can not get JConsole to work filling in the IP and port.

Seems like it should be so simple, but no errors, no noise, no nothing. Just doesn't work.

Anyone know the hot tip for this?

Claude answered 29/9, 2008 at 23:54 Comment(2)
If youre using tomcat this may be the solution : #1264491Roderic
Did you forget to accept something here @Will?Spoilsman
D
131

I have a solution for this:

If your Java process is running on Linux behind a firewall and you want to start JConsole / Java VisualVM / Java Mission Control on Windows on your local machine to connect it to the JMX Port of your Java process.

You need access to your linux machine via SSH login. All Communication will be tunneled over the SSH connection.

TIP: This Solution works no matter if there is a firewall or not.

Disadvantage: Everytime you restart your java process, you will need to do all steps from 4 - 9 again.


1. You need the putty-suite for your Windows machine from here:

http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html

At least the putty.exe


2. Define one free Port on your linux machine:

<jmx-remote-port>

Example:

jmx-remote-port = 15666      


3. Add arguments to java process on the linux machine

This must be done exactly like this. If its done like below, it works for linux Machines behind firewalls (It works cause of the -Djava.rmi.server.hostname=localhost argument).

-Dcom.sun.management.jmxremote
-Dcom.sun.management.jmxremote.port=<jmx-remote-port>
-Dcom.sun.management.jmxremote.ssl=false
-Dcom.sun.management.jmxremote.authenticate=false
-Dcom.sun.management.jmxremote.local.only=false
-Djava.rmi.server.hostname=localhost

Example:

java -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=15666 -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.local.only=false -Djava.rmi.server.hostname=localhost ch.sushicutta.jmxremote.Main


4. Get Process-Id of your Java Process

ps -ef | grep <java-processname>

result ---> <process-id>

Example:

ps -ef | grep ch.sushicutta.jmxremote.Main

result ---> 24321


5. Find arbitrary Port for RMIServer stubs download

The java process opens a new TCP Port on the linux machine, where the RMI Server-Stubs will be available for download. This port also needs to be available via SSH Tunnel to get a connection to the Java Virtual Machine.

With netstat -lp this port can be found also the lsof -i gives hints what port has been opened form the java process.

NOTE: This port always changes when java process is started.

netstat -lp | grep <process-id>

tcp        0      0 *:<jmx-remote-port>     *:*     LISTEN      24321/java
tcp        0      0 *:<rmi-server-port>     *:*     LISTEN      24321/java


result ---> <rmi-server-port>

Example:

netstat -lp | grep 24321

tcp        0      0 *:15666     *:*     LISTEN      24321/java
tcp        0      0 *:37123     *:*     LISTEN      24321/java


result ---> 37123


6. Enable two SSH-Tunnels from your Windows machine with putty

Source port: <jmx-remote-port>
Destination: localhost:<jmx-remote-port>
[x] Local       
[x] Auto       

Source port: <rmi-server-port>
Destination: localhost:<rmi-server-port>
[x] Local       
[x] Auto

Example:

Source port: 15666
Destination: localhost:15666
[x] Local       
[x] Auto       

Source port: 37123
Destination: localhost:37123
[x] Local       
[x] Auto


Settings to open an SSL tunnel via Putty


7. Login to your Linux machine with Putty with this SSH-Tunnel enabled.

Leave the putty session open.

When you are logged in, Putty will tunnel all TCP-Connections to the linux machine over the SSH port 22.

JMX-Port:

Windows machine: localhost:15666   >>> SSH >>>   linux machine: localhost:15666

RMIServer-Stub-Port:

Windows Machine: localhost:37123   >>> SSH >>>   linux machine: localhost:37123


8. Start JConsole / Java VisualVM / Java Mission Control to connect to your Java Process using the following URL

This works, cause JConsole / Java VisualVM / Java Mission Control thinks you connect to a Port on your local Windows machine. but Putty send all payload to the port 15666 to your linux machine.

On the linux machine first the java process gives answer and send back the RMIServer Port. In this example 37123.

Then JConsole / Java VisualVM / Java Mission Control thinks it connects to localhost:37123 and putty will send the whole payload forward to the linux machine

The java Process answers and the connection is open.

[x] Remote Process:
service:jmx:rmi:///jndi/rmi://localhost:<jndi-remote-port>/jmxrmi

Example:

[x] Remote Process:
service:jmx:rmi:///jndi/rmi://localhost:15666/jmxrmi


Connect via jmx service url


9. ENJOY #8-]

Dowzall answered 3/7, 2013 at 20:19 Comment(10)
Just a little question here - not possible to do a JMX connection without rmi?Tessatessellate
I got a hint, that we can set a rmi.port to a fixed port number, so we can set the arbitrary port for RMIServer stubs download. this should work with the Java property "com.sun.management.jmxremote.rmi.port=<rmi-server-port>". It looks like an undocumented feature in the Oracle Java VM.Dowzall
Sure beats having to setup keystores and trustedstoresSwifter
Same process, but I got no such object in tableHetero
What if it's a production server and customer doesn't want to share user credentials on remote host. This is required in step 7Doorkeeper
It worked for 1st time but second time its showing 'cannot connect to service' . redid entire oricess, still no help. Any clue ?Anlage
@Dowzall can you add this hint in your answer, its working perfectly fine, and can remove the steps from 4 to 6, the catch is that your forwarded port has to be the same as original port and both jmx and rmi port also have to be sameNonviolence
On weblogic10.3.4 (maybe other versions as well), you MUST use the weblogic console's username/password for authentication EVEN IF you've added the java start argument: -Dcom.sun.management.jmxremote.authenticate=false (!!!)Horning
By setting the same rmi port as the choosen <jndi-remote-port>, my steps 4 & 5 can be skipped, see the answers down below. In the step 6 only one port forward to the port <jndi-remote-port> (f.ex. 15666) needs to be done. f.ex. -Dcom.sun.management.jmxremote.rmi.port=<jndi-remote-port> -Dcom.sun.management.jmxremote.rmi.port=15666Dowzall
Worked but only when this property is also added -Dcom.sun.management.jmxremote.rmi.port=5002Phenobarbital
D
80

Adding -Djava.rmi.server.hostname='<host ip>' resolved this problem for me.

Dukedom answered 22/5, 2009 at 21:8 Comment(6)
In my case I have to add ip address (-Djava.rmi.server.hostname=<ip>). hostname -i gave me two ip addresses and the correct one was second in the list.Slit
didn't resolve the issue for me. connecting windows-2-windows is not a problem for me BUT when I try to connect from a JVM Jvisualvm.exe on Windows to monitor a java service running on SUSE with Oracle JDK 1.6.024 , it fails the connection. For this reason I think this persons question still stands unanswered.Diversification
This solved the issue for me. This plus the usual 3 (authenticate/port/ssl) set and i can remotely connect now. The box is listening on multiple virtual interfaces though, may have been why not specifying the host confused the jvm.Velocipede
Finally solved my problems connecting jconsole on my osx laptop. Thanks.Borneol
Wtf? What if the server has dynamic IP?? This makes no sense!! Who the hell invented this??Thornton
I also had to add -Dcom.sun.management.jmxremote.rmi.port=6002 (see also the answer below: #151738)Toh
S
65

Tried with Java 8 and newer versions

This solution works well also with firewalls

1. Add this to your java startup script on remote-host:

-Dcom.sun.management.jmxremote.port=1616
-Dcom.sun.management.jmxremote.rmi.port=1616
-Dcom.sun.management.jmxremote.ssl=false
-Dcom.sun.management.jmxremote.authenticate=false
-Dcom.sun.management.jmxremote.local.only=false
-Djava.rmi.server.hostname=localhost

2. Execute this on your computer.

  • Windows users:

    putty.exe -ssh user@remote-host -L 1616:remote-host:1616

  • Linux and Mac Users:

    ssh user@remote-host -L 1616:remote-host:1616

3. Start jconsole on your computer

jconsole localhost:1616

4. Have fun!

P.S.: during step 2, using ssh and -L you specify that the port 1616 on the local (client) host must be forwarded to the remote side. This is an ssh tunnel and helps to avoids firewalls or various networks problems.

Stolen answered 24/11, 2015 at 11:47 Comment(6)
Really sucks that Oracle does not mention "com.sun.management.jmxremote.rmi.port", "java.rmi.server.hostname" docs.oracle.com/javase/8/docs/technotes/guides/management/… I guess that was my issue.Zonked
Because, AFAIK, this problem is not about JMX, but how RMI works. For example, after this case, I had the same problem with jmeter, which use rmi in its client/server implementation.Stolen
It works. Just adding my experience with the tunnels: 1) can use "localhost" in "-L 1616:localhost:1616" 2) cannot change the source port, i.e. this won't not work: "-L 9999:localhost:1616"Conduct
Is there a reason for choosing JConsole over Java VisualVM / Java Mission Control ?Demetria
you should add that putty.exe -ssh user@remote-host -L 1616:remote-host:1616 will open a putty window asking for your password and then nothing will happen into it but it should remain open :)Obliterate
I have no idea why but the port 1616 works in my environment while 9998, 9123, 1975 do not. Though I use SSH forward proxy.Icarian
B
21

After putting my Google-fu to the test for the last couple of days, I was finally able to get this to work after compiling answers from Stack Overflow and this page http://help.boomi.com/atomsphere/GUID-F787998C-53C8-4662-AA06-8B1D32F9D55B.html.

Reposting from the Dell Boomi page:

To Enable Remote JMX on an Atom

If you want to monitor the status of an Atom, you need to turn on Remote JMX (Java Management Extensions) for the Atom.

Use a text editor to open the <atom_installation_directory>\bin\atom.vmoptions file.

Add the following lines to the file:

-Dcom.sun.management.jmxremote.port=5002
-Dcom.sun.management.jmxremote.rmi.port=5002
-Dcom.sun.management.jmxremote.authenticate=false
-Dcom.sun.management.jmxremote.ssl=false

The one line that I haven't seen any Stack Overflow answer cover is

-Dcom.sun.management.jmxremote.rmi.port=5002

In my case, I was attempting to retrieve Kafka metrics, so I simply changed the above option to match the -Dcom.sun.management.jmxremote.port value. So, without authentication of any kind, the bare minimum config should look like this:

-Dcom.sun.management.jmxremote
-Dcom.sun.management.jmxremote.authenticate=false
-Dcom.sun.management.jmxremote.ssl=false
-Dcom.sun.management.jmxremote.port=(jmx remote port)

-Dcom.sun.management.jmxremote.local.only=false
-Dcom.sun.management.jmxremote.rmi.port=(jmx remote port)
-Djava.rmi.server.hostname=(CNAME|IP Address)
Bula answered 16/12, 2014 at 19:10 Comment(3)
Plus one for "Google-fu"Fritts
"com.sun.management.jmxremote.rmi.port" was the key for me as well. See also this answer: https://mcmap.net/q/175254/-spring-cannot-connect-to-a-jmx-server-using-rmi-from-behind-a-firewallSimonton
I did not need "com.sun.management.jmxremote.local.only" so I don't think your configuration is truly "bare minimum"Simonton
G
19

You are probably experiencing an issue with a firewall. The 'problem' is that the port you specify is not the only port used, it uses 1 or maybe even 2 more ports for RMI, and those are probably blocked by a firewall.

One of the extra ports will not be know up front if you use the default RMI configuration, so you have to open up a big range of ports - which might not amuse the server administrator.

There is a solution that does not require opening up a lot of ports however, I've gotten it to work using the combined source snippets and tips from

http://forums.sun.com/thread.jspa?threadID=5267091 - link doesn't work anymore

http://blogs.oracle.com/jmxetc/entry/connecting_through_firewall_using_jmx - link doesn't work anymore, see Wayback Machine

http://java.sun.com/javase/6/docs/technotes/guides/management/agent.html

It's even possible to setup an ssh tunnel and still get it to work :-)

Gynaeceum answered 12/12, 2008 at 14:45 Comment(3)
I was able to work around firewall using only the alias described in simplygenius.com/2010/08/jconsole-via-socks-ssh-tunnel.html along with setting -Djava.rmi.server.hostname as mentionned in another answer here.Nicosia
Note to future readers: the link to forums.sun.com is brokenRacemose
Note to future readers: the link to blogs.oracle.com is broken.Lavoie
U
11

Are you running on Linux? Perhaps the management agent is binding to localhost:

http://java.sun.com/j2se/1.5.0/docs/guide/management/faq.html#linux1

Unstrained answered 30/9, 2008 at 3:13 Comment(0)
O
8

Sushicutta's steps 4-7 can be skipped by adding the following line to step 3:

-Dcom.sun.management.jmxremote.rmi.port=<same port as jmx-remote-port>

e.g. Add to start up parameters:

-Dcom.sun.management.jmxremote
-Dcom.sun.management.jmxremote.port=12345
-Dcom.sun.management.jmxremote.rmi.port=12345
-Dcom.sun.management.jmxremote.ssl=false
-Dcom.sun.management.jmxremote.authenticate=false
-Dcom.sun.management.jmxremote.local.only=false
-Djava.rmi.server.hostname=localhost

For the port forwarding, connect using:

ssh -L 12345:localhost:12345 <username>@<host>

if your host is a stepping stone, simply chain the port forward by running the following on the step stone after the above:

ssh -L 12345:localhost:12345 <username>@<host2>

Mind that the hostname=localhost is needed to make sure the jmxremote is telling the rmi connection to use the tunnel. Otherwise it might try to connect directy and hit the firewall.

Ore answered 8/4, 2016 at 12:18 Comment(1)
This method helps me: (1) I add missed JMX parameters and restart app (2) Then run ssh -L <JMX_port>:localhost:<JMX_port> <remote_user>@<remote_host> on local machine (3) Then I connect to remote JMX using: jconsole <remote_host>:<JMX_port>Hot
A
6

PROTIP:

The RMI port are opened at arbitrary portnr's. If you have a firewall and don't want to open ports 1024-65535 (or use vpn) then you need to do the following.

You need to fix (as in having a known number) the RMI Registry and JMX/RMI Server ports. You do this by putting a jar-file (catalina-jmx-remote.jar it's in the extra's) in the lib-dir and configuring a special listener under server:

<Listener className="org.apache.catalina.mbeans.JmxRemoteLifecycleListener"
      rmiRegistryPortPlatform="10001" rmiServerPortPlatform="10002" />

(And ofcourse the usual flags for activating JMX

    -Dcom.sun.management.jmxremote  \
    -Dcom.sun.management.jmxremote.ssl=false \
    -Dcom.sun.management.jmxremote.authenticate=false \
    -Djava.rmi.server.hostname=<HOSTNAME> \

See: JMX Remote Lifecycle Listener at http://tomcat.apache.org/tomcat-6.0-doc/config/listeners.html

Then you can connect using this horrific URL:

service:jmx:rmi://<hostname>:10002/jndi/rmi://<hostname>:10001/jmxrmi
Ative answered 11/10, 2013 at 9:48 Comment(1)
Tried the above w/ the extras jar, and can see the RMI ports listening as specified, but random ports still used by RMI after connecting to the JVM port with VisualVM. Workaround: watch for ports with 'lsof -i' and open those with blocked connections.Sculpt
P
5

Check if your server is behind the firewall. JMX is base on RMI, which open two port when it start. One is the register port, default is 1099, and can be specified by the com.sun.management.jmxremote.port option. The other is for data communication, and is random, which is what cause problem. A good news is that, from JDK6, this random port can be specified by the com.sun.management.jmxremote.rmi.port option.

export CATALINA_OPTS="-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=8991 -Dcom.sun.management.jmxremote.rmi.port=8991 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false"
Philous answered 4/6, 2015 at 6:37 Comment(0)
L
4

Getting JMX through the Firewall is really hard. The Problem is that standard RMI uses a second random assigned port (beside the RMI registry).

We have three solution that work, but every case needs a different one:

  1. JMX over SSH Tunnel with Socks proxy, uses standard RMI with SSH magic http://simplygenius.com/2010/08/jconsole-via-socks-ssh-tunnel.html

  2. JMX MP (alternative to standard RMI), uses only one fixed port, but needs a special jar on server and client http://meteatamel.wordpress.com/2012/02/13/jmx-rmi-vs-jmxmp/

  3. Start JMX Server form code, there it is possible to use standard RMI and use a fixed second port: https://issues.apache.org/bugzilla/show_bug.cgi?id=39055

Liger answered 17/6, 2013 at 7:34 Comment(1)
All other answers should be an addition to this oneFlowerlike
E
2

When testing/debugging/diagnosing remote JMX problems, first always try to connect on the same host that contains the MBeanServer (i.e. localhost), to rule out network and other non-JMX specific problems.

Extensometer answered 12/12, 2008 at 14:17 Comment(0)
O
2

There are already some great answers here, but, there is a slightly simpler approach that I think it is worth sharing.

sushicutta's approach is good, but is very manual as you have to get the RMI Port every time. Thankfully, we can work around that by using a SOCKS proxy rather than explicitly opening the port tunnels. The downside of this approach is JMX app you run on your machine needs to be able to be configured to use a Proxy. Most processes you can do this from adding java properties, but, some apps don't support this.

Steps:

  1. Add the JMX options to the startup script for your remote Java service:

    -Dcom.sun.management.jmxremote=true
    -Dcom.sun.management.jmxremote.port=8090
    -Dcom.sun.management.jmxremote.ssl=false
    -Dcom.sun.management.jmxremote.authenticate=false
    
  2. Set up a SOCKS proxy connection to your remote machine:

    ssh -D 9696 [email protected]
    
  3. Configure your local Java monitoring app to use the SOCKS proxy (localhost:9696). Note: You can sometimes do this from the command line, i.e.:

    jconsole -J-DsocksProxyHost=localhost -J-DsocksProxyPort=9696
    
Overshadow answered 27/1, 2015 at 9:31 Comment(0)
B
2

The following worked for me (though I think port 2101 did not really contribute to this):

-Dcom.sun.management.jmxremote.port=2100
-Dcom.sun.management.jmxremote.authenticate=false
-Dcom.sun.management.jmxremote.ssl=false
-Dcom.sun.management.jmxremote.local.only=false
-Dcom.sun.management.jmxremote.rmi.port=2101
-Djava.rmi.server.hostname=<IP_ADDRESS>OR<HOSTNAME>

I am connecting from a remote machine to a server which has Docker running and the process is inside the container. Also, I stopped firewallD but I don't think that was the issue as I could telnet to 2100 even with the firewall open. Hope it helps.

Bigley answered 10/10, 2018 at 7:56 Comment(0)
T
1

I am running JConsole/JVisualVm on windows hooking to tomcat running Linux Redhat ES3.

Disabling packet filtering using the following command did the trick for me:

/usr/sbin/iptables -I INPUT -s jconsole-host -p tcp --destination-port jmxremote-port -j ACCEPT

where jconsole-host is either the hostname or the host address on which JConsole runs on and jmxremote-port is the port number set for com.sun.management.jmxremote.port for remote management.

Towny answered 15/7, 2010 at 13:57 Comment(1)
didn't work for me on a SUSE Amazon EC2 instance. i think the problem lies elswhere.Diversification
E
1

I'm using boot2docker to run docker containers with Tomcat inside and I've got the same problem, the solution was to:

  • Add -Djava.rmi.server.hostname=192.168.59.103
  • Use the same JMX port in host and docker container, for instance: docker run ... -p 9999:9999 .... Using different ports does not work.
Endplay answered 29/5, 2015 at 8:11 Comment(0)
B
0

You need to also make sure that your machine name resolves to the IP that JMX is binding to; NOT localhost nor 127.0.0.1. For me, it has helped to put an entry into hosts that explicitly defines this.

Brachylogy answered 28/3, 2013 at 15:46 Comment(0)
F
0

Getting JMX through the firewall isn't that hard at all. There is one small catch. You have to forward both your JMX configured port ie. 9010 and one of dynamic ports its listens to on my machine it was > 30000

Faucal answered 26/8, 2013 at 23:44 Comment(0)
K
0

These are the steps that worked for me (debian behind firewall on the server side, reached over VPN from my local Mac):

check server ip

hostname -i

use JVM params:

-Dcom.sun.management.jmxremote
-Dcom.sun.management.jmxremote.port=[jmx port]
-Dcom.sun.management.jmxremote.local.only=false
-Dcom.sun.management.jmxremote.authenticate=false
-Dcom.sun.management.jmxremote.ssl=false
-Djava.rmi.server.hostname=[server ip from step 1]

run application

find pid of the running java process

check all ports used by JMX/RMI

netstat -lp | grep [pid from step 4]

open all ports from step 5 on the firewall

Voila.

Kaikaia answered 10/2, 2015 at 20:22 Comment(0)
W
0

In order to make a contribution, this is what I did on CentOS 6.4 for Tomcat 6.

  1. Shutdown iptables service

    service iptables stop
    
  2. Add the following line to tomcat6.conf

    CATALINA_OPTS="${CATALINA_OPTS} -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=8085 -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false -Djava.rmi.server.hostname=[host_ip]"
    

This way I was able to connect from another PC using JConsole.

Whodunit answered 23/9, 2015 at 20:22 Comment(0)
M
0

I'm trying to JMC to run the Flight Recorder (JFR) to profile NiFi on a remote server that doesn't offer a graphical environment on which to run JMC.

Based on the other answers given here, and upon much trial and error, here is what I'm supplying to the JVM (conf/bootstrap.conf)when I launch NiFi:

java.arg.90=-Dcom.sun.management.jmxremote=true
java.arg.91=-Dcom.sun.management.jmxremote.port=9098
java.arg.92=-Dcom.sun.management.jmxremote.rmi.port=9098
java.arg.93=-Dcom.sun.management.jmxremote.authenticate=false
java.arg.94=-Dcom.sun.management.jmxremote.ssl=false
java.arg.95=-Dcom.sun.management.jmxremote.local.only=false
java.arg.96=-Djava.rmi.server.hostname=10.10.10.92  (the IP address of my server running NiFi)

I did put this in /etc/hosts, though I doubt it's needed:

10.10.10.92   localhost

Then, upon launching JMC, I create a remote connection with these properties:

Host: 10.10.10.92
Port: 9098
User: (nothing)
Password: (ibid)

Incidentally, if I click the Custom JMX service URL, I see:

service:jmx:rmi:///jndi/rmi://10.10.10.92:9098/jmxrmi

This finally did it for me.

Mandal answered 22/12, 2016 at 0:21 Comment(0)

© 2022 - 2024 — McMap. All rights reserved.