How to find out number of files currently open by Java application?

Question

Suppose a lot of what your application does deals with reading contents of files. It goes without saying that files that are opened then closed and life is good unless ... new files come in faster then old files get closed. This is the pickle of a situation I found myself in.

Now, is there a way to reliably know how many files are opened by the process? Something that is as reliable as looking at ls /proc/my_pid/fd | wc -l from inside the JVM?

I suspect the answer may be OS-specific, so let me add that I am running Java on Linux.

Zahara

Can't you just increment a count whenever you open a file...and decrement it whenever you close one? – Acerbate

I suppose that's one way of doing it .. Is there a cleaner solution? – Zahara

You could execute that command in java. Have a look: docs.oracle.com/javase/6/docs/api/java/lang/Runtime.html I cannot imagine that java memorizes which and how many files you have opened. – Bettinabettine

I guess the simplest way is just .... to check how many entries can be found in /proc/self/fd/. That's what the proc filesystem is for: Get (and set) process information at runtime. hierarchical tree of values with node-dependent read/write semantics + mapping of the same into the hierarchical tree of streams that is a filesystem = unified win (well, in a typeless Plan 9 kind of way - even better: open a JDBC conn to "somewhere" then query/update sys & proc using SQL, a bit like OSQuery but far more complete) – Fermin

Answer 1

On Unix, one way is using the ManagementFactory to get the OperatingSystemMxBean and if it is a UnixOperatingSystemMXBean, you can use the getOpenFileDescriptorCount() method.

Example code:

import java.lang.management.ManagementFactory;
import java.lang.management.OperatingSystemMXBean;
import com.sun.management.UnixOperatingSystemMXBean;

public class OpenFileCount{
    public static void main(String[] args){
        OperatingSystemMXBean os = ManagementFactory.getOperatingSystemMXBean();
        if(os instanceof UnixOperatingSystemMXBean){
            System.out.println("Number of open fd: " + ((UnixOperatingSystemMXBean) os).getOpenFileDescriptorCount());
        }
    }
}

Siddra

Does performing this task by itself increase number of FD? – Zahara

Does not seem to. Thanks. This is exactly what i was looking for – Zahara

@Siddra Great answer! Any idea on how to try this behavior in Windows? – Reverse

Answer 2

If you want to get the number of ls /proc/my_pid/fd | wc -l in java, you could use JMX.

When you have MBeanServerConnection, you can get attribute "OpenFileDescriptorCount". this will give you the same result as the above ls..|wc -l.

also there is another attribute "MaxFileDescriptorCount", which tells your the max count allowed.

If you just want to know the value, but not getting it in your code, you could either do your ls..|wc -l way, or read from jconsole (with GUI).

NOTE

ls /proc/my_pid/fd | wc -l

this number indicates how many fd opened by your process(java application). e.g. your business files are counted, also those jars, libraries files are counted too. If you just want to get the count of your business files, you have to implement a counter by yourself. Or, say you want to do it with shellscript, grep something out then wc -l

EDIT

add code example, but it is just an example. not written in IDE, not tested with compiler. :)

ObjectName oName = new ObjectName("java.lang:type=OperatingSystem");
MBeanServerConnection conn ; // you should get the connection following the api, take a look the java api/ google some example
javax.management.AttributeList list = conn.getAttributes(oName, new String[]{"OpenFileDescriptorCount", "MaxFileDescriptorCount"});
for(Attribute attr: list.asList()){
    System.out.println(attr.getName() + ": " + attr.getValue());
}

Roseberry

Answer 3

For the sake of completing/adding to the answer:

Many people uses LSOF in Linux based system to monitor file descriptors and their details. However LSOF lists all kind of FDs(CWD,MEM) which is not returned by UnixOperatingSystemMXBean.getOpenFileDescriptorCount().

Details can be found in http://www.ibm.com/developerworks/aix/library/au-lsof.html This may cause some confusion.

To clarify UnixOperatingSystemMXBean.getOpenFileDescriptorCount() only shows application opened files descriptors. But LSOF (lsof -a -p ) lists other file descriptors opened by kernel on behalf of the process.

To list only application level FDS one can filter out other type of FDs

lsof -a -p yourprocid -d ^txt,^mem,^cwd,^rtd,^DEL

Cassey

This answer has the benefit of working on macOS as well. – Svetlanasvoboda

Answer 4

Sorry for necroposting, but guys, you're ignoring -K option of lsof cause java have threads, each of them have their own FD that are living in /proc/$PID/tasks/fd/

and there is A HUGE difference:

root@xxx:/root# lsof -p 13553 | wc -l
612
root@xxx:/root# lsof -p 13553 -K | wc -l
112108

Crosson

Not sure if its my OS, but if you use -K argument it will report the files opened in all processes. So if I do: lsof -p <non existant PID> -K | wc -l outputs the same as lsof | wc -l. Only without -K seems to work as expected. – Wagonette

-K option gives a number of open descriptors for all process tasks. Useful when your process spawn a bunch of subprocesses live java applications do – Crosson

Answer 5

In ubuntu, the below lsof command works

sudo lsof -c java

Mirza

Answer 6

Better than counting the open files is listing them, and why they were opened (and never closed, or closed only by garbage collection). https://github.com/jenkinsci/lib-file-leak-detector can do this (in a non-production environment).

Augean