How can I follow a file like "Tail -f" does in Java without holding the file open (Prevent rename/delete)

Question

i'd like to "Tail -f" a lot of logfiles from within a java app.

I've gotten this to work by monitoring the size and last update and repeatedly opening the file and reading the last few bytes whenever the file size or last update time changes--then closing it right away.

This seems problematic because I might have it open when the logger decides to rename the file which will cause some kind of problem.

I'd also like to detect a "Rolled" file with a mechanism more sure than noticing that the file size reduced... seems error prone but unlikely.

Since I don't seem to have access to the file descriptor or other low-level file utilities I may not be able to reproduce tail's behavor--but are there any tricks to reading a file without "Locking" it for rename/delete (Windows 7)

I guess another possibility is to actually spawn a tail -f process and read the process output, but that seems somewhat heavy--I'm scanning like 60 logfiles here, some of which have a lot of output (Most will be idle).

Rufous

If you're using JDK 7, you can poll for file using java.nio.file.WatchService.poll(). – Cardinale

Really neat class! I think that will solve the problem of detecting truncates, but I still have the problem of having to open the file and read the text (although it may let me do that too... I'll have to look more closely). Thanks – Rufous

I lost you.Which OS?Windows or Linux? – Underside

Windows 7, Perhaps the fact that I use "Tail" from git threw you? – Rufous

Answer 1

Apache Commons has a Tailer class, would that do what you want? If it would, it has rolling detection mechanism as well as the reading stuff, so you'd get all you need.

If that cannot do it, it might be that there's no way of doing it with pure java. You'd need some help of, for example, C code, utilizing fopen with SH_DENYNO parameter allowing shared open on windows. Or then just call a tail implementation by executing a system command.

But since the code opening the log file is the one causing the lock on it, even that might not help. In that case the only real option is to change the way your logging works, as that is the culprit locking the file. Log4j can use a SocketAppender, among other things.

Samora

That Tailer class looks awesome. We aren't currently using that part of commons, but it could be worth pulling in! – Rufous

Answer 2

Use apache.commons.io

And here is a simple example

public class LogTailTest {

/**
* TailerListener implementation.
*/
static public class ShowLinesListener extends TailerListenerAdapter {
    @Override
    public void handle(String line) {
        System.out.println(line);
    }
}

public static void main(String args[]) {

    TailerListener listener  = new ShowLinesListener();
    File file = new File("./test.log");

    Tailer tailer = new Tailer(file, listener, 1000);
    tailer.run();

    try {
        Thread.sleep(100000);
    } catch(InterruptedException ex) {
        Thread.currentThread().interrupt();
    }

    tailer.stop();
}

}

Bentley

tailer.run() blocks so you will never get to the lines following it. Need to run Tailer in a separate thread. – Flea

Answer 3

In Linux you would not have a problem since the locking is only advisory. But in Windows things are different.
This should depend on the mode that your logger (I assume log4j) opens the file to log.
Looking in this answer for C# it seems that there is a parameter dwShareMode that can be used for this kind of sharing.

I believe NIO would be the way to go. Look if dwShareMode parameter is available as part of NIO APIs

Underside