Opening and reading from a file without locking it
Asked Answered
Q

4

13

I need to be able to mimic tail -f with Java. I'm trying to read a log file as it's being written by another process, but when I open the file to read it, it locks the file and the other process can't write to it anymore. Any help would be greatly appreciated!

Here is the code that I'm using currently:

public void read() {
    Scanner fp = null;
    try {
        fp = new Scanner(new FileReader(this.filename));
        fp.useDelimiter("\n");
    } catch (java.io.FileNotFoundException e) {
        System.out.println("java.io.FileNotFoundException e");
    }

    while (true) {
        if (fp.hasNext()) {
            this.parse(fp.next());
        }           
    }       
}
Quickstep answered 29/3, 2010 at 10:52 Comment(8)
If nothing else works, you can use jni to get to the win32 api (assuming you're on windows, otherwise whatever api you're working with).Materialism
using the win32 api would be the ugliest thing ;)Spongioblast
Heh since I don't know Java that well, that's the best advice I have :)Materialism
I selected Java for portability, so this isn't an option. I will be looking into the FileChannel API though that Cshah mentioned when I get home. Thanks for the helpQuickstep
The JNI suggestion is way off base. It is the Win32 API that is doing the locking here. Java doesn't do that. Using FileChannel won't help either. I would review the entire requirement. Log files aren't there to be parsed, they are for humans. If you need communications from another part of the application, use a Socket or a database.Microclimatology
I talked about the "un"locking below, but if you care about the tail part: (tailing files) could be solved with the Apache Commons IO Tailer.Granddaughter
The JNI suggestion is valid since Windows does not force locking anymore as of Windows 2000. However only the new Java file API tells Windows to not lock. See bugs.java.com/view_bug.do?bug_id=6357433 for more details. Still, you shouldn't use JNI as this breaks portability for no reason; Java NIO can be used instead.Hobbism
https://mcmap.net/q/906510/-using-java-filechannel-filelock-to-prevent-file-writes-but-allow-readsKrieg
H
12

Rebuilding tail is tricky due to some special cases like file truncation and (intermediate) deletion. To open the file without locking, use StandardOpenOption.READ with the new Java file API:

try (InputStream is = Files.newInputStream(path, StandardOpenOption.READ)) {
    InputStreamReader reader = new InputStreamReader(is, fileEncoding);
    BufferedReader lineReader = new BufferedReader(reader);
    // Process all lines.
    String line;
    while ((line = lineReader.readLine()) != null) {
        // Line content content is in variable line.
    }
}

For my attempt to create a tail in Java see:

Feel free to take inspiration from that code or simply copy the parts you require. Let me know if you find any issues that I'm not aware of.

Hobbism answered 12/11, 2014 at 1:20 Comment(0)
B
2

java.io gives you a mandatory file lock and java.nio gives you an advisory file lock

If you want to read any file without any lock you can use below classes

import java.nio.channels.FileChannel;
import java.nio.file.Paths;

If you want to tail a file line by line use below code for the same

public void tail(String logPath){
    String logStr = null;
    FileChannel fc = null;
    try {
        fc = FileChannel.open(Paths.get(logPath), StandardOpenOption.READ);
        fc.position(fc.size());
    } catch (FileNotFoundException e1) {
        System.out.println("FileNotFoundException occurred in Thread : " + Thread.currentThread().getName());
        return;
    } catch (IOException e) {
        System.out.println("IOException occurred while opening FileChannel in Thread : " + Thread.currentThread().getName());
    }
    while (true) {
        try {
            logStr = readLine(fc);
            if (logStr != null) {
                System.out.println(logStr);
            } else {
                Thread.sleep(1000);
            }
        } catch (IOException|InterruptedException e) {
            System.out.println("Exception occurred in Thread : " + Thread.currentThread().getName());
            try {
                fc.close();
            } catch (IOException e1) {
            }
            break;
        }
    }
}

private String readLine(FileChannel fc) throws IOException {
    ByteBuffer buffers = ByteBuffer.allocate(128);
    // Standard size of a line assumed to be 128 bytes
    long lastPos = fc.position();
    if (fc.read(buffers) > 0) {
        byte[] data = buffers.array();
        boolean foundTmpTerminator = false;
        boolean foundTerminator = false;
        long endPosition = 0;
        for (byte nextByte : data) {
            endPosition++;
            switch (nextByte) {
            case -1:
                foundTerminator = true;
                break;
            case (byte) '\r':
                foundTmpTerminator = true;
                break;
            case (byte) '\n':
                foundTmpTerminator = true;
                break;
            default:
                if (foundTmpTerminator) {
                    endPosition--;
                    foundTerminator = true;
                }
            }
            if (foundTerminator) {
                break;
            }
        }
        fc.position(lastPos + endPosition);
        if (foundTerminator) {
            return new String(data, 0, (int) endPosition);
        } else {
            return new String(data, 0, (int) endPosition) + readLine(fc);
        }
    }
    return null;
}
Bouley answered 22/11, 2018 at 6:10 Comment(0)
C
1

Look at the FileChannel API here. For locking the file you can check here

Cerebroside answered 29/3, 2010 at 11:1 Comment(5)
You can look at the possible duplicate question in stackoverflow at #128538Cerebroside
Hmm. I've been looking it over and tried several strategies -- even using java.io.RandomAccessFile in addition to FileChannel, but I still can't manage to let another process write to a file that I'm reading in Java.Quickstep
rogue780 - post the strategies you've tried. Note that if the other process requires an exclusive lock before it writes to the file, then you are pretty much toast (of course, tail wouldn't work in this situation either). But have you tried locking (with a non-exclusive read lock) the top portion of the file?Highball
The behavior of locks is also specific to the underlying operating system.I havent tried shared locking but i guess you can try that in linux. I doubt whether Windows would support itCerebroside
more like docs.oracle.com/javase/7/docs/api/java/nio/channels/… and docs.oracle.com/javase/7/docs/api/java/nio/channels/….Fiddlewood
G
1

Windows uses mandatory locking for files unless you specify the right share flags while you open. If you want to open a busy file, you need to Win32-API CreateFile a handle with the sharing flags FILE_SHARE_DELETE | FILE_SHARE_READ | FILE_SHARE_WRITE.

This is used inside the JDK in a few places to open files for reading attributes and stuff, but as far as I can see it is not exported/available to Java Class Library level. So you would need to find a native library to do that.

I think as a quick work around you can read process.getInputStream() from the command "cmd /D/C type file.lck"

Granddaughter answered 12/11, 2014 at 0:49 Comment(1)
I'm not sure when it changed, but my file opened using RandomAccessFile opens with SHARE_READ and SHARE_WRITE. I tested using ProcMon.exeKrieg

© 2022 - 2024 — McMap. All rights reserved.