Java, the fastest class to read from a txt file [closed]

Question

I have to read txt files in my program. Im currently using FileReader and BufferedReader. I tried to use Scanner but its slower than FileReader and BufferedReader. Is there any class, which can read files faster ? It must be written in Java Language.

I need to read all words(strings splited by white space) from text file

Kassey

Do you have concrete evidence that reading from the file is the bottleneck in your program? What are you doing with the data afterwards, and how large is the file? Note that I would avoid FileReader myself - use InputStreamReader wrapping FileInputStream, so you can explicitly set the encoding. – Verada

Maybe your bottleneck is splitting the line rather than reading the file? There are ways to accelerate String.split(). From my experience, BufferedReader is very fast. – Breechloader

Theoretically, java NIO can read files faster than IO – Sinotibetan

Answer 1

If the files being read is huge then you would want to use BufferedReader on top of a FileReader to improve read performance.

or you may try something likethis:-

 BufferedReader br = new BufferedReader(new FileReader("file.txt"));
try {
    StringBuilder sb = new StringBuilder();
    String line = br.readLine();

    while (line != null) {
        sb.append(line);
        sb.append("\n");
        line = br.readLine();
    }
    String everything = sb.toString();
   } finally {
    br.close();
}

or you can try this program. It works faster for larger files:-

public String readDoc(File f) {
String text = "";
int read, N = 1024 * 1024;
char[] buffer = new char[N];

try {
    FileReader fr = new FileReader(f);
    BufferedReader br = new BufferedReader(fr);

    while(true) {
        read = br.read(buffer, 0, N);
        text += new String(buffer, 0, read);

        if(read < N) {
            break;
        }
    }
} catch(Exception ex) {
    ex.printStackTrace();
}

return text;
}

Warthman

The files that Im going to read can be big(more than 100 MB). Now im doing this like that pastebin.com/WUaJUT1G Can I do it faster? I'm doing it for project on University and I need to make all fragments of program (reading, creating tree and searching in tree) the fastest I can. – Kassey

This approach is a better approach. I have checked it. – Warthman

@user1736332:- I have just updated my answer with the code that works fast for larger files. Kindly check it. – Warthman

Thank you. The second one is faster. Writting and creating tree now takes 6,5s not 7,1 :) – Kassey

Answer 2

Assuming you read all the file in memory, the fastest, from a code writing perspective, is:

List<String> lines = Files.readAllLines(yourFile, charset);

I would expect performance, from an execution perspective, to be as good if not better (this has supposedly been optimised by the team who wrote it).

You can then split or do whatever you need.

Iced

Answer 3

The speed of reading and splitting is 85 MB/sec. I used 560 MB file with 20 columns in each line. Here is the code:

package csvreader_speedtest;

import java.io.*;

public class Csvreader_SpeedTest {

    final char delimiter = ',';
    String[] splitted = new String[64];

    Csvreader_SpeedTest(String filename) throws Throwable {
        File file = new File(filename);
        BufferedReader reader = new BufferedReader(new FileReader(file));
        String line;
        long t0 = System.currentTimeMillis();
        while ((line = reader.readLine()) != null) {
            split(line);
        }
        long t1 = System.currentTimeMillis();
        reader.close();
        System.out.println("read " + file.length() + " bytes in " + (t1 - t0) + " ms");
    }

    private void split(String line) {
        int idxComma, idxToken = 0, fromIndex = 0;
        while ((idxComma = line.indexOf(delimiter, fromIndex)) != -1) {
            splitted[idxToken++] = line.substring(fromIndex, idxComma);
            fromIndex = idxComma + 1;
        }
        splitted[idxToken] = line.substring(fromIndex);
    }
}

output:

read 561362951 bytes in 6575 ms

update: if I use splitted = line.split(","); instead of split(line);, the speed drops to 32 MB/sec update 2: without splitting, the speed is 194 MB/sec. How fast do you need it to be?

Breechloader

I need to replace all signs like / . , ! ? - ( ) [ ] { } : ; ' " to " " (space) and then split words flr all white spaces. How to do that fast? – Kassey

This is definitely different question. In order to get most reliable answer I suggest to post it as such, e.i.: "what is the fastest way to replace all appearances of ... in strings by space?" – Breechloader

Answer 4

If your file is big Files.readAllLines wont work. But if you still want to try out NIO it's easy:

FileInputStream fis = new FileInputStream("test.txt");
Reader rdr = Channels.newReader(fis.getChannel(), "UTF-8");
BufferedReader br = new BufferedReader(rdr);
...

Sinotibetan