Dumping a Java StringBuilder to File

Question

What is the most efficient/elegant way to dump a StringBuilder to a text file?

You can do:

outputStream.write(stringBuilder.toString().getBytes());

But is this efficient for a very long file?

Is there a better way?

Rhodia

How big would you say "very long" is? On the order of KB, MB, or more? Will your StringBuilder's size approach any practical limits, like the maximum memory allocated to your JVM? – Pepin

Maybe a couple of megabytes... – Rhodia

I cannot do without the StringBuilder, it's what my API is returning. – Rhodia

Answer 1

As pointed out by others, use a Writer, and use a BufferedWriter, but then don't call writer.write(stringBuilder.toString()); instead just writer.append(stringBuilder);.

EDIT: But, I see that you accepted a different answer because it was a one-liner. But that solution has two problems:

1. it doesn't accept a java.nio.Charset. BAD. You should always specify a Charset explicitly.

2. it's still making you suffer a stringBuilder.toString(). If the simplicity is really what you're after, try the following from the Guava project:

Files.write(stringBuilder, file, Charsets.UTF_8)

Bianchi

writer.write(); does not take StringBuilder as argument. I can specify the encoding with FileUtils.writeStringToFile(file, String, String encoding) – Rhodia

Terribly sorry -- that was supposed to say writer.append()! Fixing it. – Bianchi

Also, thanks for clarifying about FileUtils, but still, specifying a Charset as a String is lame. – Bianchi

+1. Is there any reason to use a BufferedWriter when we only do one write/append call? – Cartouche

Looking at the source of Writer.append() we see a call write(csq.toString() so it still calls toString() on the string builder object. So nothing gained. – Lamas

Given time and opinions, I am marking this as the correct answer now. But the other options below might still be useful to many. – Rhodia

I downvoted as it doesn't show a full example. – Trotta

Answer 2

You should use a BufferedWriter to optimize the writes (always write character data using a Writer instead of an OutputStream). If you weren't writing character data, you would use a BufferedOutputStream.

File file = new File("path/to/file.txt");
BufferedWriter writer = null;
try {
    writer = new BufferedWriter(new FileWriter(file));
    writer.append(stringBuilder);
} finally {
    if (writer != null) writer.close();
}

or, using try-with-resources (Java 7 and up)

File file = new File("path/to/file.txt");
try (BufferedWriter writer = new BufferedWriter(new FileWriter(file))) {
    writer.append(stringBuilder);
}

Since you're ultimately writing to a file, a better approach would be to write to the BufferedWriter more often instead of creating a huge StringBuilder in-memory and writing everything at the end (depending on your use-case, you might even be able to eliminate the StringBuilder entirely). Writing incrementally during processing will save memory and will make better use of your limited I/O bandwidth, unless another thread is trying to read a lot of data from the disk at the same time you're writing.

Pepin

Nice answer, thanks. However, for the sake of completeness: You may also want to call writer.flush() and / or writer.close() to make sure that the complete String is actually written before the program or thread terminates. – Acetate

First version doesn't compile in Java 5. Writer isn't found in the finally clause – Spittoon

@Acetate Thanks for pointing out that we need to have 'writer.close();'. I was facing a problem that not all my strings were written into the file and I debugged for several hours but cannot find out the reason. Then I try 'writer.close();', everything just works perfectly. – Rieger

Answer 3

You could use the Apache Commons IO library, which gives you FileUtils:

FileUtils.writeStringToFile(file, stringBuilder.toString(), Charset.forName("UTF-8"))

Minutely

I am choosing this as the preferred answer just because it abstracts the complications from me. Given that it might not be the most efficient. But the other answers are great too, might use them if efficiency starts to suffer. – Rhodia

this has two major problems which I've explained in my answer. – Bianchi

Answer 4

Well, if the string is huge, toString().getBytes() will create duplicate bytes (2 or 3 times). The size of the string.

To avoid this, you can extract chunk of the string and write it in separate parts.

Here is how it may looks:

final StringBuilder aSB = ...;
final int    aLength = aSB.length();
final int    aChunk  = 1024;
final char[] aChars  = new char[aChunk];

for(int aPosStart = 0; aPosStart < aLength; aPosStart += aChunk) {
    final int aPosEnd = Math.min(aPosStart + aChunk, aLength);
    aSB.getChars(aPosStart, aPosEnd, aChars, 0);                 // Create no new buffer
    final CharArrayReader aCARead = new CharArrayReader(aChars); // Create no new buffer

    // This may be slow but it will not create any more buffer (for bytes)
    int aByte;
    while((aByte = aCARead.read()) != -1)
        outputStream.write(aByte);
}

Hope this helps.

Aquaplane

+1 It's not really slower, just tested it with a 50MB String. But it really saves memory. (approx. 2MB vs. 130MB for the other methods) – Interfere

@Aquaplane The "big" performance differences come from the underlying OutputStream. In many cases the write(array) method call decomposes into a while-loop internally. Nice example. – Tammitammie

Is this more memory efficient than the .append solution? I figure writer may be doing similar stuff under the hood. – Cartouche

@Thomas Ahle: As far as I know (and tried), append is the every if not the most efficient one. Another one that is very efficient (for Stream) is write(byte). Java is open sourced now so you can see the code and as I remember the implementation of append and write are always related. – Aquaplane

@Aquaplane Yeah, just checked it, append(CharacterStream cs) = write(cs.toString()) – Cartouche

Answer 5

For character data better use Reader/Writer. In your case, use a BufferedWriter. If possible, use BufferedWriter from the beginning on instead of StringBuilder to save memory.

Note that your way of calling the non-arg getBytes() method would use the platform default character encoding to decode the characters. This may fail if the platform default encoding is for example ISO-8859-1 while your String data contains characters outside the ISO-8859-1 charset. Better use the getBytes(charset) where in you can specify the charset yourself, such as UTF-8.

Tepper

Answer 6

Since java 8 you only need to do this:

Files.write(Paths.get("/path/to/file/file_name.extension"), stringBuilder.toString().getBytes());

You don't need any third party libraries to do that.

Hertz

The problem is calling stringBuilder.toString() if the corresponding String is large. And your answer doesn't help. – Threnody

Answer 7

If the string itself is long, you definitely should avoid toString(), which makes another copy of the string. The most efficient way to write to stream should be something like this,

OutputStreamWriter writer = new OutputStreamWriter(
        new BufferedOutputStream(outputStream), "utf-8");

for (int i = 0; i < sb.length(); i++) {
    writer.write(sb.charAt(i));
}

Lysimeter

Please don't use writer.append(sb). It's same as writer.write(sb.toString()). So it defeats the purpose. – Lysimeter

It's not very efficient to write it char by char, is it? Solutions based on buffer are probably faster. – Threnody

Answer 8

Based on https://mcmap.net/q/352598/-dumping-a-java-stringbuilder-to-file

I create this function that use OutputStreamWriter and the write(), this is memory optimized too, better than just use StringBuilder.toString().

public static void stringBuilderToOutputStream(
        StringBuilder sb, OutputStream out, String charsetName, int buffer)
        throws IOException {
    char[] chars = new char[buffer];
    try (OutputStreamWriter writer = new OutputStreamWriter(out, charsetName)) {
        for (int aPosStart = 0; aPosStart < sb.length(); aPosStart += buffer) {
            buffer = Math.min(buffer, sb.length() - aPosStart);
            sb.getChars(aPosStart, aPosStart + buffer, chars, 0);
            writer.write(chars, 0, buffer);
        }
    }
}

Mercurate

Answer 9

Benchmarks for most answers here + improved implementation: https://www.genuitec.com/dump-a-stringbuilder-to-file/

The final implementation is along the lines of

try {
    BufferedWriter bw = new BufferedWriter(
            new OutputStreamWriter(
                    new FileOutputStream(file, append), charset), BUFFER_SIZE);
    try {
        final int length = sb.length();
        final char[] chars = new char[BUFFER_SIZE];
        int idxEnd;
        for ( int idxStart=0; idxStart<length; idxStart=idxEnd ) {
            idxEnd = Math.min(idxStart + BUFFER_SIZE, length);
            sb.getChars(idxStart, idxEnd, chars, 0);
            bw.write(chars, 0, idxEnd - idxStart);
        }
        bw.flush();
    } finally {
        bw.close();
    }
} catch ( IOException ex ) {
    ex.printStackTrace();
}

Lodger

Hmm. Is idxStart=idxEnd a feature or a bug? – Threnody

Definitely a feature… – Lodger