Java ByteBuffer to String

Question

Is this a correct approach to convert ByteBuffer to String in this way,

String k = "abcd";
ByteBuffer b = ByteBuffer.wrap(k.getBytes());
String v = new String(b.array());

if(k.equals(v))
    System.out.println("it worked");
else
    System.out.println("did not work");

The reason I ask is that is this looks too simple, whereas other approaches like Java: Converting String to and from ByteBuffer and associated problems looks more complex.

Tetrabrach

Well, did you try it? – Cloudburst

Yes I did and it works. But I have seen other implementations which are more complex, like #1252968 – Tetrabrach

@Doorknob et. al. He's missing encoding and his example (when syntax is corrected) will work, but his method is still not right. – Hanseatic

Answer 1

There is simpler approach to decode a ByteBuffer into a String without any problems, mentioned by Andy Thomas.

String s = StandardCharsets.UTF_8.decode(byteBuffer).toString();

Colorimeter

Be aware that UTF-8 might not be the optimal charset for converting bytes to strings and vice-versa. For a 1-to-1 mapping of bytes to chars better use ISO-8859-1, see #9098522. – Skindeep

Also, of you don't really need a string, the CharBuffer decode() returns is a CharSequence (like String), so you can avoid an extra copy and use it directly. – Rags

@DavidEhrmann CharBuffer does not have a decode method - what are you referring to here? – Conflux

@TomAnderson Charset has a decode() method. – Rags

@DavidEhrmann Ah! I completely misparsed your comment, sorry! – Conflux

Answer 2

EDIT (2018): The edited sibling answer by @xinyongCheng is a simpler approach, and should be the accepted answer.

Your approach would be reasonable if you knew the bytes are in the platform's default charset. In your example, this is true because k.getBytes() returns the bytes in the platform's default charset.

More frequently, you'll want to specify the encoding. However, there's a simpler way to do that than the question you linked. The String API provides methods that converts between a String and a byte[] array in a particular encoding. These methods suggest using CharsetEncoder/CharsetDecoder "when more control over the decoding [encoding] process is required."

To get the bytes from a String in a particular encoding, you can use a sibling getBytes() method:

byte[] bytes = k.getBytes( StandardCharsets.UTF_8 );

To put bytes with a particular encoding into a String, you can use a different String constructor:

String v = new String( bytes, StandardCharsets.UTF_8 );

Note that ByteBuffer.array() is an optional operation. If you've constructed your ByteBuffer with an array, you can use that array directly. Otherwise, if you want to be safe, use ByteBuffer.get(byte[] dst, int offset, int length) to get bytes from the buffer into a byte array.

Avelar

and in the ByteBuffer.get function, the input is again an array of bytes, how can I get it? it doesn't make any sense to say again k.getbytes, does it? – Cohen

@WilliamKinaan - You have the byte[] you fed to ByteBuffer.get(byte[] dst, int offset, int length). You you can build a String out of it with the String() constructor `String(byte[] bytes, int offset, int length, Charset charset). You can use the same offset and length values for both calls. – Avelar

There's no k.getBytes() method in java.nio.ByteBuffer (may be not in the version am using). So I used k.array() method which will return byte[]. – Kennith

@MaduraPradeep - In the example code in the question and this answer, k is a String, not a ByteBuffer. – Avelar

Be aware that UTF-8 might not be the optimal charset for converting bytes to strings and vice-versa. For a 1-to-1 mapping of bytes to chars better use ISO-8859-1, see #9098522 – Skindeep

There is yet another problem with this solution that bit me. The ByteBuffer has an arrayOffset(). Normally, this only happens when you slice() a Buffer, but Android was allocating() buffers with an offset! Extremely hard issue to find. – Rager

Answer 3

Try this:

new String(bytebuffer.array(), "ASCII");

NB. you can't correctly convert a byte array to a String without knowing its encoding.

I hope this helps

Racing

UTF-8 is probably a better default guess than ASCII? – Hanseatic

Neither should be specified, given the OP's use of k.getBytes(), which uses the platform's default charset. – Avelar

Not all buffers are backed by an array, so .array() may throw an exception. – Were

Not all bytebuffers support the .array() method. – Roehm

new String(buffer.array(), 0, length, "UTF-8") good approach would be to specify lengths, start and end – Nosography

Careful! If you use array(), you must also use arrayOffset() to start at the correct position in the array! This is a subtle pitfall, because usually arrayOffset() is 0; but in those rare cases where it isn't you will get hard-to-find bugs if you don't take it into account. – Hemorrhoidectomy

This was the only method that worked for me on a java.nio.ByteBuffer instance. String fileContent = new String(bb.array(), StandardCharsets.UTF_8); – Puzzle

Answer 4

Just wanted to point out, it's not safe to assume ByteBuffer.array() will always work.

byte[] bytes;
if(buffer.hasArray()) {
    bytes = buffer.array();
} else {
    bytes = new byte[buffer.remaining()];
    buffer.get(bytes);
}
String v = new String(bytes, charset);

Usually buffer.hasArray() will always be true or false depending on your use case. In practice, unless you really want it to work under any circumstances, it's safe to optimize away the branch you don't need. But the rest of the answers may not work with a ByteBuffer that's been created through ByteBuffer.allocateDirect().

Orvas

If the buffer is created via ByteBuffer.wrap(bytes, offset, size) factory .array() will return the entire bytes array. Better use the form xinyong Cheng suggested – Marsland

The .decode() on Charset is a better solution, agreed. I do feel the context of my answer is useful information, but much less so now. – Orvas

Careful! If you use array(), you must also use arrayOffset() to start at the correct position in the array! This is a subtle pitfall, because usually arrayOffset() is 0; but in those rare cases where it isn't you will get hard-to-find bugs if you don't take it into account. – Hemorrhoidectomy

Answer 5

The answers referring to simply calling array() are not quite correct: when the buffer has been partially consumed, or is referring to a part of an array (you can ByteBuffer.wrap an array at a given offset, not necessarily from the beginning), we have to account for that in our calculations. This is the general solution that works for buffers in all cases (does not cover encoding):

if (myByteBuffer.hasArray()) {
    return new String(myByteBuffer.array(),
        myByteBuffer.arrayOffset() + myByteBuffer.position(),
        myByteBuffer.remaining());
} else {
    final byte[] b = new byte[myByteBuffer.remaining()];
    myByteBuffer.duplicate().get(b);
    return new String(b);
}

For the concerns related to encoding, see Andy Thomas' answer.

Chevet

Answer 6

the root of this question is how to decode bytes to string?

this can be done with the JAVA NIO CharSet:

public final CharBuffer decode(ByteBuffer bb)

FileChannel channel = FileChannel.open(
  Paths.get("files/text-latin1.txt", StandardOpenOption.READ);
ByteBuffer buffer = ByteBuffer.allocate(1024);
channel.read(buffer);

CharSet latin1 = StandardCharsets.ISO_8859_1;
CharBuffer latin1Buffer = latin1.decode(buffer);

String result = new String(latin1Buffer.array());

• First we create a channel and read it in a buffer
• Then decode method decodes a Latin1 buffer to a char buffer
• We can then put the result, for instance, in a String

Expound

Your code is not decoding from latin1 to utf8. While your code is correct, calling the CharBuffer utf8Buffer is somewhat misleading because it has no encoding. – Jabe

Answer 7

Convert a String to ByteBuffer, then from ByteBuffer back to String using Java:

import java.nio.charset.Charset;
import java.nio.*;

String babel = "obufscate thdé alphebat and yolo!!";
System.out.println(babel);
//Convert string to ByteBuffer:
ByteBuffer babb = Charset.forName("UTF-8").encode(babel);
try{
    //Convert ByteBuffer to String
    System.out.println(new String(babb.array(), "UTF-8"));
}
catch(Exception e){
    e.printStackTrace();
}

Which prints the printed bare string first, and then the ByteBuffer casted to array():

obufscate thdé alphebat and yolo!!
obufscate thdé alphebat and yolo!!

Also this was helpful for me, reducing the string to primitive bytes can help inspect what's going on:

String text = "こんにちは";
//convert utf8 text to a byte array
byte[] array = text.getBytes("UTF-8");
//convert the byte array back to a string as UTF-8
String s = new String(array, Charset.forName("UTF-8"));
System.out.println(s);
//forcing strings encoded as UTF-8 as an incorrect encoding like
//say ISO-8859-1 causes strange and undefined behavior
String sISO = new String(array, Charset.forName("ISO-8859-1"));
System.out.println(sISO);

Prints your string interpreted as UTF-8, and then again as ISO-8859-1:

こんにちは
ããã«ã¡ã¯

Subserve

Answer 8

Notice (aside from the encoding issue) that some of the more complicated code linked goes to the trouble of getting the "active" portion of the ByteBuffer in question (for example by using position and limit), rather than simply encoding all of the bytes in the entire backing array (as many of the examples in these answers do).

Fiden

Answer 9

private String convertFrom(String lines, String from, String to) {
    ByteBuffer bb = ByteBuffer.wrap(lines.getBytes());
    CharBuffer cb = Charset.forName(to).decode(bb);
    return new String(Charset.forName(from).encode(cb).array());
};
public Doit(){
    String concatenatedLines = convertFrom(concatenatedLines, "CP1252", "UTF-8");
};

Takahashi

Answer 10

Here is a simple function for converting a byte buffer to string:

public String byteBufferToString(ByteBuffer bufferData) {
    byte[] buffer = new byte[bufferData.readableByteCount()];
    // read bufferData and insert into buffer 
    data.read(buffer);
    // CharsetUtil supports UTF_16, ASCII, and many more
    String text = new String(buffer, CharsetUtil.UTF_8);
    System.out.println("Text: "+text);
    return text;
}

Pam

Answer 11

This was the only method that worked for me on a java.nio.ByteBuffer instance:

String fileContent = new String(bb.array(), StandardCharsets.UTF_8);

Related code snippet below:

import java.nio.file.Path;
import java.nio.file.Paths;
import java.nio.channels.FileChannel;
import java.nio.ByteBuffer;


Path path = Paths.get("/home/binita/testbb");
FileChannel fileChannel = FileChannel.open(path, 
                 EnumSet.of(StandardOpenOption.READ
                    )
                 );  
            
ByteBuffer bb = ByteBuffer.allocate(1024);
int bytesRead = fileChannel.read(bb);
if(bytesRead > 0) {
 String fileContent = new String(bb.array(), StandardCharsets.UTF_8);
}

Puzzle

Answer 12

May be toooo late but here is a solution that works for me.

fun byteBufferToByteString(byteBuffer: ByteBuffer) : ByteString {
        var size = byteBuffer.capacity()
        byteBuffer.position(0)        
        return ByteString.copyFrom(byteBuffer,size)
}

Note: Setting buffer size is important here , as without that, it might give error while converting the ByteString back to bytearray at receivers end.

Apennines