How is ArrayOutOfBoundsException possible in String.valueOf(int)?

Asked 19/9, 2014 at 20:35 Answered 20/9, 2014 at 22:21

Why does this code sometimes produce ArrayOutOfBoundsException? How is that even possible for String.valueOf(int)?

public static String ipToString(ByteString bs) {
  if (bs == null || bs.isEmpty()) {
    return null;
  } else {
    StringBuilder sb = new StringBuilder();
    boolean started = false;
    for (Byte byt : bs) {
      if (started) {
        sb.append(".");
      }
      sb.append(String.valueOf(byt & 0xFF));
      started = true;
    }

    return sb.toString();
  }
}


java.lang.ArrayIndexOutOfBoundsException: -81914
  at java.lang.Integer.getChars(Integer.java:458)
  at java.lang.Integer.toString(Integer.java:402)
  at java.lang.String.valueOf(String.java:3086)
  at com.mystuff.mypackage.ipToString(MyCode.java:1325)
  ...
  at java.util.concurrent.FutureTask.run(FutureTask.java:266)
  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
  at java.lang.Thread.run(Thread.java:745)

Updates

I don't know the value of the byte when this occurs, but it doesn't seem like it should be possible for any possible value of byte.

Once it happens once, every invocation then errors out with the same exception.

Environment:

java version "1.8.0_20"
Java(TM) SE Runtime Environment (build 1.8.0_20-b26)
Java HotSpot(TM) 64-Bit Server VM (build 25.20-b23, mixed mode)

Chops answered 19/9, 2014 at 20:35 Comment(34)

Does this fall over outside of a multithreaded environment? The stack trace suggests you're running this as part of a thread pool. Is there a synchronization problem? – Hydroscope 19/9, 2014 at 20:42

And what jdk are you using? – Carolynecarolynn 19/9, 2014 at 20:42

@Hydroscope Whatever you do with multithreading, a pure function like Integer.toString must not fail. – Reinold 19/9, 2014 at 20:44

In case you are using IBM JVM, there seems to be a bug in the JIT compiler.. #16060294 – Sthilaire 19/9, 2014 at 20:45

@MarkoTopolnik Well, yes, indeed. Quite right. But I think I would still check whether it fails in as simple a context as possible. The point is that Integer.toString is failing somehow! – Hydroscope 19/9, 2014 at 20:46

@Hydroscope Unsafe publishing would be the best bet, but a simple int would have to be resilient even to that. – Reinold 19/9, 2014 at 20:46

@NareshVavilala Unrelated, this exception does not occur in StringBuilder code. – Reinold 19/9, 2014 at 20:47

@JeanLogeart I can exactly match the line numbers to my JDK 8 source code. – Reinold 19/9, 2014 at 20:48

@MarkoTopolnik I figured it out since it was not working with 7 :o – Carolynecarolynn 19/9, 2014 at 20:51

java version "1.8.0_20" Java(TM) SE Runtime Environment (build 1.8.0_20-b26) Java HotSpot(TM) 64-Bit Server VM (build 25.20-b23, mixed mode) – Hume 19/9, 2014 at 20:51

@HotLicks It's part of Google's Protocol Buffers. It's basically just an immutable list of bytes. – Hydroscope 19/9, 2014 at 20:52

@Hydroscope - So it would be the Android JVM? – Hamlet 19/9, 2014 at 20:53

@HotLicks No, Protocol Buffers is a general Java implementation, not restricted to Android. – Hydroscope 19/9, 2014 at 20:54

It does appear to be more consistent with a JITC bug than anything else. – Hamlet 19/9, 2014 at 20:56

yea it does seem like that – Hume 19/9, 2014 at 20:58

Really, the only way to get to the bottom of this is to find out what the value of byt is. I suggest echoing byt & 0xFF to the console just before that line, so that you can catch the last one before it blows up. Then try a minimal program that just invokes String.valueOf() on that byte value, and see what happens. – Hydroscope 19/9, 2014 at 21:2

there are only 255 possible byte values – Hume 19/9, 2014 at 21:4

@Hydroscope Chances of reproducing like that are one in a million. – Reinold 19/9, 2014 at 21:4

You will probably find byt & 0xFF is an integer; you might try byt & ((byte)0xFF) or add another (byte) in front of it. It should not cause this error though – Camellia 19/9, 2014 at 21:4

@Hydroscope Chances are that it has nothing to do with the actual value of that byte: the code is bulletproof, the only thing that could trigger that issue would be a bug in JVM. – Supper 19/9, 2014 at 21:5

@dasblinkenlight Sure, but you'd want to find a minimal (non-)working example, and that means reducing the code to the smallest point where you can reliably trigger the bug. – Hydroscope 19/9, 2014 at 21:7

@Hydroscope There's no reliable triggering of this bug... I've just written a jmh benchmark utilizing all 4 cores to do work like in the question, going through all byte values, no exceptions. – Reinold 19/9, 2014 at 21:8

Do you get this consistently, on every run? – Herpetology 19/9, 2014 at 21:11

@BruceMartin byt & 0xFF is intentional. It is the standard way to interpret the byte as an unsigned 8-bit integer. – Reinold 19/9, 2014 at 21:12

it's intermittent. There are only 255 possible byte values and this code is executed at high volume of requests on a server serving production traffic – Hume 19/9, 2014 at 21:16

Do you run your app with any VM arguments? – Herpetology 19/9, 2014 at 21:17

When this error occurs, is the index out of bounds always -81914 or does that number change? – Turnkey 19/9, 2014 at 21:18

always that value, and once it happens once every subsequent invocation bombs in the exact same way – Hume 19/9, 2014 at 21:18

Your last update is important: it quite cleanly points to JIT compiler issue (inlining or possibly even some assembler macro code). – Reinold 19/9, 2014 at 21:19

yup, it's like it's caching a bad block – Hume 19/9, 2014 at 21:19

There are 256 possible byte values... – Hydroscope 19/9, 2014 at 21:22

will try -Xjit:exclude={ProgramClass.callStringGetChars*} when I get the chance – Hume 19/9, 2014 at 21:27

-Xjit:exclude={ProgramClass.callStringGetChars*} isn't accepted... – Hume 19/9, 2014 at 21:56

It's probably a different option in the Oracle JVM. – Hamlet 20/9, 2014 at 12:39

This is a JIT compiler bug that has been introduced in JDK 8u20 as a side-effect of another fix:
JDK-8042786

The problem is related to auto-boxing elimination optimization.
The work-around is to switch the optimization off by -XX:-EliminateAutoBox JVM flag

Looks like the problem also exists in the most recent JDK 9 source base.
I've submitted the bug report: https://bugs.openjdk.java.net/browse/JDK-8058847 with 100% reproducible minimal test case included.

Treasurehouse answered 20/9, 2014 at 22:21 Comment(1)

a patch has been accepted to resolve the issue hg.openjdk.java.net/jdk9/hs-comp/hotspot/rev/7723d5b0fca3 – Hume 24/10, 2014 at 21:41

I can reliably reproduce your issue with this code:

public class Main
{
  public static StringBuilder intToString(byte[] bs) {
    final StringBuilder sb = new StringBuilder();
    boolean started = false;
    for (Byte byt : bs) {
      if (started) sb.append(".");
      sb.append(String.valueOf(byt & 0xFF));
      started = true;
    }
    return sb;
  }

  public static void main(String[] args) {
    final byte[] bs = {-2, -1, 0, 1, 2};
    while (true) intToString(bs);
  }
}

The issue will almost certainly be traced to a JIT compiler bug. Your observation that, once it happens the first time, it happens reliably on every subsequent call, points cleanly to a JIT compilation event which introduces the buggy code into the codepath.

If that's available to you, you could activate diagnostic JVM options which will print all compilation events (-XX:PrintCompilation). Then you may be able to correlate such an event with the moment when the exception starts appearing.

Reinold answered 19/9, 2014 at 20:35 Comment(2)

if the problem is caused by for(Byte b : bs) as opposed to for(byte b : bs) - see comments in my answer - can you see if you can reproduce the problem with that change? – Turnkey 22/9, 2014 at 12:50

See my update: that change makes it easily reproducible. One thing that wasn't obvious from your code was whether bs returned a byte or a Byte, and that turns out to be a key difference. – Reinold 22/9, 2014 at 13:21

I am leaving the code snippet here, as it still ought to be run faster than the original code - at a cost of memory - but be advised it doesn't actually fix the problem.

private static final String[] STRING_CACHE = new String[256];

static {
  for(int i = 0; i <= 255; i++) {
    STRING_CACHE[i] = String.valueOf(i);
  }
}

public static String ipToString(ByteString bs) {
  if (bs == null || bs.isEmpty()) {
    return null;
  } else {
    StringBuilder sb = new StringBuilder();
    boolean started = false;
    for (Byte byt : bs) {
      if (started) {
        sb.append(".");
      }
      sb.append(STRING_CACHE[byt & 0xFF]);
      started = true;
    }

    return sb.toString();
  }
}

Turnkey answered 19/9, 2014 at 21:28 Comment(8)

certainly is not enough and workaround is also not asked by OP. – Understructure 19/9, 2014 at 21:33

@KumarAbhinav From the help center How to answer: "What, specifically, is the question asking for? Make sure your answer provides that – or a viable alternative" – Turnkey 19/9, 2014 at 21:37

It should be noted that this bug is described elsewhere as having been fixed a year ago. The OP needs to update his JVM version. – Hamlet 20/9, 2014 at 2:10

@HotLicks But OP is clearly using the newest JVM available? Still, a link to the bug would be the closest to a real answer to this question! – Reinold 20/9, 2014 at 6:29

@MarkoTopolnik - Well, I guess the fixed version (to a nearly identical bug) was in a IBM JVM, whereas this is apparently an Oracle version, so perhaps it's not the same. (Or perhaps it is the same but Oracle hasn't fixed it yet.) – Hamlet 20/9, 2014 at 12:38

I've verified that your suggested workaround crashes JVM - even worse than ArrayIndexOutOfBoundsException. – Treasurehouse 22/9, 2014 at 7:17

See bugs.openjdk.java.net/browse/JDK-8058847. The bug is not in String.valueOf but rather in autoboxing optimization. If replace for (Byte byt : bs) with for (byte byt : bs) the problem will disappear. – Treasurehouse 22/9, 2014 at 12:37

This can be reproduced only on JDK 8u20 and JDK 9. Earlier versions work fine since they have EliminateAutoBox disabled by default. – Treasurehouse 22/9, 2014 at 12:41

Hot tags

Godot Unity Godot Help Programming Godot 4.X GUI GDScript 3D 2D Physics CSharp Godot 3.X VR XR Projects C++

Recommended topics

Hot tags