Why does IndexOutOfBoundsException now have a constructor with a long index as a parameter in Java 16?

Question

I was checking the implementation of IndexOutOfBoundsException in JDK 16, and I have noticed that a new constructor with a long index has been introduced:

/**
 * Constructs a new {@code IndexOutOfBoundsException} class with an
 * argument indicating the illegal index.
 *
 * <p>The index is included in this exception's detail message.  The
 * exact presentation format of the detail message is unspecified.
 *
 * @param index the illegal index.
 * @since 16
 */
public IndexOutOfBoundsException(long index) {
    super("Index out of range: " + index);
}

From what I know, array indices are usually int values, and this is confirmed in the Language Specification section §10.4:

Arrays must be indexed by int values; short, byte, or char values may also be used as index values because they are subjected to unary numeric promotion (§5.6) and become int values.

An attempt to access an array component with a long index value results in a compile-time error.

Any idea when (and why) this long index constructor would be used ?

Pernik

maybe for Lists? – Shaefer

But lists should not be facing index out of bounds issues. – Pernik

"lists should not be facing index out of bounds issues" ... are you sure? What expect you to happen with new ArrayList<>().get(0)? Are you confusing IndexOutOfBoundsException with ArrayIndexOutOfBoundsException? – Clavicle

@Clavicle For List public E get(int index) { try { return listIterator(index).next(); } catch (NoSuchElementException exc) { throw new IndexOutOfBoundsException("Index: "+index); } it is also handled as int – Styria

IndexOutOfBoundsException is thrown by NIO / IO classes also, so isn't just for occasions where an int might be used. – Bizet

@Boug That doesn't prevent you from writing your own subclass of list with your own get variant with long to match you specific needs to that new subtype. Just because Javas standard classes don't have that out of the box means that you can't either. – Clavicle

@Clavicle You're right, but I was still wondering why there was a need for the new constructor, given that a list accesses elements with an int. – Pernik

Usually it does and it looks like an oversight to limit Lists to int. A LinkedList doesn't need that restriction. No one would use that to store more than Integer.MAX item in it, due to the bad performance, but it would be technically possible. Being then not able to throw a proper exception with the invalid index, because there were only IndexOutOfBoundsException(int), would be confusing for the receiver of the exception. But I don't think that new constructor will be used for List, it is impractical with that many items. – Clavicle

The speculation about List is misplaced (though we would eventually like to lift the 32 bit limits there too.) This was precipitated by Project Panama, which brings better native heap access to Java. The Foreign Memory API (a replacement for direct byte buffers) allows long-indexed heap access to native memory segments, motivating this change to IOOBE. – Helminthiasis

While it is most likely related to Panama, this woul also benefit Project Lilliput. "- If we leave arraylength in its own 64-bit field, perhaps we should consider 64-bit addressable arrays?" – Talich

If the largest index has type int, its largest value is Integer.MAX_VALUE. If your array goes out of bounds you could have tried a larger value than this. So you need a long :-) – Korykorzybski

Answer 1

Quoting from the comments for future reference:

This was precipitated by Project Panama, which brings better native heap access to Java. The Foreign Memory API (a replacement for direct byte buffers) allows long-indexed heap access to native memory segments, motivating this change to IOOBE. – Brian Goetz

TL;DR It is related with the following feature Enhancement (JDK-8255150) : Add utility methods to check long indexes and ranges

Description
This is related to JDK-8135248. The goal is to add a similar set of methods but rather than operate on int arguments, the new methods operate on long arguments.

The new methods in Objects are:

public static long checkIndex(long index, long length) public static long checkFromToIndex(long fromIndex, long toIndex, long length) public static long checkFromIndexSize(long fromIndex, long size, long length)

They mirror the int utility methods.

As is the case with the int checkIndex(), the long checkIndex() method will be JIT compiled as an intrinsic. That allows the JIT to compile checkIndex to an unsigned comparison and properly recognize it as range check that then becomes a candidate for the existing range check optimizations. This has proven to be important for panama's MemorySegment and a prototype of this change (with some extra c2 improvements) showed that panama micro benchmark results improve significantly.

From another source about the subject : JDK 16: Checking Indexes and Ranges of Longs:

In my last post, I described the day period support added with JDK 16 Early Access Build 25. That same build also added methods for checking indexes and ranges of long values, which is the subject of this post. JDK-8255150 (“Add utility methods to check long indexes and ranges”) is the Enhancement used to add utility methods for checking long indexes and ranges similar to what JDK-8135248 (“Add utility methods to check indexes and ranges”) added for integers with JDK 9. JDK-8255150 states, “The goal is to add a similar set of methods [as JDK-8135248] but rather than operate on int arguments, the new methods operate on long arguments.”

The greatest beneficiary of these newly added long-supporting methods may be the authors, maintainers, and users of the foreign memory access API as described in this mailing list message: “We have to jump through quite a few hoops in the implementation of the foreign memory access API in order to leverage the intrinsification of int-based index checks, and even then we are not covering the cases where the numbers are larger than ints. Looking forward to being able to remove those hacks!”

Dorolice

Answer 2

I have found another Ticket in OpenJdk which is relevant to that change. As stated there

Bounds checking is not difficult to write explicitly but it can be easy to make trivial mistakes, such as introducing overflow bugs. It is advantageous to consolidate such checks from a correctness and security/integrity perspective. Further more in certain cases it is an opportunity to optimize, via an intrinsic, certain checks and guide hotspot towards unsigned comparisons.

Enhancements to the Java platform will enable optimizations of loops over bounds greater than the minimum and maximum range of int values, requiring bounds checking operating on long values.

In the Foreign Memory Access API (JEP 393), the bounds of a memory segments are expressed as long values. Since bound checks involving longs are not currently optimized, the implementation of the foreign memory access API had to resort to several tricks to gauge whether a memory segment can be considered "small" (e.g. whose size fits in an int value) and then use int operations on small segments, accordingly. While in most cases, these workarounds are hidden inside the API implementation, they add a significant cost in terms of complexity and long term maintenance.

Solution Overload the existing int accepting bounds check methods defined in java.util.Objects with long accepting bounds check methods.

The following static methods are added to java.util.Objects. The specification is identical to that of the existing int accepting bounds check methods of the same method name.

/**
 * Checks if the {@code index} is within the bounds of the range from
 * {@code 0} (inclusive) to {@code length} (exclusive).
 *
 * <p>The {@code index} is defined to be out of bounds if any of the
 * following inequalities is true:
 * <ul>
 *  <li>{@code index < 0}</li>
 *  <li>{@code index >= length}</li>
 *  <li>{@code length < 0}, which is implied from the former inequalities</li>
 * </ul>
 *
 * @param index the index
 * @param length the upper-bound (exclusive) of the range
 * @return {@code index} if it is within bounds of the range
    
 * @throws IndexOutOfBoundsException if the {@code index} is out of bounds
   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^     
 * @since 16
 */
public static
long checkIndex(long index, long length)

/**
 * Checks if the sub-range from {@code fromIndex} (inclusive) to
 * {@code toIndex} (exclusive) is within the bounds of range from {@code 0}
 * (inclusive) to {@code length} (exclusive).
 *
 * <p>The sub-range is defined to be out of bounds if any of the following
 * inequalities is true:
 * <ul>
 *  <li>{@code fromIndex < 0}</li>
 *  <li>{@code fromIndex > toIndex}</li>
 *  <li>{@code toIndex > length}</li>
 *  <li>{@code length < 0}, which is implied from the former inequalities</li>
 * </ul>
 *
 * @param fromIndex the lower-bound (inclusive) of the sub-range
 * @param toIndex the upper-bound (exclusive) of the sub-range
 * @param length the upper-bound (exclusive) the range
 * @return {@code fromIndex} if the sub-range within bounds of the range
 * @throws IndexOutOfBoundsException if the sub-range is out of bounds
   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 * @since 16
 */
public static
long checkFromToIndex(long fromIndex, long toIndex, long length)

/**
 * Checks if the sub-range from {@code fromIndex} (inclusive) to
 * {@code fromIndex + size} (exclusive) is within the bounds of range from
 * {@code 0} (inclusive) to {@code length} (exclusive).
 *
 * <p>The sub-range is defined to be out of bounds if any of the following
 * inequalities is true:
 * <ul>
 *  <li>{@code fromIndex < 0}</li>
 *  <li>{@code size < 0}</li>
 *  <li>{@code fromIndex + size > length}, taking into account integer overflow</li>
 *  <li>{@code length < 0}, which is implied from the former inequalities</li>
 * </ul>
 *
 * @param fromIndex the lower-bound (inclusive) of the sub-interval
 * @param size the size of the sub-range
 * @param length the upper-bound (exclusive) of the range
 * @return {@code fromIndex} if the sub-range within bounds of the range
 * @throws IndexOutOfBoundsException if the sub-range is out of bounds
   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 * @since 16
 */
public static
long checkFromIndexSize(long fromIndex, long size, long length)

The following constructor is added to java.lang.IndexOutOfBoundsException:

/**
 * Constructs a new {@code IndexOutOfBoundsException} class with an
 * argument indicating the illegal index.
 *
 * <p>The index is included in this exception's detail message.  The
 * exact presentation format of the detail message is unspecified.
 *
 * @param index the illegal index.
 * @since 16
 */
public IndexOutOfBoundsException(long index)

Jira Issue: Add utility methods to check long indexes and ranges

Styria

Answer 3

An IndexOutOfBoundsException is not necessarily thrown only when accessing an array (there is the subclass ArrayIndexOutOfBoundsException for that) or a List; you can throw it from any of your own code, and it seems reasonable to allow user-written classes or third-party libraries to throw IndexOutOfBoundsException even if their indices are long rather than int.

Inhabiter