Can object access be reordered with that object's final field access in Java?

Question

Below code sample is taken from JLS 17.5 "final Field Semantics":

class FinalFieldExample { 
    final int x;
    int y; 
    static FinalFieldExample f;

    public FinalFieldExample() {
        x = 3; 
        y = 4; 
    } 

    static void writer() {
        f = new FinalFieldExample();
    } 

    static void reader() {
        if (f != null) {
            int i = f.x;  // guaranteed to see 3  
            int j = f.y;  // could see 0
        } 
    } 
}

Since the instance of FinalFieldExample is published through a data race, is it possible that the f != null check evaluates successfully, yet subsequent f.x dereference sees f as null?

In other words, is it possible to get a NullPointerException on line that is commented with "guaranteed to see 3"?

Loveridge

Well, you are working with initalized instance. So constructor has already been invoked thats why x will be initialized – Jewish

What if writer() and reader() are called by different threads? In absence of synchronization can f.x be "reordered" with respect to f != null? – Loveridge

No. In your example it won't happen – Jewish

@Jewish some explanation would be nice - why reordering won't happen in this case. Preferably with some JLS link. – Oosperm

Well. This example is got from JLS 17.5 where we see the comment: Because the writer method writes f after the object's constructor finishes, the reader method will be guaranteed to see the properly initialized value for f.x: it will read the value 3. However, f.y is not final; the reader method is therefore not guaranteed to see the value 4 for it. – Jewish

It looks like you reason about the program's behavior from the point of view of sequential consistency. Which is not necessarily applicable here, since SC is not guaranteed in the presence of data races (publishing the object is done via data race here). I believe there is a difference between reading a field, and reading a reference to the object that contains the field. The reference itself is neither final, nor volatile so it has no special semantics under the JMM (as far as I understand it, at least). Which makes me think that reordering is possible – Loveridge

It is probably worth emphasizing, that I'm not asking if it's possible to see 0 instead of 3 - that's clearly not the case. I am interested in the possibility of NPE specifically – Loveridge

This partially answers my question: #6599558 . In short, there seems to be nothing in the JMM that prevents this kind of reordering. So either some kind of synchronization is needed to publish the object after all, or the code in reader() can be turned into a benign race by "caching" the static field in a local variable, and using the latter for both accesses: the null check, and the field dereference – Loveridge

Well if we look at the The JSR-133 Cookbook gee.cs.oswego.edu/dl/jmm/cookbook.html we see: The initial load (i.e., the very first encounter by a thread) of a final field cannot be reordered with the initial load of the reference to the object containing the final field. This comes into play in: x = sharedRef; ... ; i = x.finalField; A compiler would never reorder these since they are dependent, but there can be consequences of this rule on some processors. – Jewish

I don't think application programmers should ever resort to JSR-133 Cookbook as it is just a set of recommendations on how a JVM can possibly be implemented. Even HotSpot does not always follow the cookbook, let alone the other JVM implementations. Language specification should be the single source of truth in such matters, and as stated earlier there is nothing in the JMM that prevents this reordering. – Loveridge

Additional clarification: I meant "reordering" in the sense of reader thread seeing the results of operations in the writer thread in a way that is inconsistent with the program order. The exact nature of reordering is of little interest to me. Whether compiler reorders the actual instructions, CPU executes them in a different order or cache writes are propagated to main memory out of order is irrelevant. – Loveridge

I couldn't resist peeking into the cookbook after all :) "These rules imply that reliable use of final fields by Java programmers requires that the load of a shared reference to an object with a final field itself be synchronized, volatile, or final" - which is obviously not the case in my example, as the shared reference to an object with a final field is published through a data race. So looks like the cookbook too hints that this reordering is possible. Not to say this is a strong argument though, I would still prefer to stick solely to the JMM – Loveridge

Ok. I dont have any more arguments:) Just for interest to read article by Shipilev, there is a similar example i think shipilev.net/blog/2014/jmm-pragmatics/#_part_v_finals – Jewish

The funny thing is this exact article (or, rather, the talk he gave based on it) was what prompted me to ask this :) Notice how "Test Your Basic Understanding" slide in the Finals part contains "if (a != null) println(a.f)", and the next slide where he fixes the races contains "A ta = a; if (ta != null) println(ta.f)". He mentioned it very briefly during the talk, that storing the result in a local var is needed to avoid NPE. Only after that he introduced the final modifier, which made me wonder whether the "cache in local var" trick could be dropped – Loveridge

youtube.com/watch?v=Ky1_5mabd18 - the relevant part in the talk 20:20 to 21:00. I couldn't find an english version of this particular talk, so the one reference above is in Russian – Loveridge

Answer 1

Okay, here is my own take on it, based on quite a detailed talk (in Russian) on final semantics given by Vladimir Sitnikov, and subsequent revisit of JLS 17.5.1.

Final field semantics

The specification states:

Given a write w, a freeze f, an action a (that is not a read of a final field), a read r1 of the final field frozen by f, and a read r2 such that hb(w, f), hb(f, a), mc(a, r1), and dereferences(r1, r2), then when determining which values can be seen by r2, we consider hb(w, r2).

In other words, we are guaranteed to see the write to a final field if the following chain of relations can be built:

hb(w, f) -> hb(f, a) -> mc(a, r1) -> dereferences(r1, r2)

1. hb(w, f)

w is the write to the final field: x = 3

f is the "freeze" action (exiting FinalFieldExample constructor):

Let o be an object, and c be a constructor for o in which a final field f is written. A freeze action on final field f of o takes place when c exits, either normally or abruptly.

As the field write comes before finishing the constructor in program order, we can assume that hb(w, f):

If x and y are actions of the same thread and x comes before y in program order, then hb(x, y)

2. hb(f, a)

Definition of a given in the specification is really vague ("action, that is not a read of a final field")

We can assume that a is publishing a reference to the object (f = new FinalFieldExample()) since this assumption does not contradict the spec (it is an action, and it is not a read of a final field)

Since finishing constructor comes before writing the reference in program order, these two operations are ordered by a happens-before relationship: hb(f, a)

3. mc(a, r1)

In our case r1 is a "read of the final field frozen by f" (f.x)

And this is where it starts to get interesting. mc (Memory Chain) is one of the two additional partial orders introduced in "Semantics of final Fields" section:

There are several constraints on the memory chain ordering:

• If r is a read that sees a write w, then it must be the case that mc(w, r).
• If r and a are actions such that dereferences(r, a), then it must be the case that mc(r, a).
• If w is a write of the address of an object o by a thread t that did not initialize o, then there must exist some read r by thread t that sees the address of o such that mc(r, w).

For the simple example given in question we're really only interested in the first point, as the other two are needed to reason about more complicated cases.

Below is the part that actually explains why it is possible to get an NPE:

• notice the bold part in the spec quote: mc(a, r1) relation only exists if the read of the field sees the write to the shared reference
• f != null and f.x are two distinct read operations from the JMM standpoint
• there is nothing in the spec that says that mc relations are transitive with respect to program-order or happens-before
• therefore if f != null sees the write done by another thread, there are no guarantees that f.x sees it too

I won't go into the details of the Dereference Chain constraints, as they are needed only to reason about longer reference chains (e.g. when a final field refers to an object, which in turn refers to another object).

For our simple example it suffices to say that JLS states that "dereferences order is reflexive, and r1 can be the same as r2" (which is exactly our case).

Safe way of dealing with unsafe publication

Below is the modified version of the code that is guaranteed to not throw an NPE:

class FinalFieldExample { 
    final int x;
    int y; 
    static FinalFieldExample f;

    public FinalFieldExample() {
        x = 3; 
        y = 4; 
    } 

    static void writer() {
        f = new FinalFieldExample();
    } 

    static void reader() {
        FinalFieldExample local = f;
        if (local != null) {
            int i = local.x;  // guaranteed to see 3  
            int j = local.y;  // could see 0
        } 
    } 
}

The important difference here is reading the shared reference into a local variable. As stated by JLS:

Local variables ... are never shared between threads and are unaffected by the memory model.

Therefore, there is only one read from shared state from the JMM standpoint.

If that read happens to see the write done by another thread, it would imply the two operations are connected with a memory chain (mc) relationship. Furthermore, local = f and i = local.x are connected with dereference chain relationship, which gives us the whole chain mentioned in the beginning:

hb(w, f) -> hb(f, a) -> mc(a, r1) -> dereferences(r1, r2)

Loveridge

Another great post by Alexey Shipilev that confirms that reordering is possible: shipilev.net/blog/2014/safe-public-construction/… . Example with FinalWrapperFactory class and corresponding comment are worth checking out: "Also notice we only do a single non-synchronized read of wrapper. Even though this read is racy, we recover from accidentally reading "null". If we were to read wrapper a second time before returning, that would set us up for an opportunity to read "null" again, and then return it." – Loveridge

Answer 2

Your analysis is beautiful (1+), if I could upvote twice - I would. Here is one more link to the same problem with "independent reads" here, for example.

I have also tried to approach this problem in a different answer too.

I think if we introduce the same concept here, things could be provable, too. Let's take that method and slightly change it:

static void reader() {

    FinalFieldExample instance1 = f;

    if (instance1 != null) {

        FinalFieldExample instance2 = f;
        int i = instance2.x;    

        FinalFieldExample instance3 = f;
        int j = instance3.y;  
    } 
}

And a compiler can now do some eager reads (move those reads before the if statement):

static void reader() {

    FinalFieldExample instance1 = f;
    FinalFieldExample instance2 = f;
    FinalFieldExample instance3 = f;

    if (instance1 != null) {
        int i = instance2.x;    
        int j = instance3.y;  
    } 
}

Those reads can be further re-ordered between them:

static void reader() {

    FinalFieldExample instance2 = f;
    FinalFieldExample instance1 = f;
    FinalFieldExample instance3 = f;

    if (instance1 != null) {
        int i = instance2.x;    
        int j = instance3.y;  
    } 
}

Things should be trivial from here: ThreadA reads FinalFieldExample instance2 = f; to be null, before it does the next read : FinalFieldExample instance1 = f; some ThreadB calls writer (as such f != null) and the part:

 FinalFieldExample instance1 = f;

is resolved to non-null.

Dutton

Thanks! It does help to view this from a different angle, reasoning not only about the theoretical possibility, but also about how and why a compiler would want to do this kind of reordering – Loveridge

@NikitaTkachenko you might also like this, where the man himself shows just how amazing this stuff is... IMHO. – Dutton