How are the final multi-threading guarantees and the memory model related in Java?
Asked Answered
K

1

8

The memory model is defined in 17.4. Memory Model.

The final field multi-threading guarantees are given in 17.5. final Field Semantics.

I don't understand why these are separate sections.

AFAIK both final and the memory model provide some guarantees.
And any real program execution must respect both guarantees.
But it's now clear whether the final guarantees work for the intermediate executions used to validate causality requirements in 17.4.8. Executions and Causality Requirements.

Another unclear moment is that 17.5.1. Semantics of final Fields defines a new "special" happens-before, which differs from the happens-before in the memory model:

This happens-before ordering does not transitively close with other happens-before orderings.

If these are the same happens-before, then the happens-before isn't a partial order anymore (because it isn't transitive).
I don't understand how this doesn't break things.

If these are different happens-before, then it's not clear what the one in 17.5. final Field Semantics does.
The happens-before in 17.4. Memory Model is used to restrict what a read can return:

Informally, a read r is allowed to see the result of a write w if there is no happens-before ordering to prevent that read.

But 17.5. final Field Semantics is a different section.

Kaiserism answered 31/7, 2022 at 23:9 Comment(5)
Hi @poq, this is a really interesting although somewhat rarified line of questioning, but it seems too open-ended, asking several "why" questions that are hard to answer succinctly. I am personally very interested in Java concurrency, so I'd like to see this question improved so that it can be answered and be helpful to others. IMHO, it would really help to include some code examples where you can point to a specific uncertainty in the specification, and ask "is this specific sequence guaranteed to happen?" or "can this code break under certain circumstances?". Thanks!Priestridden
@Priestridden I think the question, whilst its got plenty of question marks, is still effectively asking a single question, which is admittedly a bit vague ("Given that the H-B guarantees for final fields listed in a separate chapter vs. all the other ones, explain what the differences are between §17.5 H-B guarantees vs all the other ones in §17.4?" - that's the essential question here). However, I echo your request for code snippets, that would be a fantastic way to improve this question.Hurley
@Priestridden I've converted all the questions except the main one into normal sentences.Kaiserism
@Kaiserism this is one part of the JMM that also isn't clear to me yet. This is an interesting read btw: shipilev.net/blog/2014/all-fields-are-finalGreaseball
You are reading too much into the chapter structure of the specification. Being placed into a subsequent section instead of a subsection doesn’t mean anything; it’s still part of the same specification and there’s no semantic separation at all.Accusative
H
10

The special 'final field guarantees' part was a later add-on. Documentation sometimes follows the quirks of history - possibly, had the 'final field guarantee' issue been discovered prior to the first release of the JMM, the documentation would have been structured differently.

In other words, you're asking for 'why is this stuff in a separate chapter' and perhaps the answer is: "Because it was added in a later version of java, and therefore it was written at a completely different time; a new chapter is presumably the simplest way to add some more documentation". We're talking about decades ago at this point, of course.

§17.5 explains its purpose. Quote:

The usage model for final fields is a simple one: Set the final fields for an object in that object's constructor; and do not write a reference to the object being constructed in a place where another thread can see it before the object's constructor is finished. If this is followed, then when the object is seen by another thread, that thread will always see the correctly constructed version of that object's final fields. It will also see versions of any object or array referenced by those final fields that are at least as up-to-date as the final fields are.

In other words, in the distant past, you could do this:

Thread A:

  • Make a new object. The constructor is 'well behaved' 1
  • Communicate the ref to this new object to another thread. Possibly in an unsafe way.

Thread B:

  • The receiving thread gets the correct ref (either because you did it safely with synchronization, i.e. happens-before relationship set up properly, or because you did it unsafely, but the JMM does not guarantee that unsafe code fails to work: It may still work).
  • It calls a method of this object.
  • Said object witnesses a final-marked field that isn't initialized, because the initialization did occur in thread A, but no happens-before relationship exists, and re-ordering and other shenanigans means that this thread doesn't see it yet.

This is extremely annoying. Part of the point of immutable classes is that you can more or less print out the JMM and set it on fire. You just don't need to care about virtually every tricky rule in it if your system is an amalgamation of immutable types. Except, it didn't actually work out that way prior to the existence of §17.5

The JMM as a general principle is designed to give any JVM implementations as few 'handcuffs' as possible whilst making developing for the JVM as uncomplicated as possible. It's a fine line - for example, had the JMM simply stated: "The JVM is free to re-order whatever it wants at any time, and cache whatever it wants, at any time, for whatever duration it wants", then writing JVMs that run code quickly and according to spec would be 'easier' (JVM impls would be faster), but, writing multithreaded code that actually does what you intended it to becomes borderline impossible. On the flipside, the JMM could also have guaranteed that re-ordering in the JVM is impossible to observe regardless of circumstance or architecture. But then JVMs would be slow as molasses, see Python and its much maligned global interpreter lock.

The JMM tries to be the happy compromise. And §17.5 is written with the same spirit.

It basically says:

  • You CAN rely on the notion that any well-behaved construction means that final fields will just work out without having to worry about happens-before relationships whatsoever.
  • However, you CANNOT, at all, rely on HOW the JVM implements the guarantee. In particular, we have defined what you can exactly rely on in terms of Happens-Before, but it's not the same H-B that the rest of the JMM talks about. We guarantee you that well-behaved construction means final fields won't be an issue but that's as far as our guarantee goes: You cannot use this guarantee to then force other guarantees out of the JMM; you can't use this mechanism as a wonky way to establish H-B for other stuff, for example.

The JMM buys room to maneuver for JVM impls. Whether a JVM impl actually uses it, is up to the JVM implementor. In other words, a JVM implementor may well decide to implement §17.5 by using the same locking mechanisms it uses to guarantee the H-B stuff in §17.4, and thus effectively you can apply properties like 'H-B relationships are transitive'. The point of the JMM is partly to allow JVM impls to take some pretty drastically different approaches to how the guarantees it dictates are in fact guaranteed. That's because JVMs have to be written so that they can run code about as fast as native code could on a wide variety of hardware, whilst still being a target platform that isn't impossible to develop for.

Quite the tightrope walk. This is the primary underlying explanation for the JMM can be obtuse and bizarre at times.

[1] A 'well behaved' constructor:

  • Does not pass its own reference (this) to any code outside of its own class during construction.
  • Does not invoke any of its own instance methods that then read its own fields (or, especially problematic, which can be overridden by subclasses, whose implementation uses its own fields). Basically: Calling any non-final method is an instant "You are not well behaved" violation.
  • Does not send any object refs of things I wishes to store in fields to code in other classes during construction. Even if it has already assigned it to the final field before doing so.
Hurley answered 31/7, 2022 at 23:33 Comment(10)
Why do you think that final chapter was written in a completely different time and added later? I've look at the JSR 133 page and it seems that both sections appeared first in Java 5.0. Also after a quick look at the JSR 133 mailing list it seems like both sections were discussed since 1999.Kaiserism
Well maybe not. But that is in no way central to the point of this Answer. The way that the spec is organized doesn't alter its meaning. And you shouldn't "read" anything into it.Duologue
@StephenC does happens-before used throughout 17.4. Memory Model includes new non-transitive happens-before added in 17.5. final Field Semantics?Kaiserism
You need to read it as "HB is transitive except in the final case". If you think they have written it the wrong way, feel free to submit an issue for it, explaining clearly what you think the problem is.Duologue
@StephenC As I understand "transitive except in the final case" means "not transitive", which contradicts multiple places in 17.4. Memory Model where happens-before is referred to as "partial order" (which must be transitive). It's is even stated explicitly: "It must be a valid partial order: reflexive, transitive and antisymmetric." That would be weird if a later section added something to an already defined term and thus broke the logic in the previous sections.Kaiserism
As I said in this answer, the 'happens-before' that is talked about in §17.5 is not the same 'happens-before' in §17.4. At some point you're asking: "If I wrote this I would write it differently". Okay. Well, it's not written that way. It's not a contradiction: 17.5 spells out: This isn't the same kind of H-B.Hurley
@Kaiserism - If you are still confused about this, or unhappy with the way that the spec is written, you need to talk to the spec writers themselves about it. Raise an OpenJDK issue or email a question to the relevant mailing list. My guess would be that it the "jls-jvms-spec-comments" list.Duologue
@Hurley In this case, do the rules for final fields defined in 17.5. final Field Semantics work in 17.4. Memory Model? Or do they always apply after the rules from 17.4?Kaiserism
They apply separately and do not interact.Hurley
@Hurley Thank you. At this point you've basically answered my question fully and clarified every nuance I didn't understand. I don't accept your question because I have a strong suspicion that your statement "'The special final field guarantees' part was a later add-on" may be factually incorrect. But I upvoted both your answer and your comments, because IMO as a whole this answers my question.Kaiserism

© 2022 - 2024 — McMap. All rights reserved.