Why must wait() always be in synchronized block

Question

We all know that in order to invoke Object.wait(), this call must be placed in synchronized block, otherwise an IllegalMonitorStateException is thrown. But what's the reason for making this restriction? I know that wait() releases the monitor, but why do we need to explicitly acquire the monitor by making particular block synchronized and then release the monitor by calling wait()?

What is the potential damage if it was possible to invoke wait() outside a synchronized block, retaining it's semantics - suspending the caller thread?

Hierology

Answer 1

A wait() only makes sense when there is also a notify(), so it's always about communication between threads, and that needs synchronization to work correctly. One could argue that this should be implicit, but that would not really help, for the following reason:

Semantically, you never just wait(). You need some condition to be satsified, and if it is not, you wait until it is. So what you really do is

if(!condition){
    wait();
}

But the condition is being set by a separate thread, so in order to have this work correctly you need synchronization.

A couple more things wrong with it, where just because your thread quit waiting doesn't mean the condition you are looking for is true:

• You can get spurious wakeups (meaning that a thread can wake up from waiting without ever having received a notification), or

• The condition can get set, but a third thread makes the condition false again by the time the waiting thread wakes up (and reacquires the monitor).

To deal with these cases what you really need is always some variation of this:

synchronized(lock){
    while(!condition){
        lock.wait();
    }
}

Better yet, don't mess with the synchronization primitives at all and work with the abstractions offered in the java.util.concurrent packages.

Melmela

There's a detailed discussion here as well, saying essentially the same thing. coding.derkeiler.com/Archive/Java/comp.lang.java.programmer/… – Perineum

btw, if you are not to ignore interrupted flag the loop shall check Thread.interrupted() as well. – Authority

I can still do something like: while(!condition){synchronized(this){wait();}} which means there's still a race between checking the condition and waiting even if wait() is correctly called in a synchronized block. So is there any other reason behind this restriction, perhaps due to the way it's implemented in Java? – Gagliano

@Michael Good answer! Small point: spurious wakeups come from the underlying OS and are not triggered by a notify() (and hence no third thread involved). Spurious interrupts on Wikipedia , Interesting read on why they exist – Twinkle

Another nasty scenario: condition is false, we're about to go into wait() and then another thread changes the condition and invokes notify(). Because we're not in wait() yet, we will miss this notify(). In other words, test and wait, as well as change and notify must be atomic. – Chafer

This does not seem to answer the use of lock-free synchronization primitives. If we only can use locks, then yes we need to lock the condition variable before, but if we used lock-free primitives why do we need to use synchronized. What's the rationale of this? – Trautman

@user3438 How could that scenario ever happen? The code block is holding a monitor, so it won't stop in the middle of its execution. Why should one make it atomic? – Pinery

Answer 2

What is the potential damage if it was possible to invoke wait() outside a synchronized block, retaining it's semantics - suspending the caller thread?

Let's illustrate what issues we would run into if wait() could be called outside of a synchronized block with a concrete example.

Suppose we were to implement a blocking queue (I know, there is already one in the API :)

A first attempt (without synchronization) could look something along the lines below

class BlockingQueue {
    Queue<String> buffer = new LinkedList<String>();
    
    public void give(String data) {
        buffer.add(data);
        notify();                   // Since someone may be waiting in take!
    }
    
    public String take() throws InterruptedException {
        while (buffer.isEmpty())    // don't use "if" due to spurious wakeups.
            wait();
        return buffer.remove();
    }
}

This is what could potentially happen:

1. A consumer thread calls take() and sees that the buffer.isEmpty().

2. Before the consumer thread goes on to call wait(), a producer thread comes along and invokes a full give(), that is, buffer.add(data); notify();

3. The consumer thread will now call wait() (and miss the notify() that was just called).

4. If unlucky, the producer thread won't produce more give() as a result of the fact that the consumer thread never wakes up, and we have a dead-lock.

Once you understand the issue, the solution is obvious: Use synchronized to make sure notify is never called between isEmpty and wait.

This synchronization issue turns out to be universal. As Michael Borgwardt points out, wait/notify is all about communication between threads, so without synchronization you'll always end up with a race condition similar to the one described above. This is why the "only wait inside synchronized" rule is enforced.

---

A paragraph from the link posted by @Willie summarizes it quite well:

You need an absolute guarantee that the waiter and the notifier agree about the state of the predicate. The waiter checks the state of the predicate at some point slightly BEFORE it goes to sleep, but it depends for correctness on the predicate being true WHEN it goes to sleep. There's a period of vulnerability between those two events, which can break the program.

The predicate that the producer and consumer need to agree upon is in the above example buffer.isEmpty(). And the agreement is resolved by ensuring that the wait and notify are performed in synchronized blocks.

---

This post has been rewritten as an article here: Java: Why wait must be called in a synchronized block

Deluca

In addition also to make sure the changes made to the condition is seen immediately after the wait() finishes, I guess. Otherwise, also a dead-lock since the notify() has been already called. – Smog

Interesting, but note that just calling synchronized actually won't always solve such problems due to the "unreliable" nature of wait() and notify(). Read more here: #21439855. The reason why synchronized is needed lays within the hardware architecture (see my answer below). – Except

but if add return buffer.remove(); in while block but after wait();, it works? – Lumen

@BobJiang, no, the thread can be woken up for reasons other than someone calling give. In other words, buffer may be empty even after wait returns. – Deluca

I have only Thread.currentThread().wait(); in the main function surrounded by try-catch for InterruptedException. Without synchronized block, it gives me the same exception IllegalMonitorStateException. What makes it reach illegal state now? It works inside synchronized block though. – Chil

@Chil Thread.currentThread().wait(); doesn't make sense. You probably want to add a Object lock = new Object(); field in your class, and do lock.wait(); (or something similar). I've never seen anyone call wait() on a Thread object. – Deluca

It's not on the Thread object bu the java main function. I understand that it doesn't make sense. But why does it give exception? What exactly is going on inside wait? – Chil

synchronized makes sure you own that monitor. You can only call wait on monitors you own. See javadoc for wait: "The current thread must own this object's monitor." – Deluca

@Deluca So it will be best if Java ensures that both the condition+wait/notify is inside the synchronized method/block. But at least it enforces for wait() and notify(), right? Is there any situation where the condition can be put outside synchronized and only wait and notify inside? – Myself

But at least it enforces for wait() and notify(), right? That's right. I can't think of any situation where the condition can be put outside synchronized. That would be a situation where the evaluation of the condition doesn't really matter at the point where wait or notify is called. – Deluca

@Deluca "Always perform give/notify and isEmpty/wait atomically" - it's true that there must be a total order between all notify and checkCondition&wait actions, and it is achieved by executing notify and checkCondition&wait in synchronized blocks. But there is no reason to universally require a total order between setConditionToTrue&notify and checkCondition&wait actions, thus there is no reason to universally require putting setConditionToTrue and notify into a synchronized block. Do you agree? – Forint

The answer chosend as correct theoretically tells you why wait can only be in synchronized but it is this answer that gives you the elaborated illustration on why that is so which helps appreciate Michael Borgwardt's chosen answer. – Johansen

It's worth pointing out that a) the term deadlock is hardly appropriate where the issue is a missed notify() => a lack of progress - OK I guess it's used loosely here; b) this bad scenario that is supposed to illustrate the main reason why the check-wait loop has to be atomic.. This can be easily fixed by putting a timeout on the wait e.g. wait(100) and that scenario is gone. A slight delay consuming the element is all that'll happen. Sure, the thing has to be atomic (for this reason+others), even check-wait-remove must be atomic, but I reckon it's an unconvincing example. – Unlikely

@almondandapricot, a) I wrote "if unlucky this could result in a deadlock." It could very well be the case that missed notify causes no more give to be invoked (as part of some higher level protocol outside of this class). I'm not saying that a missed notify will result in a deadlock. b) Yes, you can solve it by notifying again if the first one happens to be missed. Personally however, I would classify that as a bug. Are you saying that this give/take example is unconvincing? Do you have any more convincing example in mind? – Deluca

but you would have to synchronise on the same lock, right? – Safranine

Answer 3

@Rollerball is right. The wait() is called, so that the thread can wait for some condition to occur when this wait() call happens, the thread is forced to give up its lock.
To give up something, you need to own it first. Thread needs to own the lock first. Hence the need to call it inside a synchronized method/block.

Yes, I do agree with all the above answers regarding the potential damages/inconsistencies if you did not check the condition within synchronized method/block. However as @shrini1000 has pointed out, just calling wait() within synchronized block will not avert this inconsistency from happening.

Here is a nice read..

Preoccupancy

Answer 4

The problem it may cause if you do not synchronize before wait() is as follows:

1. If the 1st thread goes into makeChangeOnX() and checks the while condition, and it is true (x.metCondition() returns false, means x.condition is false) so it will get inside it. Then just before the wait() method, another thread goes to setConditionToTrue() and sets the x.condition to true and notifyAll().
2. Then only after that, the 1st thread will enter his wait() method (not affected by the notifyAll() that happened few moments before). In this case, the 1st thread will stay waiting for another thread to perform setConditionToTrue(), but that might not happen again.

But if you put synchronized before the methods that change the object state, this will not happen.

class A {

    private Object X;

    makeChangeOnX(){
        while (! x.getCondition()){
            wait();
            }
        // Do the change
    }

    setConditionToTrue(){
        x.condition = true; 
        notifyAll();

    }
    setConditionToFalse(){
        x.condition = false;
        notifyAll();
    }
    bool getCondition(){
        return x.condition;
    }
}

Rokach

Answer 5

This basically has to do with the hardware architecture (i.e. RAM and caches).

If you don't use synchronized together with wait() or notify(), another thread could enter the same block instead of waiting for the monitor to enter it. Moreover, when e.g. accessing an array without a synchronized block, another thread may not see the changement to it...actually another thread will not see any changements to it when it already has a copy of the array in the x-level cache (a.k.a. 1st/2nd/3rd-level caches) of the thread handling CPU core.

But synchronized blocks are only one side of the medal: If you actually access an object within a synchronized context from a non-synchronized context, the object still won't be synchronized even within a synchronized block, because it holds an own copy of the object in its cache. I wrote about this issues here: https://stackoverflow.com/a/21462631 and When a lock holds a non-final object, can the object's reference still be changed by another thread?

Furthermore, I'm convinced that the x-level caches are responsible for most non-reproducible runtime errors. That's because the developers usually don't learn the low-level stuff, like how CPU's work or how the memory hierarchy affects the running of applications: http://en.wikipedia.org/wiki/Memory_hierarchy

It remains a riddle why programming classes don't start with memory hierarchy and CPU architecture first. "Hello world" won't help here. ;)

Except

Just discovered an website that explains it perfectly and in-depth: javamex.com/tutorials/… – Except

Hmm.. not sure I follow. If caching was the only reason for putting wait and notify inside synchronized, why isn't the synchronization put inside the implementation of wait / notify? – Deluca

Good question, since wait / notify could very well be synchronized methods...maybe Sun's former Java developers know the answer? Have a look into the link above, or maybe this will also help you: docs.oracle.com/javase/specs/jls/se7/html/jls-17.html – Except

A reason might be: In the early days of Java there were no compile errors when not calling synchronized before doing these multithreading operations. Instead there were only runtime errors (e.g. coderanch.com/t/239491/java-programmer-SCJP/certification/…). Maybe they really thought @SUN that when programmers are getting these errors they're being contacted, which may have given them the opportunity to sell more of their servers. When did it change? Maybe Java 5.0 or 6.0, but actually I don't remember to be honest... – Except

TBH I see a few issues with your answer 1) Your second sentence doesn't make sense: It doesn't matter which object a thread has a lock on. Regardless what object two threads synchronize on, all changes are made visible. 2) You say another thread "will not" see any changes. This should be "may not". 3) I don't know why you're bringing up 1st/2nd/3rd level caches... What matters here is what the Java Memory Model says and that is specified in JLS. While hardware architecture may help in understanding why JLS says what it does, it is strictly speaking irrelevant in this context. – Deluca

Your comments are a bit confusing too. There is still no compile error for calling wait/notify outside a synchronized block. This has always been the case and didn't change in 5.0 or 6.0. (Judging whether this is the case in statically is even undecidable!) The fact that you don't understand why the synchronization can't be done inside wait/notify seems to indicate that you don't understand why wait/notify must only be called when holding the lock in the first place. I think my and Michael Borgwardt answers explain it well enough if you're curious about the underlying reason. – Deluca

I don't know what you mean since I'm getting these compiler errors when calling wait outside a sync block...also I disagree with you about the lock: docs.oracle.com/javase/tutorial/essential/concurrency/… and my links above say that a lock is on the object, therefore synchronizing the object's state with main memory (but not synchronizing all objects between all waiting threads). Please don't say I'm misunderstanding things when I have sources which prove my words while you didn't post any source yet, thanks (And yes, I'm waiting for your links sources...). – Except

And yes, I know you're probably refering to Doug Lea when he wrote (gee.cs.oswego.edu/dl/cpj/jmm.html): "A writing thread releases a synchronization lock and a reading thread subsequently acquires that same synchronization lock. In essence, releasing a lock forces a flush of all writes from working memory employed by the thread, and acquiring a lock forces a (re)load of the values of accessible fields. While lock actions provide exclusion only for the operations performed within a synchronized method or block, these memory effects are defined to cover all fields used by the thread... – Except

...performing the action." Obviously there are some problems with these wordings. Because it's not clear which fields of which object he means: The hole thread's fields or just all objects within the synchronized block? Well, the latter seems to be true, but we can't be sure, as others say: https://mcmap.net/q/101292/-memory-effects-of-synchronization-in-java and https://mcmap.net/q/101293/-does-java-synchronized-keyword-flush-the-cache (note that Doug is just one guy any may be completely wrong, so I wouldn't solely rely on him...and IMHO if somebody doesn't explain something clearly he didn't understand it too well!) – Except

But to answer your points in detail: 1) I wrote "another thread holding the same object MAY not see the changements you're doing" which is correct. 2) I said "will not [...] WHEN"...you must read the hole sentence. 3) What the Java Memory Model says and what is really implemented are two different shoes. See my three comments just above. I also like this article from a guy saying that "flushing" is wrong and that it's just "fencing": mechanical-sympathy.blogspot.ch/2013/02/…. If even Doug Lea is wrong...we can say only one thing for sure: Never be too sure. – Except

What compile errors do you get? Could you please post a snippet of code that gives you a compile error due to having wait or notify outside of a synchronized block? (Either in a comment or on for instance pastebin.com.) – Deluca

Simply an "Invoking Object.wait outside synchronized context"-error, and the code is just wait() or wait(integer) somewhere in a run() method. Btw I'm using NetBeans (IDE 7.2)...which allows me to write code many times faster as in Eclipse and without any nonsense-error messages. ;-) Oh, btw I need to thank you for your remarks, because it kept me thinking about the hole synchronized stuff once more and in more depth, and I must admit that I only partially understood what was really going on (I'm not saying that my views of the JVM were completely wrong, but partially!). Which finally... – Except

...lead to this infamous question: #31052906. On which some people with many k's basically admit that they also don't understand how the JVM really works, like I didn't before. But due to the extreme crazy and inpolite reactions to this question, and because SO doesn't remove it even upon my request (!), I now choosed not to update all my other partially wrong answers. In fact I'll be hibernating my account here on SO, and only if this question is completely removed, I'm thinking... – Except

...of coming back and correct all my answers concerning the synchronized matter. Speaking in terms of transitivity, it was basically your post here that somehow "kicked" me out of this plattform. Hope you can accept that. Good bye, Marcus – Except

The "Invoking Object.wait outside synchronized context" is a NetBeans specific message and not something that is mandated by JLS. As for my remarks, you're welcome. Helping others improves my knoledge too. As I learn new stuff I try to update my posts. I recommend you to do so too. If you learn that your answer is incorrect but don't have time to update it, plase consider deleting it to avoid spreading the confusion. Finally, no one kicked you out, but if you can't stand the heat, get out of the kitchen ;-) – Deluca

That memory hierarchy has anything to do with this is a myth. Google "MESI protocol". – Zina

Answer 6

We all know that wait(), notify() and notifyAll() methods are used for inter-threaded communications. To get rid of missed signal and spurious wake up problems, waiting thread always waits on some conditions. e.g.-

boolean wasNotified = false;
while(!wasNotified) {
    wait();
}

Then notifying thread sets wasNotified variable to true and notify.

Every thread has their local cache so all the changes first get written there and then promoted to main memory gradually.

Had these methods not invoked within synchronized block, the wasNotified variable would not be flushed into main memory and would be there in thread's local cache so the waiting thread will keep waiting for the signal although it was reset by notifying thread.

To fix these types of problems, these methods are always invoked inside synchronized block which assures that when synchronized block starts then everything will be read from main memory and will be flushed into main memory before exiting the synchronized block.

synchronized(monitor) {
    boolean wasNotified = false;
    while(!wasNotified) {
        wait();
    }
}

Thanks, hope it clarifies.

Pansophy

Answer 7

as per docs:

The current thread must own this object's monitor. The thread releases ownership of this monitor.

wait() method simply means it releases the lock on the object. So the object will be locked only within the synchronized block/method. If thread is outside the sync block means it's not locked, if it's not locked then what would you release on the object?

Corrinecorrinne

Answer 8

Thread wait on the monitoring object (object used by synchronization block), There can be n number of monitoring object in whole journey of a single thread. If Thread wait outside the synchronization block then there is no monitoring object and also other thread notify to access for the monitoring object, so how would the thread outside the synchronization block would know that it has been notified. This is also one of the reason that wait(), notify() and notifyAll() are in object class rather than thread class.

Basically the monitoring object is common resource here for all the threads, and monitoring objects can only be available in synchronization block.

class A {
   int a = 0;
  //something......
  public void add() {
   synchronization(this) {
      //this is your monitoring object and thread has to wait to gain lock on **this**
       }
  }

Irredentist

Answer 9

directly from this java oracle tutorial:

When a thread invokes d.wait, it must own the intrinsic lock for d — otherwise an error is thrown. Invoking wait inside a synchronized method is a simple way to acquire the intrinsic lock.

Walliw

From the question the author made, does not seem that the question author has a clear understanding of what I quoted from the tutorial.And moreover, my answer, explains "Why". – Walliw

Answer 10

When you call notify() on an object t, Java notifies a particular t.wait() method. But, how does Java search and notify a particular wait method.

Java only looks into the synchronized block of code which was locked by object t. Java cannot search the whole code to notify a particular t.wait().

Busiek