How do I find the lockholder (reader) of my ReaderWriterLock in windbg

Question

I've got a dump of a .Net process that has hung due to a deadlock (the gui thread is no longer responding, and my logs show that some threads have stopped responding). I have taken a snapshot and am now looking through it in windbg, and all threads bar one are waiting for the last one. Looking at that one thread's stacktrace with !clrstack -p I can see that it is trying to aquire a write on a ReaderWriterLock

How do I tell which other thread holds that lock so I can start figuring out how the deadlock happened?

thanks

[edit] apparently there was a command !rwlocks in the .Net1.1 sos.dll to help with this, but it isn't there in the .Net2.0 version. The hunt continues

Kimberly

Answer 1

So far the best approach is to look at the !dso for all thread stacks, and see which ones reference the lock. A quick check after that has alles us to track down which threads hold locks. Really not a pretty or quick way though...

Kimberly

Answer 2

I'm not absolutely sure but you might be able to use !SyncBlk to look at the sync block objects, if you invoke it without any arguments I think you should see the sync blocks that are owned by a thread.

If you have a sync block deadlock, the extension SOSEX might be what you need. This extension offers the command !dlk that shows which threads are waiting for which locks. This only works for sync blocks though, deadlocks on other sync objects will not be detected, if you are using lock() (Monitor.Enter) this should not be a problem for you.

Blend

we are using th builtin ReaderWriterLocks which don't show up on a !dlk – Kimberly

Answer 3

Try sosex and !dlk

Goodsell

!dlk only works if you have sync blocks (like lock(myObject.SyncRoot){} or Monitors), not with ReaderWriterLocks – Kimberly

Answer 4

I posted a simular topic a while back here, Using C# is it possible to test if a lock is held on a file

I referenced a number of articles and such, but wait chain traversial (WCT) can help you, it's somewhat touchy but this msdn mag bugslayer article show's how to use WCT in windbg in a managed context.

Aerosphere

nice article, pity it requires us to be running Vista but one day we'll get there... Strange that even there there will be no support for WaitForMultipleObjects though – Kimberly

Answer 5

An approach that would provide traceability is to wrap your locks into an IDisposable interface and replace:

lock( mylock) { ... }

with

using( new DisposeableLock() ) { ... }

You can log the constructor and Dispose() methods either to the console, or log4net, or some other mechanism. This will allow you to see what is being lock and what is blocking on what.

Cepheus

we have very many locks in our app, so two lines of logging for each thread would very quickly add up. The issue also seems to happen only oonce per two weeks across ten users, and has never happened in the developpment environment, so it is quite an expensive approach. But if nothing else works... – Kimberly

An IDisposable lock will likely degrade performance since each "DisposableLock" will have to go through a finalization round of the GC followed by the actual collection round of the GC = sometimes leading to an out-of-memory exception due to allocating memory faster than it reaches the GC queue. – Goldschmidt