Large number of dead threads in .Net memory dump
Asked Answered
H

1

4

during the analysis of a memory dump for a .Net4.5 WCF w3wp process, I encountered many threads identified as dead. !threads shows 68 out of 107 threads are dead which appear to be quite high. I was wondering if these threads could hold large amount of memory since the process eventually goes as high as 20GB+ and seem to never go down.

How can I inspect such threads and see the objects/memory held by these? Is it normal to have so many?

0:000> !threads
ThreadCount:      107
UnstartedThread:  0
BackgroundThread: 35
PendingThread:    0
DeadThread:       68
Hosted Runtime:   no

       ID OSID ThreadOBJ           State GC Mode     GC Alloc Context                  Domain           Count Apt Exception
   7    1 16fc 0000009d253a36e0    28220 Preemptive  0000000000000000:0000000000000000 0000009d25385d70 0     Ukn 
  14    2  a64 000000a1702d7560    2b220 Preemptive  0000000000000000:0000000000000000 0000009d25385d70 0     MTA (Finalizer) 
XXXX    3    0 000000a1702f9390  8039820 Preemptive  0000000000000000:0000000000000000 0000009d25385d70 0     MTA (Threadpool Completion Port) 
XXXX    4    0 000000a1702fa270  8038820 Preemptive  0000000000000000:0000000000000000 0000009d25385d70 0     Ukn (Threadpool Completion Port) 
  16    6 21c8 000000a17031f310  102a220 Preemptive  0000000000000000:0000000000000000 0000009d25385d70 0     MTA (Threadpool Worker) 
  17    7 2af4 000000a170327ef0    21220 Preemptive  0000000000000000:0000000000000000 0000009d25385d70 0     Ukn 
  19    9 1b50 000000a1703cccd0  1020220 Preemptive  0000000000000000:0000000000000000 0000009d25385d70 0     Ukn (Threadpool Worker) 
  21   10  85c 000000a170416570  202b020 Preemptive  000000A0945502B8:000000A094550FD0 000000a1703360c0 0     MTA 
  25   11 13cc 000000a1711823f0  202b020 Preemptive  000000A094554D60:000000A094554FD0 000000a1703360c0 0     MTA 
  26   12 2044 000000a1711921d0  3029220 Preemptive  0000000000000000:0000000000000000 000000a1703360c0 0     MTA (Threadpool Worker) 
XXXX   16    0 000000a17128a690  8039820 Preemptive  0000000000000000:0000000000000000 0000009d25385d70 0     MTA (Threadpool Completion Port) 
XXXX   17    0 000000a1712bd610  8039820 Preemptive  0000000000000000:0000000000000000 0000009d25385d70 0     MTA (Threadpool Completion Port) 
XXXX   18    0 000000a1712c5e30  8039820 Preemptive  0000000000000000:0000000000000000 0000009d25385d70 0     MTA (Threadpool Completion Port) 
XXXX   19    0 000000a1712c4e90  8039820 Preemptive  0000000000000000:0000000000000000 0000009d25385d70 0     MTA (Threadpool Completion Port) 
   2   20  8a4 000000a1712c6600    20220 Preemptive  0000009E8B81C238:0000009E8B81DFD0 0000009d25385d70 0     Ukn 
  18   21 28f8 000000a1712c3720    20220 Preemptive  0000000000000000:0000000000000000 0000009d25385d70 0     Ukn 
  22   22  bfc 000000a1712c3ef0    20220 Preemptive  0000000000000000:0000000000000000 0000009d25385d70 0     Ukn 
  20   23 257c 000000a1712c5660    20220 Preemptive  000000A09457AC30:000000A09457AFD0 0000009d25385d70 0     Ukn 
  23   24 13e0 000000a1712c6dd0    20220 Preemptive  0000009F87F0B5C8:0000009F87F0CFD0 0000009d25385d70 0     Ukn 
XXXX   26    0 000000a1713d8fb0  8039820 Preemptive  0000000000000000:0000000000000000 0000009d25385d70 0     Ukn (Threadpool Completion Port) 
  28   27 2aac 000000a1713dbe90  a029220 Preemptive  0000000000000000:0000000000000000 000000a1703360c0 0     MTA (Threadpool Completion Port) 
XXXX   29    0 000000a1713dc660  8039820 Preemptive  0000000000000000:0000000000000000 0000009d25385d70 0     MTA (Threadpool Completion Port) 
  29   30 284c 000000a1713d9f50  202b220 Preemptive  0000000000000000:0000000000000000 000000a1703360c0 0     MTA 
XXXX   31    0 000000a1713da720  8039820 Preemptive  0000000000000000:0000000000000000 0000009d25385d70 0     Ukn (Threadpool Completion Port) 
XXXX   32    0 000000a1713db6c0  8039820 Preemptive  0000000000000000:0000000000000000 0000009d25385d70 0     MTA (Threadpool Completion Port) 
XXXX   33    0 000000a174347600  8039820 Preemptive  0000000000000000:0000000000000000 0000009d25385d70 0     Ukn (Threadpool Completion Port) 
XXXX   34    0 000000a174344720  8039820 Preemptive  0000000000000000:0000000000000000 0000009d25385d70 0     Ukn (Threadpool Completion Port) 
XXXX   35    0 000000a174345e90  8039820 Preemptive  0000000000000000:0000000000000000 0000009d25385d70 0     Ukn (Threadpool Completion Port) 
XXXX   36    0 000000a174346660    39820 Preemptive  0000000000000000:0000000000000000 0000009d25385d70 0     Ukn 
XXXX   37    0 000000a174346e30  8039820 Preemptive  0000000000000000:0000000000000000 0000009d25385d70 0     Ukn (Threadpool Completion Port) 
XXXX   38    0 000000a1743456c0  8039820 Preemptive  0000000000000000:0000000000000000 0000009d25385d70 0     Ukn (Threadpool Completion Port) 
XXXX   39    0 000000a1741b9d10  8039820 Preemptive  0000000000000000:0000000000000000 0000009d25385d70 0     Ukn (Threadpool Completion Port) 
XXXX   40    0 000000a1741bc420  8039820 Preemptive  0000000000000000:0000000000000000 0000009d25385d70 0     Ukn (Threadpool Completion Port) 
XXXX   41    0 000000a1741bcbf0  8039820 Preemptive  0000000000000000:0000000000000000 0000009d25385d70 0     Ukn (Threadpool Completion Port) 
XXXX   42    0 000000a1741ba4e0  8039820 Preemptive  0000000000000000:0000000000000000 0000009d25385d70 0     Ukn (Threadpool Completion Port) 
XXXX   43    0 000000a1741be360  8039820 Preemptive  0000000000000000:0000000000000000 0000009d25385d70 0     Ukn (Threadpool Completion Port) 
   3   44 1e94 000000a1741bd3c0    20220 Preemptive  0000009F87E511F8:0000009F87E52FD0 0000009d25385d70 0     Ukn 
XXXX   45    0 000000a1741bdb90  8039820 Preemptive  0000000000000000:0000000000000000 0000009d25385d70 0     Ukn (Threadpool Completion Port) 
  35   46 12dc 000000a1741bacb0    20220 Preemptive  0000000000000000:0000000000000000 0000009d25385d70 0     MTA 
XXXX   47    0 000000a1741beb30    30820 Preemptive  0000000000000000:0000000000000000 0000009d25385d70 0     Ukn 
XXXX   48    0 000000a1741bf300  8039820 Preemptive  0000000000000000:0000000000000000 0000009d25385d70 0     Ukn (Threadpool Completion Port) 
XXXX   49    0 000000a171171f40  8039820 Preemptive  0000000000000000:0000000000000000 0000009d25385d70 0     Ukn (Threadpool Completion Port) 
  36   50 2bb4 000000a171173e80  202b020 Preemptive  0000000000000000:0000000000000000 000000a1703360c0 0     MTA 
  37   51  9e4 000000a171177530  202b020 Preemptive  000000A0945528D0:000000A094552FD0 000000a1703360c0 0     MTA 
  39   53  6d0 000000a171174e20    21220 Preemptive  0000000000000000:0000000000000000 0000009d25385d70 0     Ukn 
  40   54  f34 000000a171172ee0    21220 Preemptive  0000000000000000:0000000000000000 0000009d25385d70 0     Ukn 
  41   55  f74 000000a1711755f0    21220 Preemptive  0000000000000000:0000000000000000 0000009d25385d70 0     Ukn 
  42   56 2198 000000a171174650    21220 Preemptive  0000000000000000:0000000000000000 0000009d25385d70 0     Ukn 
XXXX   57    0 000000a171175dc0  1039820 Preemptive  0000000000000000:0000000000000000 0000009d25385d70 0     Ukn (Threadpool Worker) 
XXXX   60    0 000000a171176590  1039820 Preemptive  0000000000000000:0000000000000000 0000009d25385d70 0     MTA (Threadpool Worker) 
XXXX   62    0 000000a171177d00  1039820 Preemptive  0000000000000000:0000000000000000 0000009d25385d70 0     MTA (Threadpool Worker) 
XXXX   64    0 000000a171178ca0  1039820 Preemptive  0000000000000000:0000000000000000 0000009d25385d70 0     Ukn (Threadpool Worker) 
XXXX   65    0 000000a1741bfad0  1039820 Preemptive  0000000000000000:0000000000000000 0000009d25385d70 0     Ukn (Threadpool Worker) 
XXXX   70    0 000000a174344ef0  1039820 Preemptive  0000000000000000:0000000000000000 0000009d25385d70 0     Ukn (Threadpool Worker) 
XXXX   71    0 000000a1713d9780  1039820 Preemptive  0000000000000000:0000000000000000 0000009d25385d70 0     Ukn (Threadpool Worker) 
XXXX   69    0 000000a171171770  1039820 Preemptive  0000000000000000:0000000000000000 0000009d25385d70 0     MTA (Threadpool Worker) 
XXXX   68    0 000000a1711736b0  1039820 Preemptive  0000000000000000:0000000000000000 0000009d25385d70 0     MTA (Threadpool Worker) 
XXXX   67    0 000000a171172710  1039820 Preemptive  0000000000000000:0000000000000000 0000009d25385d70 0     MTA (Threadpool Worker) 
XXXX   66    0 000000a171176d60  1039820 Preemptive  0000000000000000:0000000000000000 0000009d25385d70 0     MTA (Threadpool Worker) 
XXXX   59    0 000000a1711784d0  1039820 Preemptive  0000000000000000:0000000000000000 0000009d25385d70 0     MTA (Threadpool Worker) 
XXXX   58    0 000000a1741bbc50  1039820 Preemptive  0000000000000000:0000000000000000 0000009d25385d70 0     MTA (Threadpool Worker) 
XXXX   63    0 000000a1741c1240  1039820 Preemptive  0000000000000000:0000000000000000 0000009d25385d70 0     Ukn (Threadpool Worker) 
XXXX   61    0 000000a1741c02a0  1039820 Preemptive  0000000000000000:0000000000000000 0000009d25385d70 0     Ukn (Threadpool Worker) 
XXXX   28    0 000000a1741c0a70  1039820 Preemptive  0000000000000000:0000000000000000 0000009d25385d70 0     Ukn (Threadpool Worker) 
XXXX   25    0 000000a1712c46c0  1039820 Preemptive  0000000000000000:0000000000000000 0000009d25385d70 0     Ukn (Threadpool Worker) 
XXXX   15    0 000000a1713daef0  1039820 Preemptive  0000000000000000:0000000000000000 0000009d25385d70 0     Ukn (Threadpool Worker) 
XXXX   14    0 000000a174347dd0  1039820 Preemptive  0000000000000000:0000000000000000 0000009d25385d70 0     Ukn (Threadpool Worker) 
XXXX   13    0 000000a16744b400  1039820 Preemptive  0000000000000000:0000000000000000 0000009d25385d70 0     Ukn (Threadpool Worker) 
XXXX   52    0 000000a167448520  1039820 Preemptive  0000000000000000:0000000000000000 0000009d25385d70 0     Ukn (Threadpool Worker) 
XXXX    8    0 000000a16744bbd0  1039820 Preemptive  0000000000000000:0000000000000000 0000009d25385d70 0     Ukn (Threadpool Worker) 
XXXX   72    0 000000a16744ac30  8039820 Preemptive  0000000000000000:0000000000000000 0000009d25385d70 0     MTA (Threadpool Completion Port) 
XXXX   73    0 000000a16744a460  1039820 Preemptive  0000000000000000:0000000000000000 0000009d25385d70 0     MTA (Threadpool Worker) 
XXXX   74    0 000000a171268f50  1039820 Preemptive  0000000000000000:0000000000000000 0000009d25385d70 0     MTA (Threadpool Worker) 
XXXX   75    0 000000a1712658a0  1039820 Preemptive  0000000000000000:0000000000000000 0000009d25385d70 0     MTA (Threadpool Worker) 
XXXX   76    0 000000a171269720  1039820 Preemptive  0000000000000000:0000000000000000 0000009d25385d70 0     MTA (Threadpool Worker) 
XXXX   77    0 000000a171266070  1039820 Preemptive  0000000000000000:0000000000000000 0000009d25385d70 0     MTA (Threadpool Worker) 
XXXX   78    0 000000a1712677e0  1039820 Preemptive  0000000000000000:0000000000000000 0000009d25385d70 0     MTA (Threadpool Worker) 
XXXX   79    0 000000a171269ef0  1039820 Preemptive  0000000000000000:0000000000000000 0000009d25385d70 0     MTA (Threadpool Worker) 
XXXX   80    0 000000a171266840  1039820 Preemptive  0000000000000000:0000000000000000 0000009d25385d70 0     MTA (Threadpool Worker) 
XXXX   81    0 000000a17126a6c0  1039820 Preemptive  0000000000000000:0000000000000000 0000009d25385d70 0     MTA (Threadpool Worker) 
XXXX   82    0 000000a171267010  1039820 Preemptive  0000000000000000:0000000000000000 0000009d25385d70 0     MTA (Threadpool Worker) 
XXXX   83    0 000000a17126ae90  8039820 Preemptive  0000000000000000:0000000000000000 0000009d25385d70 0     Ukn (Threadpool Completion Port) 
XXXX    5    0 000000a171268780  8039820 Preemptive  0000000000000000:0000000000000000 0000009d25385d70 0     MTA (Threadpool Completion Port) 
  43   84  dcc 000000a17126b660  8029220 Preemptive  0000009D9D1B3B88:0000009D9D1B3FD0 0000009d25385d70 0     MTA (Threadpool Completion Port) 
XXXX   85    0 000000a171267fb0  1039820 Preemptive  0000000000000000:0000000000000000 0000009d25385d70 0     MTA (Threadpool Worker) 
XXXX   86    0 000000a17126be30  1039820 Preemptive  0000000000000000:0000000000000000 0000009d25385d70 0     MTA (Threadpool Worker) 
  46   87 1e54 000000a17126c600  1029220 Preemptive  000000A094575068:000000A094576FD0 0000009d25385d70 0     MTA (Threadpool Worker) 
XXXX   88    0 000000a17126cdd0  1039820 Preemptive  0000000000000000:0000000000000000 0000009d25385d70 0     MTA (Threadpool Worker) 
  45   89 1db8 000000a16744c3a0  1029220 Preemptive  000000A094577250:000000A094578FD0 0000009d25385d70 0     MTA (Threadpool Worker) 
XXXX   90    0 000000a167448cf0  1039820 Preemptive  0000000000000000:0000000000000000 0000009d25385d70 0     Ukn (Threadpool Worker) 
XXXX   91    0 000000a16744cb70  1039820 Preemptive  0000000000000000:0000000000000000 0000009d25385d70 0     MTA (Threadpool Worker) 
XXXX   92    0 000000a1674494c0  1039820 Preemptive  0000000000000000:0000000000000000 0000009d25385d70 0     MTA (Threadpool Worker) 
XXXX   93    0 000000a16744d340  1039820 Preemptive  0000000000000000:0000000000000000 0000009d25385d70 0     Ukn (Threadpool Worker) 
  50   94 15a4 000000a16744db10  1029220 Preemptive  000000A09456AF80:000000A09456AFD0 0000009d25385d70 0     MTA (Threadpool Worker) 
  47   95 29c8 000000a167449c90  1029220 Preemptive  000000A094573D08:000000A094574FD0 0000009d25385d70 0     MTA (Threadpool Worker) 
  48   96 28c4 000000a16744e2e0  1029220 Preemptive  000000A094548ED8:000000A094548FD0 0000009d25385d70 0     MTA (Threadpool Worker) 
  49   97  69c 000000a16744eab0  1029220 Preemptive  0000009D9D1863F0:0000009D9D187FD0 0000009d25385d70 0     MTA (Threadpool Worker) 
XXXX   98    0 000000a16744fa50  8039820 Preemptive  0000000000000000:0000000000000000 0000009d25385d70 0     Ukn (Threadpool Completion Port) 
  51   99 2bac 000000a16744f280  8029220 Preemptive  0000009F87F32660:0000009F87F32FD0 0000009d25385d70 0     MTA (Threadpool Completion Port) 
  52  101  c40 000000a174599040  1029220 Preemptive  0000009D9D178538:0000009D9D179FD0 0000009d25385d70 0     MTA (Threadpool Worker) 
  54  102 1e5c 000000a174598870  1029220 Preemptive  0000009F87F51578:0000009F87F52FD0 0000009d25385d70 0     MTA (Threadpool Worker) 
  56  103 2b68 000000a174596930  1029220 Preemptive  0000009D9D188E70:0000009D9D189FD0 0000009d25385d70 0     MTA (Threadpool Worker) 
  55  104 2924 000000a174595990  1029220 Preemptive  0000009D9D18C290:0000009D9D18DFD0 0000009d25385d70 0     MTA (Threadpool Worker) 
  53  105  2f0 000000a174599810  1029220 Preemptive  0000009E8B89EFD0:0000009E8B89FFD0 0000009d25385d70 0     MTA (Threadpool Worker) 
  57  106  f5c 000000a174596160  1029220 Preemptive  0000009E8B894828:0000009E8B895FD0 0000009d25385d70 0     MTA (Threadpool Worker) 
  58  107  20c 000000a174599fe0  1029220 Preemptive  0000009F87F53258:0000009F87F54FD0 0000009d25385d70 0     MTA (Threadpool Worker) 
  60  100 1f60 000000a17459a7b0  8029220 Preemptive  0000009F87F7B1A8:0000009F87F7CFD0 0000009d25385d70 0     MTA (Threadpool Completion Port) 
Hautesalpes answered 20/11, 2019 at 2:48 Comment(4)
You cannot assume anything was normal by the time you decided to make the minidump. A dead thread cannot own any memory allocations, thread objects not getting cleaned-up because something is a-miss with memory management is a sensible conclusion. Maybe you shouldn't wait too long before making the minidump.Enate
Each managed thread is backed by a native thread and a System.Thread object. The "dead thread" in your case really indicates that the native thread no longer exists, but the once paired System.Thread object still exists. Likely objects created on that thread are still being referenced and hence cannot be GCed. You can dump the roots for the System.Thread objects to trace the references. You should also check the finalization queue to see if it exceedingly large or blocked.Retouch
@Dono, how do I get to the Thread object using ThreadObj since I don't have a thread number because of the XXX.Hautesalpes
@Hautesalpes You can't. Just dump all System.Threading.Thread objects from the managed heap. You'll have to filter through the results.Retouch
S
3

I was wondering if these threads could hold large amount of memory

Remember the following rule: a process provides memory, a thread consumes CPU time. The inverse is also true: a process does not run and a thread does not hold memory. If someone says "my process still runs", that's a simplification of the sentence "my process has at least one thread that still runs".

A dead thread (marked with XXXX) means that there is a .NET Thread object in memory and the "real" thread (the kernel object maintained by the operating system) is gone.

The following is an MCVE for that situation:

using System;
using System.Collections.Generic;
using System.Threading;

namespace DeadThreadExample
{
    class Program
    {
        static List<Thread> AllThreadsIEverStarted = new List<Thread>();
        static void Main()
        {
            for(int i=0; i<1000; i++)
            {
                Thread t = new Thread(DoNothing);
                t.Start();
                AllThreadsIEverStarted.Add(t);
                t.Join();
            }
            Console.WriteLine("There should be 1000 dead threads now. Debug it with WinDbg and SOS !threads");
            Console.ReadLine();
        }

        private static void DoNothing()
        {
            // Just nothing
        }
    }
}

The debugging session is:

0:006> !threads
PDB symbol for clr.dll not loaded
ThreadCount:      1002
UnstartedThread:  0
BackgroundThread: 1
PendingThread:    0
DeadThread:       1000
Hosted Runtime:   no
[...]

could hold large amount of memory

0:006> !dumpheap -stat
Statistics:
      MT    Count    TotalSize Class Name
[...]
53dde9b0     1000        20000 System.Threading.ThreadHelper
53d66bf0     1000        44000 System.Threading.ExecutionContext
53d62e10     1001        52052 System.Threading.Thread
53dad5cc     2000        64000 System.Threading.ThreadStart

So, yes, there is a "memory leak", if you call the static collection a leak. Maybe it's not a leak, because you need that information at some point in time. Once the collection is cleared, it's no longer a leak.

1000 dead threads is equivalent to ~180 kB "memory leak". I wouldn't call that "large amount". Even if you pass an object as an argument (use ParameterizedThreadStart), it seems that the property m_ThreadStartArg of the Thread object is not set, so I can hardly see how a larger amount of memory would be leaked.

If you don't like that situation, use a memory profiler and find out which GC roots still has a reference to those threads.

Is it normal to have so many?

Maybe you were just unlucky. They might all be gone with the next garbage collection.

How can I inspect such threads and see the objects/memory held by these?

Use !dumpheap -stat -type, then dumpheap -mt and then !do:

0:006> !dumpheap -stat -type Thread
Statistics:
      MT    Count    TotalSize Class Name
[...]
53d62e10     1001        52052 System.Threading.Thread

0:006> !dumpheap -mt 53d62e10  
 Address       MT     Size
02ec247c 53d62e10       52     
02ec2504 53d62e10       52     
[...]  

Statistics:
      MT    Count    TotalSize Class Name
53d62e10     1001        52052 System.Threading.Thread
Total 1001 objects

0:006> !do 02ec247c
Name:        System.Threading.Thread
MethodTable: 53d62e10
EEClass:     53e679a4
Size:        52(0x34) bytes
File:        C:\WINDOWS\Microsoft.Net\assembly\GAC_32\mscorlib\v4.0_4.0.0.0__b77a5c561934e089\mscorlib.dll
Fields:
      MT    Field   Offset                 Type VT     Attr    Value Name
53d6cd68  400192d        4 ....Contexts.Context  0 instance 00000000 m_Context
53d66bf0  400192e        8 ....ExecutionContext  0 instance 00000000 m_ExecutionContext
53d624e4  400192f        c        System.String  0 instance 00000000 m_Name
53d63c70  4001930       10      System.Delegate  0 instance 00000000 m_Delegate
53d65074  4001931       14 ...ation.CultureInfo  0 instance 00000000 m_CurrentCulture
53d65074  4001932       18 ...ation.CultureInfo  0 instance 00000000 m_CurrentUICulture
53d62734  4001933       1c        System.Object  0 instance 00000000 m_ThreadStartArg
53d67b18  4001934       20        System.IntPtr  1 instance  11519f8 DONT_USE_InternalThread
53d642a8  4001935       24         System.Int32  1 instance        2 m_Priority
53d642a8  4001936       28         System.Int32  1 instance        3 m_ManagedThreadId
53d6878c  4001937       2c       System.Boolean  1 instance        0 m_ExecutionContextBelongsToOuterScope
[ ... static ... ]
Salman answered 20/11, 2019 at 21:20 Comment(9)
Thomas, perhaps I did not formulate my question correctly. When I am talking about the amount of space taken by a thread then I don't solely talk about the Thread object itself but also every references to other objects it may keep around and preventing garbage collector from cleaning them. If I have a dead Thread object holding a large DataTable object (think 100K++ rows) then the memory will never be reclaimed.Hautesalpes
When performing dumpheap on Thread then I have no idea which one is dead or not. When doing !threads I do see which one are dead but how can I consult it using ThreadObj?Hautesalpes
@Pascal: the only way I currently see for a Thread object holding additional memory is the member m_ThreadStartArg. In my tests, I was not able to see that property set. IMHO it's more likely that the DataTables are leaked some other way. Use a .NET memory profiler like JetBrains dotTrace. It will tell you, where the objects are connected to.Salman
@Pascal: I guess the only way would be through the DONT_USE_InternalThread property. But I'm not aware who to interpret that IntPtr to get back to the OS Thread.Salman
@Pascal: also, the output of !threads and the property m_ManagedThreadId should correlate.Salman
@ThomasWeller what CLI tool are you using ?Torino
@eranotzap: what do you mean by CLI tool? The output in the answer is from WinDbg, a free Debugger by MicrosoftSalman
@ThomasWeller you'r showing commands like '!dumpheap' what tool are you using ?Torino
@eranotzap: That's WinDbg (UI) or CDB (command line). The commands are equivalent in these two debuggers. It needs the SOS extension.Salman

© 2022 - 2024 — McMap. All rights reserved.