High memory usage for a .Net application with lots of handles

I'm doing postmortem analysis on a high memory pressure .net application using WinDbg with a process dump and this process is a Windows service.

I got a feeling that most of this 14GB of this process memory consumption comes from aborted threads thus lots of orphan semaphore/events/mutant etc. But I wasn't able to put all this together and add them up like how much memory it takes for a single semaphore/events/mutant and what kind of WinDbg commands that will be helpful for this kind of situation?

The following is the WinDbg output:

!handle

**Type                      Count**

None                        90
Event                       5550
Section                     41
File                        1166
Directory                   3
Mutant                      160
Semaphore                   4581
Key                         78
Token                       2
Thread                      553
IoCompletion                6
Timer                       1
TpWorkerFactory             3
ALPC Port                   9
WaitCompletionPacket        33

!address -summary

--- Usage Summary ---------------- RgnCount ----------- Total Size -------- %ofBusy %ofTotal

<unknown>                               471        3`86ea2000 (  14.108 Gb)  92.40%    0.01%

!threads(lots of thread listed have ThreadAbortException exceptions)

Lock  
   ID OSID ThreadOBJ           State GC Mode     GC Alloc Context                  Domain           Count Apt Exception

  12    3 33f0 00000017e3c23200 1282b221 Preemptive  0000000000000000:0000000000000000 00000017e3bb3930 1     MTA System.Threading.ThreadAbortException 000000181de5d668

Answers


Handles and their wrappers are likely quite small. Use dt -r to get an idea of their size:

0:025> dt -r ntdll!_KSEMAPHORE
   +0x000 Header           : _DISPATCHER_HEADER
      +0x000 Type             : UChar
      +0x001 TimerControlFlags : UChar
      +0x001 Absolute         : Pos 0, 1 Bit
      +0x001 Coalescable      : Pos 1, 1 Bit
      +0x001 KeepShifting     : Pos 2, 1 Bit
      +0x001 EncodedTolerableDelay : Pos 3, 5 Bits
      +0x001 Abandoned        : UChar
      +0x001 Signalling       : UChar
      +0x002 ThreadControlFlags : UChar
      +0x002 CpuThrottled     : Pos 0, 1 Bit
      +0x002 CycleProfiling   : Pos 1, 1 Bit
      +0x002 CounterProfiling : Pos 2, 1 Bit
      +0x002 Reserved         : Pos 3, 5 Bits
      +0x002 Hand             : UChar
      +0x002 Size             : UChar
      +0x003 TimerMiscFlags   : UChar
      +0x003 Index            : Pos 0, 6 Bits
      +0x003 Inserted         : Pos 6, 1 Bit
      +0x003 Expired          : Pos 7, 1 Bit
      +0x003 DebugActive      : UChar
      +0x003 ActiveDR7        : Pos 0, 1 Bit
      +0x003 Instrumented     : Pos 1, 1 Bit
      +0x003 Reserved2        : Pos 2, 4 Bits
      +0x003 UmsScheduled     : Pos 6, 1 Bit
      +0x003 UmsPrimary       : Pos 7, 1 Bit
      +0x003 DpcActive        : UChar
      +0x000 Lock             : Int4B
      +0x004 SignalState      : Int4B
      +0x008 WaitListHead     : _LIST_ENTRY
         +0x000 Flink            : Ptr64 _LIST_ENTRY
         +0x008 Blink            : Ptr64 _LIST_ENTRY
   +0x018 Limit            : Int4B

So that Semaphore is 0x18+4 bytes in size. Be aware that the size might increase if the Ptr parts are used, e.g. if you have a very large wait list.

I recommend a !dumpheap -stat in your situation. This will output a list of .NET objects, sorted by their total size.

Typically there are byte[], object[] (or similar) and String which eat most memory.

0:025> !dumpheap -mt 000007feef0aea80
         Address               MT     Size
[...]
000007feef0bda88     4915       329862 System.String
000007feef0be100     1419       382288 System.Object[]
000007feef05f748        2       786520 System.UInt32[]
Total 65575 objects

For a single object, you can use !objsize <address>. A System.Threading.Thread in my example had 16 kB. So even if you have 1000 threads, this would just make up for 16 MB of memory. An example of a System.Threading.ManualResetEvent accounts for only 80 bytes in my application.

0:025> !objsize 0000000011878358 
sizeof(0000000011878358) = 16736 (0x4160) bytes (System.Threading.Thread)

0:025> !objsize 00000000118af810 
sizeof(00000000118af810) = 80 (0x50) bytes (System.Threading.ManualResetEvent)

Need Your Help

Do multi-core CPUs share the MMU and page tables?

multithreading operating-system hardware multicore mmu

On a single core computer, one thread is executing at a time. On each context switch the scheduler checks if the new thread to schedule is in the same process than the previous one. If so, nothing ...

How to add <script> tag using ReactJS?

javascript reactjs

I need to declare a javascript function as described below: