Anyone encountered a INT 3 debugger break in mscorwks.dll?

We host the .NET runtime as part of a Win32 program, and lately it has begun to consistently break at a specific address, in mscorwks.dll.

At the specified address, there is a 0xCC byte, which is a INT 3 instruction, which fires the debugger.

Has anyone else seen this?

I can't see enough information in the dll to know specifically where it is, function or source-wise, but it is definitely not in any of our libraries.

The call-stack looks like this (this is from Delphi 2007):

:7a04f02a ; C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\mscorwks.dll
:7a052fc6 ; C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\mscorwks.dll
:7a053e72 ; C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\mscorwks.dll
:7a03b970 ; C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\mscorwks.dll
:7a00351e ; C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\mscorwks.dll
:7a0255e0 ; C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\mscorwks.dll
:79e71e6d ; C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\mscorwks.dll
:7e42b372 USER32.MoveWindow + 0xd4
:7e4565b7 USER32.GetRawInputDeviceInfoW + 0x5f
:7e42ce7c USER32.SetLayeredWindowAttributes + 0x6a

The address it fails on is 0x7A04F029, disassembly in memory looks like this:

7A04F020 call $7a14be35
7A04F025 cmp [esi],ebx
7A04F027 jz $7a04f02a
7A04F029 int 3                      <-- here's the culprit
7A04F02A mov byte ptr [ebp-$04],$02
7A04F02E lea ecx,[ebp-$00000224]
7A04F034 call $79e82214

It's easy enough to get rid of, just patch that byte in memory with a 0x90 (NOP) each time, but the next time we fire the debugger, the dll is of course reloaded.

I've got half a mind to try to patch the file itself, but I'm not sure that's a good idea either. I've verified that the CC byte is part of the dll, found the surrounding byte pattern with a hex editor.

Any tips on how to fix this? Anyone else seen this?

Solved

As mentioned in the answers, this was a LoaderLock problem. A DLL that was built in Win32 needed to expose a couple of functions for our .NET system, in a way not tied through the way we host the .NET runtime. The project included a couple of units that contained the functions that was going to be exposed, but unfortunately also added lots and lots of code that was useful in an application, but not for this DLL.

Separating out the functions I needed reduced the size of the DLL from 7MB to around 100KB and got rid of the LoaderLock as well.

Answers


There's a discussion about a similar problem here. The most interesting part is this:

Mscorwks.dll is just the messenger. It is telling you that it detected a loaderlock. It is your code that caused it.


Need Your Help

Heap Sort doesn't work properly

c sorting heapsort

I'm writing sorting application in C for my univesity, and I have a problem with one algorithm (heap sort). Here is my code:

Facebook wall post from an App via Graph API

facebook-graph-api facebook-wall

I have a Timeline App that creates a wall posting via the Graph API. The wall posting indicates that the posting was created by the App, which is fine. However, the link for the app points to the