Analyzing .NET Application Crashes with WinDbg: Memory Corruption and Bit Flip Issues

Initial Crash Analysis

Loading the dump file in WinDbg immediate reveals the critical exception information:


This dump file has an exception of interest stored in it.
The stored exception information can be accessed via .ecxr.
(bf8.5dc4): Access violation - code c0000005 (first/second chance not available)
For analysis of this file, run !analyze -v
clr!WKS::gc_heap::mark_object_simple1+0x220:
00007ffb`380453c4 833a00          cmp     dword ptr [rdx],0 ds:00007ffa`35451300=????????

The presence of the mark_object_simple1 method in the stack trace suggests this is likely a managed heap corruption issue, as this function is used by the garbage collector for object marking. Let's verify the heap integrity:


0:083> !verifyheap
object 00000218e96963d8: bad member 00000218E9696450 at 00000218E9696420
Last good object: 00000218E96963C0.
Could not request method table data for object 00000218E9696450 (MethodTable: 00007FFA35451300).
Last good object: 00000218E96963D8.

The heap verification confirms our suspicion - there's an object with an invalid method table (MT), which directly caused the crash.

Investigating the Corrupted Object

Let's examine the corrupted object at address 00000218e96963d8 using both !do and dp commands:


0:083> !do 00000218e96963d8
Name:        System.Threading.Tasks.Task+DelayPromise
MethodTable: 00007ffb3542b3e8
EEClass:     00007ffb3567c7c0
Size:        120(0x78) bytes
File:        C:\Windows\Microsoft.Net\assembly\GAC_64\mscorlib\v4.0_4.0.0.0__b77a5c561934e089\mscorlib.dll
Fields:
...
00007ffb35451300  40035d5       48 ...m.Threading.Timer  0 instance 00000218e9696450 Timer

0:083> dp 00000218e9696450 L6
00000218`e9696450  00007ffa`35451301 00000000`00000000
00000218`e9696460  00000218`e96964c8 00000000`00000000
00000218`e9696470  00007ffb`353e4b51 00000218`e9696368

Notice the discrepancy in the method table addresses for object 00000218e9696450: one shows 00007ffb35451300 while the memory dump shows 00007ffa35451301. Let's validate these method tables:


0:083> !dumpmt 00007ffb35451300
EEClass:         00007ffb356942f0
Module:          00007ffb353b1000
Name:            System.Threading.Timer
mdToken:         0000000002000504
File:            C:\Windows\Microsoft.Net\assembly\GAC_64\mscorlib\v4.0_4.0.0.0__b77a5c561934e089\mscorlib.dll
BaseSize:        0x20
ComponentSize:   0x0
Slots in VTable: 23
Number of IFaces in IFaceMap: 1

0:083> !dumpmt 00007ffa35451301
00007ffa35451301 is not a MethodTable

The first adress is a valid System.Threading.Timer method table, while the second is invalid. The addresses are very similar, suggesting a potential bit flip issue. Converting to binary reveals two bit differences:

Bit 0: Different between addresses
Bit 32: Different between addresses

Analyzing the Bit Flips

Bit 0 Flip Analysis

The flip in bit 0 is actually expected behavior. In the .NET garbage collection process, bit 0 of the method table is used as a mark flag to indicate that an object has been processed during the mark phase. This prevents duplicate processing in the depth-first traversal. The coreclr source code demonstrates this:


inline BOOL gc_heap::mark_object(uint8_t* obj_ptr, uint8_t* low_bound, uint8_t* high_bound, int target_gen)
{
    if ((obj_ptr >= low_bound) && (obj_ptr < high_bound))
    {
        BOOL already_marked = is_marked(obj_ptr);
        if (already_marked)
        {
            return FALSE;
        }
        set_marked_flag(obj_ptr);
        
        return TRUE;
    }
}

#define is_marked(obj_ptr) get_header(obj_ptr)->IsMarked()

BOOL IsMarked() const
{
    return !!(((size_t)GetMethodTable()) & GC_MARKED_FLAG);
}

This explains why bit 0 is set to 1 - it's a normal part of the garbage collection marking process.

Bit 32 Flip Analysis

The bit 32 flip is abnormal and problematic. This corruption caused the garbage collecter to view the object as invalid. In my experience analyzing memory dumps, this is the second time I've encountered this specific pattern of corruption.

When discussing the environment with the client, they mentioned the presence of servo motors in the vicinity. Industrial equipment like servo motors can generate electromagnetic interference that might cause memory corruption. Research confirms that certain industrial machinery can emit electromagnetic radiation capable of affecting computer memory.

Freaks City