NetInverse Developers Blog

SOS Command: !DumpDomain [<Domain address>]

When called with no parameters, !DumpDomain will list all the AppDomains in the process. It enumerates each Assembly loaded into those AppDomains as well.

In addition to your application domain, and any domains it might create, there are two special domains: the Shared Domain and the System Domain.

Any Assembly pointer in the output can be passed to !DumpAssembly. Any Module pointer in the output can be passed to !DumpModule. Any AppDomain pointer can be passed to !DumpDomain to limit output only to that AppDomain. Other functions provide an AppDomain pointer as well, such as !Threads where it lists the current AppDomain for each thread.

!dumpdomain
--------------------------------------
System Domain: 7a3bd058
LowFrequencyHeap: 7a3bd07c
HighFrequencyHeap: 7a3bd0c8
StubHeap: 7a3bd114
Stage: OPEN
Name: None
--------------------------------------
Shared Domain: 7a3bc9a8
LowFrequencyHeap: 7a3bc9cc
HighFrequencyHeap: 7a3bca18
StubHeap: 7a3bca64
Stage: OPEN
Name: None
Assembly: 001a58e8
--------------------------------------
Domain 1: 0015d2e0
LowFrequencyHeap: 0015d304
HighFrequencyHeap: 0015d350
StubHeap: 0015d39c
Stage: OPEN
SecurityDescriptor: 0015e608
Name: FindProduct.exe
Assembly: 001a58e8 [C:WINDOWSassemblyGAC_32mscorlib2.0.0.0__b77a5c561934e089mscorlib.dll]
ClassLoader: 001a5968
SecurityDescriptor: 001a4720
  Module Name
790c1000 C:WINDOWSassemblyGAC_32mscorlib2.0.0.0__b77a5c561934e089mscorlib.dll

Assembly: 001b0788 [C:svnNetInverseFindProductbinDebugFindProduct.exe]
ClassLoader: 001b0808
SecurityDescriptor: 001b0688
  Module Name
00932c5c C:svnNetInverseFindProductbinDebugFindProduct.exe

System Domain

The SystemDomain is responsible for creating and initializing the SharedDomain and the default AppDomain. It loads the system library mscorlib.dll into SharedDomain. It also keeps process-wide string literals interned implicitly or explicitly. String interning is an optimization feature that saves memory by having only a single instance of the string for a given literal across all the application domains. SystemDomain is also responsible for generating process-wide interface IDs, which are used in creating InterfaceVtableMaps in each AppDomain. SystemDomain keeps track of all the domains in the process and implements functionality for loading and unloading the AppDomains.

SharedDomain

All of the domain-neutral code is loaded into SharedDomain. Mscorlib, the system library, is needed by the user code in all the AppDomains. It is automatically loaded into SharedDomain. Fundamental types from the System namespace like Object, ValueType, Array, Enum, String, and Delegate get preloaded into this domain during the CLR bootstrapping process. User code can also be loaded into this domain, using LoaderOptimization attributes specified by the CLR hosting app while calling CorBindToRuntimeEx. Console programs can load code into SharedDomain by annotating the app’s Main method with a System.LoaderOptimizationAttribute. SharedDomain also manages an assembly map indexed by the base address, which acts as a lookup table for managing shared dependencies of assemblies being loaded into DefaultDomain and of other AppDomains created in managed code. DefaultDomain is where non-shared user code is loaded.

DefaultDomain

DefaultDomain is an instance of AppDomain within which application code is typically executed. While some applications require additional AppDomains to be created at runtime (such as apps that have plug-in architectures or apps doing a significant amount of run-time code generation), most applications create one domain during their lifetime. All code that executes in this domain is context-bound at the domain level. If an application has multiple AppDomains, any cross-domain access will occur through .NET Remoting proxies. Additional intra-domain context boundaries can be created using types inherited from System.ContextBoundObject. Each AppDomain has its own SecurityDescriptor, SecurityContext, and DefaultContext, as well as its own loader heaps (High-Frequency Heap, Low-Frequency Heap, and Stub Heap), Handle Tables (Handle Table, Large Object Heap Handle Table), Interface Vtable Map Manager, and Assembly Cache.

LoadHeaps

Frequently accessed artifacts like MethodTables, MethodDescs, FieldDescs, and Interface Maps get allocated on a HighFrequencyHeap, while less frequently accessed data structures, such as EEClass and ClassLoader and its lookup tables, get allocated on a LowFrequencyHeap. The StubHeap hosts stubs that facilitate code access security (CAS), COM wrapper calls, and P/I

SOS Comand: !TraverseHeap

!TraverseHeap [-xml] <filename>

!TraverseHeap writes out a file in a format understood by the CLR Profiler.
You can download the CLR Profiler from this link:

http://www.microsoft.com/downloads/details.aspx?FamilyId=86CE6052-D7F4-4AEB-9B7A-94635BEEBDDA&displaylang=en

It creates a graphical display of the GC heap to help you analyze the state of your application. If you pass the “-xml” flag, the file is instead written out in an easy-to-understand xml format:

    <gcheap>
        <types>
            <type id="1" name="System.String">
            ...
        </types>
        <roots>
            <root kind="handle" address="0x00a73ff0"/>
            <root kind="stack" address="0x0069f0e0"/>
            ...
        </roots>
        <objects>
            <object address="0x00b73030" typeid="1" size="300"/>
            <object address="0x00b75054" typeid="5" size="20">
                <member address="0x00b75088" />
                ...
            </object>
            ...
        </objects>
    </gcheap>

You can break into your process, load SOS, take a snapshot of your heap with this function, then continue.

!PrintException [-nested] [<Exception object address>]

This will format fields of any object derived from System.Exception. One of the more useful aspects is that it will format the _stackTrace field, which is a binary array. If _stackTraceString field is not filled in, that can be helpful for debugging. You can of course use !DumpObj on the same exception object to explore more fields.

If called with no parameters, PrintException will look for the last outstanding exception on the current thread and print it. This will be the same exception that shows up in a run of !Threads.

!PrintException will notify you if there are any nested exceptions on the current managed thread. (A nested exception occurs when you throw another exception within a catch handler already being called for another exception).

If there are nested exceptions, you can re-run !PrintException with the “-nested” option to get full details on the nested exception objects. The !Threads command will also tell you which threads have nested exceptions. The abbreviation !pe can be used for brevity.

SOS Command: !FinalizeQueue

!FinalizeQueue [-detail]

This command lists the objects registered for finalization. Here is output from a simple program:

0:000> !finalizequeue

SyncBlocks to be cleaned up: 0
MTA Interfaces to be released: 0
STA Interfaces to be released: 1

generation 0 has 4 finalizable objects (0015bc90->0015bca0)
generation 1 has 0 finalizable objects (0015bc90->0015bc90)
generation 2 has 0 finalizable objects (0015bc90->0015bc90)

Ready for finalization 0 objects (0015bca0->0015bca0)

Statistics:

      MT    Count TotalSize Class Name
5ba6cf78        1        24 Microsoft.Win32.SafeHandles.SafeFileHandle
5ba5db04        1        68 System.Threading.Thread
5ba73e28        2       112 System.IO.StreamWriter
Total 4 objects

The GC heap is divided into generations, and objects are listed accordingly. We see that only generation 0 (the youngest generation) has any objects registered for finalization. The notation “(0015bc90->0015bca0)” means that if you look at memory in that range, you’ll see the object pointers that are registered:

0:000> dd 15bc90 15bca0-4

0015bc90  00a743f4 00a79f00 00a7b3d8 00a7b47c

You could run !DumpObj on any of those pointers to learn more. In this example, there are no objects ready for finalization, presumably because they still have roots (You can use !GCRoot to find out). The statistics section provides a higher-level summary of the objects registered for finalization. Note that objects ready for finalization are also included in the statistics (if any).

If you pass -detail then you get extra information on any SyncBlocks that need to be cleaned up, and on any RuntimeCallableWrappers (RCWs) that await cleanup. Both of these data structures are cached and cleaned up by the finalizer thread when it gets a chance to run.

SOS Command: !ObjSize [<Object address>]

With no parameters, !ObjSize lists the size of all objects found on managed threads. It also enumerates all GCHandles in the process, and totals the size of any objects pointed to by those handles. In calculating object size, !ObjSize includes the size of all child objects in addition to the parent.

For example, !DumpObj lists a size of 20 bytes for this Customer object:

0:000> !do a79d40

Name: Customer
MethodTable: 009038ec
EEClass: 03ee1b84
Size: 20(0x14) bytes
 (C:pubunittest.exe)

Fields:
      MT    Field   Offset                 Type       Attr    Value Name
009038ec  4000008        4                CLASS   instance 00a79ce4 name
009038ec  4000009        8                CLASS   instance 00a79d2c bank
009038ec  400000a        c       System.Boolean   instance        1 valid

but !ObjSize lists 152 bytes:

0:000> !ObjSize a79d40
sizeof(00a79d40) =      152 (    0x98) bytes (Customer)

This is because a Customer points to a Bank, has a name, and the Bank points to an Address string. You can use !ObjSize to identify any particularly large objects, such as a managed cache in a web server. Note: above information was from SOS help.

SOS Command: !DumpHeap

!DumpHeap [-stat]
          [-strings]
          [-short]
          [-min <size>]
          [-max <size>]
          [-thinlock]
          [-startAtLowerBound]
          [-mt <MethodTable address>]
          [-type <partial type name>]
          [start [end]]

!DumpHeap is a powerful command that traverses the garbage collected heap, collection statistics about objects. With it’s various options, it can look for particular types, restrict to a range, or look for ThinLocks (see !SyncBlk documentation). Finally, it will provide a warning if it detects excessive fragmentation in the GC heap.

When called without options, the output is first a list of objects in the heap, followed by a report listing all the types found, their size and number:

0:000> !dumpheap

 Address       MT     Size

00a71000 0015cde8       12 Free
00a71024 5ba58328       68
...

total 619 objects

Statistics:
      MT    Count TotalSize Class Name

5ba7607c        1        12 System.Security.Permissions.HostProtectionResource
5ba75d54        1        12 System.Security.Permissions.SecurityPermissionFlag
5ba61f18        1        12 System.Collections.CaseInsensitiveComparer
...

0015cde8        6     10260      Free
5ba57bf8      318     18136 System.String
...

“Free” objects are simply regions of space the garbage collector can use later. If 30% or more of the heap contains “Free” objects, the process may suffer fromheap fragmentation. This is usually caused by pinning objects for a long time combined with a high rate of allocation. Here is example output where !DumpHeap provides a warning about fragmentation:

<After the Statistics section> Fragmented blocks larger than 1MB:

    Addr     Size Followed by
00a780c0    1.5MB    00bec800 System.Byte[]
00da4e38    1.2MB    00ed2c00 System.Byte[]
00f16df0    1.2MB    01044338 System.Byte[]

The arguments in detail:

-stat     Restrict the output to the statistical type summary
-strings  Restrict the output to a statistical string value summary
-short    Limits output to just the address of each object. This allows you
          to easily pipe output from the command to another debugger
          command for automation.
-min      Ignore objects less than the size given in bytes
-max      Ignore objects larger than the size given in bytes
-thinlock Report on any ThinLocks (an efficient locking scheme, see !SyncBlk
          documentation for more info)
-startAtLowerBound
          Force heap walk to begin at lower bound of a supplied address range.
          (During plan phase, the heap is often not walkable because objects
          are being moved. In this case, DumpHeap may report spurious errors,
          in particular bad objects. It may be possible to traverse more of
          the heap after the reported bad object. Even if you specify an
          address range, !DumpHeap will start its walk from the beginning of
          the heap by default. If it finds a bad object before the specified
          range, it will stop before displaying the part of the heap in which
          you are interested. This switch will force !DumpHeap to begin its
          walk at the specified lower bound. You must supply the address of a
          good object as the lower bound for this to work. Display memory at
          the address of the bad object to manually find the next method
          table (use !dumpmt to verify). If the GC is currently in a call to
          memcopy, You may also be able to find the next object's address by
          adding the size to the start address given as parameters.)
-mt       List only those objects with the MethodTable given
-type     List only those objects whose type name is a substring match of the
          string provided.
start     Begin listing from this address
end       Stop listing at this address

A special note about -type: Often, you’d like to find not only Strings, but System.Object arrays that are constrained to contain Strings. (”new String[100]” actually creates a System.Object array, but it can only hold System.String object pointers). You can use -type in a special way to find these arrays. Just pass “-type System.String[]” and those Object arrays will be returned. More generally, “-type <Substring of interesting type>[]“. The start/end parameters can be obtained from the output of !EEHeap -gc. For example, if you only want to list objects in the large heap segment:

0:000> !eeheap -gc
Number of GC Heaps: 1
generation 0 starts at 0x00c32754
generation 1 starts at 0x00c32748
generation 2 starts at 0x00a71000

 segment    begin allocated     size
00a70000 00a71000  010443a8 005d33a8(6108072)
Large object heap starts at 0x01a71000
 segment    begin allocated     size
01a70000 01a71000  01a75000 0x00004000(16384)

Total Size  0x5d73a8(6124456)
------------------------------
GC Heap Size  0x5d73a8(6124456)

0:000> !dumpheap 1a71000 1a75000

 Address       MT     Size
01a71000 5ba88bd8     2064
01a71810 0019fe48     2032 Free
01a72000 5ba88bd8     4096
01a73000 0019fe48     4096 Free
01a74000 5ba88bd8     4096
total 5 objects

Statistics:
      MT    Count TotalSize Class Name
0019fe48        2      6128      Free
5ba88bd8        3     10256 System.Object[]

Total 5 objects

Finally, if gc heap corruption is present, you may see an error like this:

0:000> !dumpheap -stat

object 00a73d24: does not have valid MT
curr_object : 00a73d24
Last good object: 00a73d14
----------------

That indicates a serious problem. See the help for !VerifyHeap for more information on diagnosing the cause.

Note: above information is compiled based on SOS online help.

SOS Command: !GCRoot

!GCRoot [-nostacks] <Object address>

!GCRoot looks for references (or roots) to an object. These can exist in four places:

1. On the stack
2. Within a GC Handle
3. In an object ready for finalization
4. As a member of an object found in 1, 2 or 3 above.

First, all stacks will be searched for roots, then handle tables, and finally the freachable queue of the finalizer. Some caution about the stack roots: !GCRoot doesn’t attempt to determine if a stack root it encountered is valid or is old (discarded) data. You would have to use !CLRStack and !U to disassemble the frame that the local or argument value belongs to in order to determine if it is still in use.

Because people often want to restrict the search to gc handles and freachable objects, there is a -nostacks option.

Sample output:

!GCRoot 012c2c10
Note: Roots found on stacks may be false positives. Run "!help gcroot" for
more info.
Error during command: Warning. Extension is using a callback which Visual Studio does not implement.

Scan Thread 580 OSTHread 244
ESP:12f05c:Root:012c2c10(System.Object[])
ESP:12f194:Root:012c2c10(System.Object[])
ESP:12f414:Root:012c2c10(System.Object[])
ESP:12f444:Root:012c2c10(System.Object[])
ESP:12f534:Root:012c2c10(System.Object[])
ESP:12f6e0:Root:012c2c10(System.Object[])
ESP:12f708:Root:012c2c10(System.Object[])
Scan Thread 3448 OSTHread d78
DOMAIN(0015D338):HANDLE(Strong):9111a8:Root:012c2c10(System.Object[])

SOS Command: !DumpObj

!DumpObj [-nofields] <object address>

!DumpObj allows you to examine the fields of an object, as well as learn important properties of the object such as the EEClass, the MethodTable, and the size.

You might find an object pointer by running !DumpStackObjects and choosing from the resultant list. Here is a simple object:

!DumpObj a79d40
Name: Customer
MethodTable: 009038ec
EEClass: 03ee1b84
Size: 20(0x14) bytes

Fields:
      MT    Field   Offset                 Type  VT     Attr    Value Name
009038ec  4000008        4             Customer   0 instance 00a79ce4 name
009038ec  4000009        8                 Bank   0 instance 00a79d2c bank

Note that fields of type Customer and Bank are themselves objects, and you can run !DumpObj on them too. You could look at the field directly in memory using the offset given. "dd a79d40+8 l1" would allow you to look at the bank field directly. Be careful about using this to set memory breakpoints, since objects can move around in the garbage collected heap.

What else can you do with an object? You might run !GCRoot, to determine what roots are keeping it alive. Or you can find all objects of that type with "!DumpHeap -type Customer".

The column VT contains the value 1 if the field is a valuetype structure, and 0 if the field contains a pointer to another object. For valuetypes, you can take the MethodTable pointer in the MT column, and the Value and pass them to the command !DumpVC.

The abbreviation !do can be used for brevity.
The arguments in detail:

-nofields: do not print fields of the object, useful for objects like String

SOS Command: !DumpStackObjects

!DumpStackObjects [-verify] [top stack [bottom stack]]

This command will display any managed objects it finds within the bounds of the current stack. Combined with the stack tracing commands like K and !CLRStack, it is a good aid to determining the values of locals and parameters.

If you use the -verify option, each non-static CLASS field of an object candidate is validated. This helps to eliminate false positives. It is not on by default because very often in a debugging scenario, you are interested in objects with invalid fields.

The abbreviation !dso can be used for brevity.

Sample output:

!dumpstackobjects
OS Thread Id: 0x244 (580)
ESP/REG  Object   Name
0012f05c 012c2c10 System.Object[]    (System.String[])
0012f194 012c2c10 System.Object[]    (System.String[])
0012f3cc 012c2c34 System.RuntimeType
0012f3d0 012c2c34 System.RuntimeType
0012f3d4 012c2c20 System.RuntimeType
0012f3e0 012c2c20 System.RuntimeType
0012f414 012c2c10 System.Object[]    (System.String[])
0012f444 012c2c10 System.Object[]    (System.String[])
0012f534 012c2c10 System.Object[]    (System.String[])
0012f6e0 012c2c10 System.Object[]    (System.String[])
0012f708 012c2c10 System.Object[]    (System.String[])

SOS Command: !BPMD

!BPMD <module name> <method name>
!BPMD -md <MethodDesc>

!BPMD provides managed breakpoint support. If it can resolve the method name to a loaded, jitted or ngen’d function it will create a breakpoint with “bp”.

If not then either the module that contains the method hasn’t been loaded yet or the module is loaded, but the function is not jitted yet. In these cases, !bpmd asks the Windows Debugger to receive CLR Notifications, and waits to receive news of module loads and JITs, at which time it will try to resolve the function to a breakpoint.

Sample output:

!bpmd LinqSelectTest.exe IEnumerableTest.Program.Main
Found 1 methods...
MethodDesc = 00933000
Setting breakpoint: bp 00FF0070 [IEnumerableTest.Program.Main(System.String[])]

This brings up a good question: “I want to set a breakpoint on the main method of my application, but SOS doesn’t work until the runtime is loaded. How can I do this?”

  1) Start the debugger and type: 

       sxe -c "" clrn

  2) g

  3) You'll get the following notification from the debugger:

     "CLR notification: module 'mscorlib' loaded"

  4) Now you can load SOS and use commands. Type

       .loadby sos mscorwks

     then

       !bpmd myapp.exe MyNamespace.MyApp.Main

  5) g

  6) You will stop at the start of MyApp.Main. If you type "bl" you will see the breakpoint listed.

!BPMD works equally well with generic types. Adding a breakpoint on a generic type sets breakpoints on all already JIT-ted generic methods and sets a pending breakpoint for any instantiation that will be JIT-ted in the future.

Example for generics:
Given the following two classes:

	class G3<T1, T2, T3>
	{
		...
		public void F(T1 p1, T2 p2, T3 p3)
		{ ... }
	}

	public class G1<T> {
		// static method
		static public void G<W>(W w)
		{ ... }
	}

One would issue the following commands to set breapoints on G3.F() and G1.G():

	!bpmd myapp.exe G3`3.F
	!bpmd myapp.exe G1`1.G

!BPMD does not accept offsets nor parameters in the method name. If there are overloaded methods, !bpmd will set a breakpoint for all of them. In the case of hosted environments such as SQL, the module name may be complex, like 'price, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null'.

For this case, just be sure to surround the module name with single quotes,
like: !bpmd 'price, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null' Price.M2