The Coding Hut

My shiny new Vista laptop has been crashing on almost a daily basis lately. Being a developer, this is unacceptable. There’s nothing like losing an hour’s worth of coding that you’ll never write the same way again. After getting fed up, I decided to do something about it. Blue screen’s suck. Plain and simple. They offer no information and your system is generally restarting before you can read what they do give you. I needed a way to get to the bottom of this. Enter Windows Debugging Tools.

Getting to work

In order to analyze the .dmp files produced after a blue screen, the Microsoft Debugging Tools must be installed. This can be downloaded (16 Megs) from http://msdl.microsoft.com/download/symbols/debuggers/dbg_x86_6.8.4.0.msi. This is not required on the computer being diagnosed. You can obtain the .dmp file from the affected computer and perform this process on your own PC.

Windows dump files are typically stored in C:\Windows\Minidump. To verify the location open control panel and system properties. Click to the Advanced tab and choose Settings under Startup and Recovery.

Once you have installed the debugging tools, run Start > Programs > Debugging Tools for Windows > WinDbg

The first time you run WinDbg, select File > Symbol File Path and enter without quotes: “srv*c:\cache*http://msdl.microsoft.com/download/symbols;” into the text box. Next choose File > Save workspace. If you receive prompts to save from that point on you will normally want to choose no.

Choose File > Open Crash Dump. Browse to and open the desired .dmp file to analyze.

WinDbg may appear to do nothing for a minute while it loads the symbols and data from the file, especially the first time it is used. When it has finished the output will look similar to the following image.

At this point, the tool has suggested the problem was caused by tdx.sys. Next click the link ‘!analyze -v’ or type it in the box at the bottom of the window and press enter. This step may appear to hang, again more so the first time you use the tool, but it is running. Do not click the link more than once or enter the command. Doing so will cause the analysis to run multiple times back to back. After it has finished, the output will resemble the following image. (For a clearer view, click the image then click the green arrow at the bottom of the screen for full-size)

There is a lot of information given in the above image. First we see the actual error that was displayed (often the PC will reboot before the user can gather the message). The error shown above is probably the most common, and also the most useless, though on occasion it can be very helpful.

Some of the other more descriptive errors I’ve gotten are

DRIVER_POWER_STATE_FAILURE (9f)
A driver is causing an inconsistent power state.

CRITICAL_OBJECT_TERMINATION (f4)
A process or thread crucial to system operation has unexpectedly exited or been terminated. Several processes and threads are necessary for the operation of the system; when they are terminated (for any reason), the system can no longer function.

VIDEO_TDR_FAILURE (116)
Attempt to reset the display driver and recover from timeout failed.

The next possibly useful piece of information is the FAULTING_IP. This is the last instruction windows carried out prior to initiating the dump. In this case, TcpSegmentTcbSend is the function within tdx.sys that was running. 252 is the offset (how far into the function it was). The next line, which is typically of little use, shows the actual instruction. In this case it was add eax, dword ptr [edx+14h]. Unless you are the author of the crashing application, this would normally only be useful when reporting issues to the software vendor.

A little further down we are given the process name. FDM.EXE was the application that was using tdx.sys.

Next we get to STACK_TEXT. This section will show the stack trace, or list of a the functions that were executed leading up to the crash. With what we know so far, and after examining the stack trace, it is fairly apparent that this crash occurred while tdx.sys (owned by fdm.exe) was attempting to send something over TCP/IP.

In some cases this information can be useful to determine the next step to take in finding the exact cause of the crash, or at the least to determine the offending application if it is unknown which is usually the case. In other instances, the information gathered with this method will not prove to be of any use.

Important side notes

There will be times, for whatever reason, that the dump will be reported as inconclusive. There will also be times, where there will be more links like the !analyze -v one from above. Generally these will reference a file, memory location, some form of driver identifier, or other various types of information. If you see them go ahead and click. It’s pretty much hit-n-miss as to whether it will spit out anything of use. Keep in mind that most links will only function [reliably] on the machine the .dmp file originated from.

There may also be times when it will report that no symbols were found. As of this article, the only occurrence of this I’ve run into was with a crashing video driver. This is different than the mention of missing symbols in the first image. If you run into this it will look something like

*************************************************************************
*** ***
*** ***
*** Your debugger is not using the correct symbols ***
*** ***
*** In order for this command to work properly, your symbol path ***
*** must point to .pdb files that have full type information. ***
*** ***
*** Certain .pdb files (such as the public OS symbols) do not ***
*** contain the required information. Contact the group that ***
*** provided you with these symbols if you need this command to ***
*** work. ***
*** ***
*** Type referenced: dxgkrnl!_TDR_RECOVERY_CONTEXT ***
*** ***
*************************************************************************

This particular one was from the VIDEO_TDR_FAILURE error above.

That’s about it for this article. WinDbg looks to be a very powerful application and this article only touches the surface of what may be possible with it. I have not done so yet, but there may be more information to be had with this process if your system is set to save a full memory dump rather than mini or kernel. I’ll post again if / when I dive a bit deeper into what WinDbg is capable of.