Windows Error Reporting (WER) (codenamed Watson) is a crash reporting technology introduced by Microsoft with Windows XP and included in later Windows versions and Windows Mobile 5.0 and 6.0. Not to be confused with the Dr. Watson debugging tool which left the memory dump on the user's local machine, Windows Error Reporting collects and offers to send post-error debug information (a memory dump) using the Internet to Microsoft when an application crashes or stops responding on a user's desktop. No data is sent without the user's consent. When a crash dump (or other error signature information) reaches the Microsoft server, it is analyzed, and information about a solution is sent back to the user if available. Solutions are served using Windows Error Reporting Responses. Windows Error Reporting runs as a Windows service. Kinshuman Kinshumann is the original architect of WER. WER was also included in the Association for Computing Machinery (ACM) hall of fame for its impact on the computing industry.
History
Windows XP
Microsoft first introduced Windows Error Reporting with Windows XP.
Windows Vista
Windows Error Reporting was improved significantly in Windows Vista, when public APIs were introduced for reporting failures other than application crashes and hangs. Using the new APIs, as documented on MSDN, developers can create custom reports and customize the reporting user interface. Windows Error Reporting was also revamped with a focus on reliability and user experience. For example, WER can now report errors even from processes in bad states such as stack exhaustions, PEB/TEB corruptions, and heap corruptions, conditions which in releases prior to Windows Vista would have resulted in silent program termination with no error report. A new Control Panel applet, "Problem Reports and Solutions" was also introduced, keeping a record of system and application errors and issues, as well as presenting probable solutions to problems.
Windows 7
The Problem Reports and Solutions Control Panel applet was replaced by the Maintenance section of the Action Center on Windows 7 and Server 2008 R2.
A new app, Problem Steps Recorder (PSR.exe), is available on all builds of Windows 7 and enables the collection of the actions performed by a user while encountering a crash so that testers and developers can reproduce the situation for analysis and debugging.
System design
WER is a distributed system. Client-side software detects an error condition, generates an error report, labels the bucket, and reports the error to the WER service. The WER service records the error occurrence and then, depending on information known about the particular error, might request additional data from the client, or direct the client to a solution. Programmers access the WER service to retrieve data for specific error reports and for statistics-based debugging.
Errors collected by WER clients are sent to the WER service. The WER service employs approximately 60 servers connected to a 65TB storage area network that stores the error report database and a 120TB storage area network that stores up to 6 months of raw CAB files. The service is provisioned to receive and process well over 100 million error reports per day, which is sufficient to survive correlated global events such as Internet worms.
It can also provide the service where it considered the object by the directory server. Information is also stored to collect and associated with the object and resource. Sometimes the directory service the user do not have to remember the physical address of a network resources by providing name and locate the resources.
Buckets
In the Microsoft Windows Error Reporting (WER) system, crash reports are organized according to "buckets". Buckets classify issues by:
- Application Name,
- Application Version,
- Application Build Date,
- Module Name,
- Module Version,
- Module Build Date,
- OS Exception Code/System Error Code,
- and Module Code Offset.
Ideally, each bucket contains crash reports that are caused by one and only one root cause. However, there are instances where this ideal one-to-one mapping is not the case. First, the heuristics that group failures can result in a single failure's being attributed to multiple buckets; for instance, each time an application with a failure is recompiled, the application will have a new Module Build Date, and resulting failures will then map to multiple buckets. Second, because only certain information about the failure state is factored into the bucketing algorithm, multiple distinct bugs can be mapped to a single bucket; for instance, if an application calls a single function like strlen with strings corrupted in different ways by different underlying code defects, the failures could map to the same bucket because they appear to be crashes in the same function from the same application, etc. This occurs because the bucket is generated on the Windows OS client without performing any symbol analysis on the memory dump: The module that is picked by the Windows Error Reporting client is the module at the top of the stack. Investigations of many reports result in a faulting module that is different from the original bucket determination.
Third-party software
Software & hardware manufacturers may access their error reports using Microsoft's Windows Dev Center Hardware and Desktop Dashboard (formerly Winqual) program. In order to ensure that error reporting data only goes to the engineers responsible for the product, Microsoft requires that interested vendors obtain a VeriSign Class 3 Digital ID or DigiCert certificate. Digital certificates provided by cheaper providers (such as Thawte, Comodo, GlobalSign, GeoTrust, Cybertrust, Entrust, GoDaddy, QuoVadis, Trustwave, SecureTrust, Wells Fargo) are not accepted.
Software and hardware manufacturers can also close the loop with their customers by linking error signatures to Windows Error Reporting Responses. This allows distributing solutions as well as collecting extra information from customers (such as reproducing the steps they took before the crash) and providing them with support links.
Impact on future software
Microsoft has reported that data collected from Windows Error Reporting has made a huge difference in the way software is developed internally. For instance, in 2002, Steve Ballmer noted that error reports enabled the Windows team to fix 29% of all Windows XP errors with Windows XP SP1. Over half of all Microsoft Office XP errors were fixed with Office XP SP2. Success is based in part on the 80/20 rule. Error reporting data reveals that there is a small set of bugs that is responsible for the vast majority of the problems users see. Fixing 20% of code defects can eliminate 80% or more of the problems users encounter. An article in the New York Times confirmed that error reporting data had been instrumental in fixing problems seen in the beta releases of Windows Vista and Microsoft Office 2007.
Privacy concerns and use by the NSA
Although Microsoft has made privacy assurances, they acknowledge that personally identifiable information could be contained in the memory and application data compiled in the 100-200 KB "minidumps" that Windows Error Reporting compiles and sends back to Microsoft. They insist that in case personal data is sent to Microsoft, it won't be used to identify users, according to Microsoft's privacy policy. But in reporting issues to Microsoft, users need to trust Microsoft's partners as well. About 450 partners have been granted access to the error reporting database to see records related to their device drivers and apps.
Older versions of WER send data without encryption; only WER from Windows 8 uses TLS encryption. In March 2014, Microsoft released an update (KB2929733) for Windows Vista, 7 and Server 2008 that encrypts the first stage of WER.
In December 2013, an independent lab found that WER automatically sends information to Microsoft when a new USB device is plugged to the PC.
According to Der Spiegel, the Microsoft crash reporter has been exploited by NSA's TAO unit to hack into the computers of Mexico's Secretariat of Public Security. According to the same source, Microsoft crash reports are automatically harvested in NSA's XKeyscore database, in order to facilitate such operations.
See also
References
- ^ What are WER Services?
- An overview of WER consent settings and corresponding UI behavior
- 0x0 0x0 Solution
- Debugging in the (Very) Large: Ten Years of Implementation and Experience
- WER CACM Award Kinshuman
- WER APIs
- Windows Error Reporting Problem Steps Recorder
- Debugging in the (Very) Large: Ten Years of Implementation and Experience
- How WER collects and classifies error reports
- "NTSTATUS values". Microsoft. Retrieved 2015-06-08.
- "Bug Check Code Reference". Microsoft. Retrieved 2015-06-08.
- "System Error Codes (Windows)". Microsoft. Retrieved 2015-06-08.
- "HRESULT Values". Microsoft. Retrieved 2015-06-08.
- MSDN Blogs > WER Services > The only thing constant is change – Part 1
- "SysDev (was Winqual) website". Archived from the original on 2018-08-03. Retrieved 2012-11-07.
- Update a code signing certificate
- Introducing Windows Error Reporting
- WinQual Registration Head Aches
- Microsoft Support Forum: WER with Thawte authenticode signed app
- The Old New Thing: How can a company get access to Windows Error Reporting data?
- The great digital certificate ripoff?
- Steve Ballmer's letter: Connecting to customers
- Markoff, John (9 October 2006). "A Challenge for Exterminators (Published 2006)". The New York Times.
- "Microsoft Privacy Statement for Error Reporting". Archived from the original on 2012-10-10. Retrieved 2007-10-07.
- Description of the end user privacy policy in application error reporting when you are using Office
- Bekker, Scott (3 October 2002). "Microsoft Error Reporting Drives Bug Fixing Efforts". Redmond Partner Channel. 1105 Redmond Media Group.
- ^ "Are Your Windows Error Reports Leaking Data?". Websense Security Labs. 29 Dec 2013. Retrieved 4 January 2014.
- "The first stage of the WER protocol is not SSL encrypted in Windows". Microsoft. 11 March 2014. Retrieved 10 January 2015.
- Inside TAO: Documents Reveal Top NSA Hacking Unit
