Recently I had a customer where they had a Certificate Authority in a lab environment and one in a production environment. At first sight both seemed to function correctly. However SCOM (System Center Operations Manager, a monitoring solution) was showing various events of application pool crashes for both environments. The application pool belonged to the CA Web Enrollment pages. When investigating the the Event log on those machines we found the following events to be recurring:
Faulting application name: w3wp.exe, version: 7.5.7601.17514, time stamp: 0x4ce7afa2
Faulting module name: scrdenrl.dll_unloaded, version: 0.0.0.0, time stamp: 0x4a5bc7f2
Exception code: 0xc0000005
Fault offset: 0x000007fef9402594
Faulting process id: 0x10b4
Faulting application start time: 0x01cdee76c8747cfb
Faulting application path: c:\windows\system32\inetsrv\w3wp.exe
Faulting module path: scrdenrl.dll
Report Id: 795f74be-5a8c-11e2-8b2c-005056ac0079
A process serving application pool 'DefaultAppPool' terminated unexpectedly. The process id was '4276'. The process exit code was '0xff'.
The events were recurring, but not very exact. Sometimes it was about every 5 minutes, but sometimes it was once an hour, or even once a day. They only thing we could say that it occurred at least once a day. Besides those events, I couldn’t find anything out of the ordinary on those machines. So off to plan B: google gave me this: TechNet Forums: Prolific number of Windows error reports pertaining to 2008 R2 certificate services.
This seemed to match my problem exactly. I tried the suggestion: removed the CA web components/IIS, reboot, reinstalled. Quickly the events reappeared. The Second thing I noticed here was that SCOM was also involved. Of course, it could be the causing it or the one noticing it…. Disabling the SCOM agent didn’t help. So I started digging deeper. I started looking at the IIS logs. I could see that whilst the CA isn’t visited that regularly, still a lot of requests where logged at frequent intervals. The user agent of the request was mentioning SCOM, so it was pretty obvious this was part of some monitoring configuration.
I asked the guy responsible for SCOM, and besides the regular host based monitoring, they also added URL monitoring (for /certsrv). After disabling this URL monitoring the events stopped occurring. So somehow SCOM doesn’t plays to nice with its requests. I’m not sure why it causes the application pool to crash, maybe it’s something which has to be fixed on the Certificate Authority side, but I’m glad at least I found out WHAT was causing it!