Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > ASP .Net > AppPool Recycle Failure

Thread Tools

AppPool Recycle Failure

Posts: n/a
I'm running a Server Farm of 5 presentation layer machines
There are 55 different apppools, each is dedicated to a single .net applicatio
Only approximately 10 are running at any time and 8 of those get enough traffic to never shut down
1 in an unknown number of app pool restarts, from both timed recycle as well as timed shut down, the application is never able to answer a request
It appears to attempt the JITC and never completes it to give a response on the application
Have looked at this through the IIS discussion group, and they are pretty sure this is a code issue on our side, but since we have been unable to reproduce the problem in a development environment it is very difficult to troublshoot

The applications use the same basic code base with modifications in presentation
All uncached data comes from a Round Robined set of Middle tier servers, which switch machine after each request. Round Robin is provided by a list and custom code calls each machine in order

The only information we really have is that this occurs on recycles/restarts it does not matter if the server is being pounded, or is under light load
In looking at the output of IISState we found one of these errors for each thread that was being orphaned by IIS and showing up as Current Anonymous Users under Web Service Performance counters, as well as the Requests Current ASP.Net Counters

Thread ID: 1
System Thread ID: f0
Kernel Time: 0:0:0.
User Time: 0:0:0.
Other information: Thread is waiting for a lock to be released. Looking for lock owner
Owning thread System ID: 1f7
Thread Type: AS
Executing Page: ASP.dll symbols not found. Unable to locate ASP page
Continuing with other analysis

# ChildEBP RetAddr
00 009ffc3c 77f43741 SharedUserData!SystemCallStub+0x
01 009ffc40 77f5d64e ntdll!ZwWaitForSingleObject+0x
02 009ffc7c 77f42044 ntdll!RtlpWaitForCriticalSection+0x12
03 009ffc84 79e8d946 ntdll!RtlEnterCriticalSection+0x4
04 009ffccc 79e75739 aspnet_isapi!GetAppDomain+0x9
05 009fff1c 79e755e9 aspnet_isapi!HttpCompletion:rocessRequestInManag edCode+0x13
06 009fff24 79e8cb80 aspnet_isapi!HttpCompletion:rocessCompletion+0x2
07 009fff68 792e9155 aspnet_isapi!CorThreadPoolWorkitemCallback+0x2
08 009fff98 792e92cf mscorwks!ThreadpoolMgr::ExecuteWorkRequest+0x2
09 009fffb8 77e4a990 mscorwks!ThreadpoolMgr::WorkerThreadStart+0x12
0a 009fffec 00000000 kernel32!BaseThreadStart+0x3

When the site fails to restore, each new connection creates a Concurrent request that is never answered. We have allowed some of these to go past 3 hours by removing the machine they were running on from the Load Balance

The fun part is when this happens on a site that has been set to use a Web Garden, this will happen to just one of the threads, so the Automated checking software won't pick it up as a failure, because only every 3rd request is failing

Where can we go from here?
Reply With Quote
Natty Gur
Posts: n/a

mmm it wont be easy but certainly will be solved.

I suggest you to use post mortem debug tolls such as adplus and windows
debug tools. I add a link that will guide you through this process :

Those tools will help you to track the problem.

Natty Gur[MVP]

blog :
Mobile: +972-(0)52-8888377

*** Sent via Devdex ***
Don't just participate in USENET...get rewarded for it!
Reply With Quote

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off

Similar Threads
Thread Thread Starter Forum Replies Last Post
SQL Server 2005 Windows authentication with AppPool running underservice account identity JM_newsgroups ASP .Net 2 12-12-2008 06:41 PM
restart AppPool by code or script =?Utf-8?B?QWxoYW1icmEgRWlkb3MgS2lxdWVuZXQ=?= ASP .Net 0 10-03-2007 11:58 AM
MS Powerpoint AppPool Identity Ben Dewey ASP .Net 2 05-12-2006 05:31 PM
ASPDOTNET AppPool Crashing Adam Clark ASP .Net 1 10-27-2005 06:00 PM
AppPool running with Domain identity - "service unavailable" Kevin C ASP .Net Security 2 11-19-2004 11:27 PM