Detecting App Pool Recycling and Mitigating Its Effects

If you wrote any web apps using ASP.NET and IIS, you are familiar with the concept of app pools in IIS.  If you also used Entity Framework or any other library that requires some amount of work to warm-up, you have seen the effects of app pool recycling.  At that time application domain for your ASP.NET application goes down.  If a library, such as Entity Framework, caches some start up data in App Domain, it will loose that information.  As a result, the next request to your web site will take longer than usual.  One thing you should always do is tune application pool recycling frequency.  Here is the configuration screen for App pool in IIS.

 

image

A few things you should do.  First of all, set Start Mode to Always Running.  Without that you will not be able to take advantage of handling recycle events.  Then, set Idle timeout to 0, keeping the application from going down after period of inactivity.  Finally, you can change regular time interval and /or setup Specific times.  You should recycle app pool periodically, but you should not do that during peak load times, as more users might notice warm up period.  If you have no load at night, just set the pool to recycle then.

Now, the next step.  You want to warm up your application after app pool recycling, but before first user hits the site.  You have to have some place of course that does the work.  The simplest one is to add some Web Api end point or a form that performs warm up actions, such as issuing a simple database query using Entity Framework, or any other actions that will “prime” the system.  Now, you need to call that endpoint when app pool recycles.  You can always write a Windows Service that you can install that will call the web page that performs the warm up action at a specific time.  What I wand to document below is a method that will enable you to get notifications of the actual recycle events.  To do this, enable all recycle events to write to Event Log.

image

Next, we will take advantage of the fact that you can actually listen to events written to log via this class I wrote:

using System;
using System.Diagnostics;
using System.Threading.Tasks;

namespace AppPoolRecycleDetection
{
    public class EventLogHandler : IDisposable
    {
        private Func<EventLogEntry, Task> _eventHandlerTask;

        private EventLog _eventLog;

        public EventLogHandler(
            string eventLogName,
            Func<EventLogEntry, Task> eventHandlerTask)
        {
            _eventHandlerTask = eventHandlerTask;
            _eventLog = new EventLog(eventLogName)
            {
                EnableRaisingEvents = true
            };
            _eventLog.EntryWritten += EntryWritten;
        }

        private async void EntryWritten(object sender, EntryWrittenEventArgs e)
        {

            await _eventHandlerTask(e.Entry);
        }

        public void Dispose()
        {
            if (_eventLog != null)
            {
                _eventLog.Dispose();
                _eventLog = null;
            }
            _eventHandlerTask = null;
        }
    }
}

It is simple class, it just takes the log name and method to call when an event is written.  After we get the entry, we need to decide if app pool needs to be warmed up.  In my case I am just looking for source, but ideally you want to add more checks.  Of course, if the app is already warmed up, calling a service or a page will have no impact on the system.

using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Linq;
using System.Net;
using System.Threading.Tasks;

namespace AppPoolRecycleDetection
{
    public class AppPoolRecyleDetector : IDisposable
    {
        private readonly Dictionary<string, IEnumerable<string>> _pools;
        private readonly Action<string> _notificationChannel;
        private EventLogHandler _systemLogEventHandler;


        public AppPoolRecyleDetector(Dictionary<string, IEnumerable<string>> pools, Action<string> notificationChannel)
        {
            _pools = pools;
            _notificationChannel = notificationChannel;
        }

        public async Task StartAsync()
        {

            await InitialWarmUp();

            _systemLogEventHandler = new EventLogHandler("System", EntryWritten);

        }

        private async Task EntryWritten(EventLogEntry eventLogEntry)
        {
            if (eventLogEntry.Source == "WAS")
            {
                _notificationChannel("Got event with message " + eventLogEntry.Message);
                foreach (var appPool in eventLogEntry.ReplacementStrings)
                {
                    if (!string.IsNullOrEmpty(appPool))
                    {
                        if (_pools.ContainsKey(appPool))
                        {
                            await WarmUpOnePool(appPool);
                        }
                    }
                }
            }
        }



        private async Task InitialWarmUp()
        {
            _notificationChannel("Warming up");
            foreach (var pool in _pools.Keys)
            {
                await WarmUpOnePool(pool);
            }
        }

        private async Task WarmUpOnePool(string appPool)
        {
            _notificationChannel("Warming up pool " + appPool);
            foreach (var url in _pools[appPool])
            {
                if (!string.IsNullOrEmpty(url))
                {
                    try
                    {
                        _notificationChannel("Warming up pool " + appPool + " with url of " + url);
                        await Download(url);
                    }
                    catch (Exception exception)
                    {
                        _notificationChannel("Exception: " + exception);
                    }
                }
            }
        }

        private async Task<string> Download(string url)
        {
            using (var client = new WebClient())
            {
                var result = await client.DownloadStringTaskAsync(new Uri(url, UriKind.Absolute));
                return result;
            }
        }

        public void Dispose()
        {
            if (_systemLogEventHandler != null)
            {
                _systemLogEventHandler.Dispose();
                _systemLogEventHandler = null;
            }

        }
    }
}

As you can see, I am simply telling the detector which app pools to monitor and what URLs to hit for each one.  Simple and easy.  I do make some assumptions, like one of the replacement strings will contain app pool name.  Based on what I observed, this is always the case.

Using the detector is easy, just use real app pool names and URLs

 var detector = new AppPoolRecyleDetector(new Dictionary<string, IEnumerable<string>>
            {
                {"DefaultAppPool", new []{ "http://www.google.com", "http://www.bing.com"}},
                {"AnotherPool", new []{ "http://www.google.com", "http://www.bing.com"}}
            }, Console.WriteLine);

            detector.StartAsync().Wait();

 

Enjoy.

3 Comments

Leave a Reply to Balajiprasad Cancel reply

Your email address will not be published. Required fields are marked *