Cannot warm up pages using applicationInitialization in webconfig

Question

I have a simple Umbraco 7.7.2 application and I'm hosting it on Azure (app-service). When I restart the server it takes 20-40 seconds for first time requesting a page which is really annoying specially when the load is high and you are Scaling out to reduce the response times.

I've tried this setting in my webconnfig, but it doesn't seem to work.

<system.webServer>
  <applicationInitialization>
    <add initializationPage="/page1/?warmup=1" hostName="mydomain.com" />
    <add initializationPage="/page1/page2/?warmup=1" hostName="mydomain.com" />
  </applicationInitialization>
</system.webServer>

I might be trying it in a wrong way, but what I did was to restart the server and I've left it for 2-3 minutes without requesting any page.

I've checked my Umbraco logs and the application wasn't even started. Then I've requested the home page and it took 40 seconds to come up.

Then I've tried mydomain.com/page1 and it also took 20 seconds since it was the first request to access it.

*P.S: after the first request, the site is very fast and each page takes less than 100 ms to load

Update

I've implemented a rewrite to stop next redirects as Kevin has suggested. As a result, my Umbraco will start up, but still the requests doesn't reach to the pages.

On my master page, I've added a line to write a line in the logs if it has the warmup in the querystring and it works it the page is hitted from the browser:

if (!string.IsNullOrWhiteSpace( Request.QueryString["warmup"]))
    {
        var pageC = Model.Content;
        logger.Info(pageC.UrlAbsolute()+" "+ Request.QueryString);
    }

However, there is nothing in my logs after

2018-02-08 15:16:51,245 [P7036/D2/T1] INFO Umbraco.Core.CoreBootManager - Umbraco application startup complete (took 12727ms) 2018-02-08 15:16:54,911 [P7036/D2/T1] INFO MyNamespace.Web.CustomStartup - base config done!

Here is the confing that I've added based on Kevin's answer:

 <rule name="No redirect on warmup request (request from localhost with warmup user agent)" stopProcessing="true">
          <match url=".*" />
          <conditions>
            <add input="{REMOTE_ADDR}" pattern="127.0.0.*" />
              </conditions>
          <action type="Rewrite" url="{URL}" />
        </rule>

Also, I've found another similar config on Microsoft:

   <rule name="No redirect on warmup request (request from localhost with warmup user agent)" stopProcessing="true">
          <match url=".*" />
          <conditions>
            <add input="{HTTP_HOST}" pattern="localhost" />
            <add input="{HTTP_USER_AGENT}" pattern="Initialization" />
          </conditions>
          <action type="Rewrite" url="{URL}" />
        </rule>

With

are you doing in place updates and so this initial load time bothers you ?. not sure what is causing this. but as a temp fix you can write a function app to make a page request once an update is done. so that you do not have to do it manually. – Supplejack

@Supplejack Thanks for reply. It is not about updates. Its when the app service scales out. I have different load of clients on my site. Usually it is only 1 instance that runs my application, but 2 hours in the morning and in the evening I have many people accessing my site and therefore it scales to 3-4 instances and for each of them I have the slow loading time for pages accessing the first time – With

Is there custom caching that needs to warm up on the new servers? – Gao

@Gao no. It is not that specially that it happens on each page other than site start up – With

Answer 1

There was so many reasons for the requests not reaching my site and thanks to Kevin and Twamley whom opened my eyes to the probable cause I could trace and find all of them.

First, as Kevin said HTTPS was one of the issues which I've fixed as below:

 <rule name="No redirect on warmup request (request from localhost with warmup user agent)" stopProcessing="true">
      <match url=".*" />
      <conditions>
        <add input="{HTTP_HOST}" pattern="localhost" />
        <add input="{HTTP_USER_AGENT}" pattern="Initialization" />
      </conditions>
      <action type="Rewrite" url="{URL}" />
    </rule>

Then I could see that Umbraco starts up but the requests didn't get to my pages.

  <rule name="Redirect rquests to www.example.com" stopProcessing="true" enabled="true">
          <match url="(.*)" />
          <conditions >
            <add input="{HTTP_HOST}" pattern="^example\.com$" />
          </conditions>
          <action type="Redirect" url="https://www.example.com/{R:0}" />
        </rule>

I didn't expect my requests to get to this redirect since it was at the end of my rewrite rules and therefore it should be stoped at No redirect on warmup request but it didn't so I've added another condition to it: <add input="{HTTP_USER_AGENT}" pattern="Initialization" negate="true" />

  <rule name="Redirect rquests to www.example.com" stopProcessing="true" enabled="true">
          <match url="(.*)" />
          <conditions logicalGrouping="MatchAll">
            <add input="{HTTP_HOST}" pattern="^example\.com$" />
            <add input="{HTTP_USER_AGENT}" pattern="Initialization" negate="true" />
          </conditions>
          <action type="Redirect" url="https://www.example.com/{R:0}" />
        </rule>

Also, I have ipSecurity in my settings since it is my test environment and I didn't want it open to public. Turns out the Initialization cannot hit my site at all if I don't open up to 127.0.0.1.....

 <security>
  <ipSecurity allowUnlisted="false">
    <add ipAddress="127.0.0.1" allowed="true" />
    <add ipAddress="x.x.x.x" allowed="true" />

With

Hi, I know this is an old thread but its directly related to the problem I am having. Does the application initialization request always end at the redirect? I need to warm up a number of WCF services in my web application. However the root of the service does not work, it needs to hit /MyService.svc so I have setup a redirect to route / to /MyService.svc, this doesn't then seem to actually make the request at the redirected URL. – Fyke

Answer 2

Note that azure will warm up the URL on http, so if you are forcing https using rewrite rules, the full site will not warm up, only the redirect module will. Then it needs to finish warming up Umbraco after azure adds it into the load balancer and the first https gets through to the umbraco code. We found that out by checking the http logs when scaling out.

We couldn't figure out how to tell azure to warmup using https, so we allowed Azure to access the site on http by making a rule before our force https rewrite to stopProcessing when {REMOTE_ADDR} matches 127.0.0.*.

    <rule name="Allow localhost to warmup" stopProcessing="true">
      <match url="(.*)"/>
      <conditions>
        <add input="{REMOTE_ADDR}" pattern="127.0.0.*" />
      </conditions>
    </rule>

Westering

OMG! Thanks Kevin. I will look at that – With

I'm not sure if it would matter, but our rule doesn't have any action to it, it just stops the processing. Have you checked your http logs to see what the response code is for the warmup request to see if it's redirecting – Westering

It also looks like we set doApppInitAfterRestart to true like <applicationInitialization doAppInitAfterRestart="true">. I think that may need to be set for these urls to work as the azure warm up just checks the site root by default. – Westering

Answer 3

I like Kevin's idea of just stopping the processing as one of the first rewrite rules. I noticed his didn't have an action but you added one to yours. Maybe try his w/o an action? Is it the first rule in the file?

Another option we use is to add this condition to any problem rules (notice the negate).

<add input="{REMOTE_ADDR}" pattern="127\.0\.0\.1" negate="true"/>

Try temporarily clearing your rewrite rules until you're convinced your warmup is working, then add the rules back a few at a time to find and fix the problem redirect. Force SSL and Trailing Slash type rules will definitely cause problems.

Also, the hostName can get you in trouble easily. It should be a perfect match for your production environment. It doesn't use DNS to resolve it, it just talks to the local site and passes that in as the HOST header.

I don't have any querystrings in my warmup list. Maybe you should try dropping those. You don't really need them because you can change your logging code to:

if (Request.IsLocal && Request.UserAgent == "IIS Application Initialization Warmup") {
    // log it
}

Logging them is a great idea because the warm up requests don't show up in the standard IIS logs. I have my servers mail me at the beginning and end of the warmup by hitting a special first and last entry that uses that if statement. Some useful details from Azure:

new {
    WEBSITE_HOSTNAME = System.Environment.GetEnvironmentVariable("WEBSITE_HOSTNAME"),
    WEBSITE_INSTANCE_ID = System.Environment.GetEnvironmentVariable("WEBSITE_INSTANCE_ID"),
    WEBSITE_SITE_NAME = System.Environment.GetEnvironmentVariable("WEBSITE_SITE_NAME"),
    COMPUTERNAME = System.Environment.GetEnvironmentVariable("COMPUTERNAME"),
    USER_AGENT = Request.UserAgent,
    URL = Request.Url,
    WARM_UP_TIME = (DateTime.UtcNow - _start).ToString()
}

Bronchitis