Umbraco: An unexpected network error on Azure Web Apps

Question

We have an Umbraco website (version 7.5.11) hosted on Azure Web Apps.

We are experiencing the following exception intermittently (3 times within the past 3 weeks). Once the exception occurs it brings the website down until we republish the home node in Umbraco. At all other times the website is working as expected, including retrieving image files from the server.

Exception type: IOException

Exception message: An unexpected network error occurred. at Umbraco.Core.Cache.HttpRuntimeCacheProvider.GetCacheItem(String cacheKey, Func1 getCacheItem, Nullable1 timeout, Boolean isSliding, CacheItemPriority priority, CacheItemRemovedCallback removedCallback, CacheDependency dependency) at Umbraco.Core.Cache.HttpRuntimeCacheProvider.GetCacheItem(String cacheKey, Func1 getCacheItem, Nullable1 timeout, Boolean isSliding, CacheItemPriority priority, CacheItemRemovedCallback removedCallback, String[] dependentFiles) at Umbraco.Core.Cache.DeepCloneRuntimeCacheProvider.GetCacheItem(String cacheKey, Func1 getCacheItem, Nullable1 timeout, Boolean isSliding, CacheItemPriority priority, CacheItemRemovedCallback removedCallback, String[] dependentFiles) at Umbraco.Web.PublishedCache.XmlPublishedCache.PublishedMediaCache.GetCacheValues(Int32 id, Func2 func) at Umbraco.Web.PublishedCache.XmlPublishedCache.PublishedMediaCache.GetUmbracoMedia(Int32 id) at Umbraco.Web.PublishedCache.XmlPublishedCache.PublishedMediaCache.GetById(UmbracoContext umbracoContext, Boolean preview, Int32 nodeId) at Umbraco.Web.PublishedCache.ContextualPublishedCache1.GetById(Boolean preview, Int32 contentId) at Umbraco.Web.PublishedContentQuery.DocumentById(Int32 id, ContextualPublishedCache cache, Object ifNotFound) at Umbraco.Web.PublishedContentQuery.Media(Int32 id) at Umbraco.Web.UmbracoHelper.Media(String id)

The media file exists, and republishing the home node brought the site back online.

At the time of the exception, no code changes were deployed and no pages were updated / published within Umbraco.

Has anyone experienced something similar, or any ideas what the root cause is?

Apparatus

Answer 1

According to the source code of PublishedMediaCache.cs, the exception is often caused by following issue.

Examine index is corrupted.

Here is a thread on umbraco forum which related to your issue.

Examine corruption issues

And here is the solution for this issue from @Shannon Deminick.

If you are using Azure web apps and are NOT auto-scaling, you should use these settings:

1. useTempStorage="Sync"
2. use this feature to store local index files: http://issues.umbraco.org/issue/U4-7614
3. Remove the {machinename} token from your index path
4. RebuildOnAppStart="true" - since this should only happen one time

If you are using Azure web apps and are load balancing w/ auto-scaling your front-end workers then:

1. useTempStorage="Sync"
2. use this feature to store local index files: http://issues.umbraco.org/issue/U4-7614
3. You must have the {machinename} token from your index path
4. RebuildOnAppStart="true" - so that when new sites come online, their indexes are built
5. ... yes in some cases this might not be ideal, please see: https://our.umbraco.org/forum/extending-umbraco-and-using-the-api/74731-examine-corruption-issues#comment-244293

Fullblown