How to force BundleCollection to flush cached script bundles in MVC4

Question

... or how I learned to stop worrying and just write code against completely undocumented APIs from Microsoft. Is there any actual documentation of the official System.Web.Optimization release? 'cuz I sure can't find any, there's no XML docs, and all the blog posts refer to the RC API which is substantially different. Anyhoo..

I am writing some code to automatically resolve javascript dependencies and am creating bundles on the fly from those dependencies. Everything works great, except if you edit scripts or otherwise make changes that would affect a bundle without restarting the application, the changes won't be reflected. So I added an option to disable caching of the dependencies for use in development.

However, apparently BundleTables caches the URL even if the bundle collection has changed. For example, in my own code when I want to re-create a bundle I do something like this:

// remove an existing bundle
BundleTable.Bundles.Remove(BundleTable.Bundles.GetBundleFor(bundleAlias));

// recreate it.
var bundle = new ScriptBundle(bundleAlias);

// dependencies is a collection of objects representing scripts, 
// this creates a new bundle from that list. 

foreach (var item in dependencies)
{
    bundle.Include(item.Path);
}

// add the new bundle to the collection

BundleTable.Bundles.Add(bundle);

// bundleAlias is the same alias used previously to create the bundle,
// like "~/mybundle1" 

var bundleUrl = BundleTable.Bundles.ResolveBundleUrl(bundleAlias);

// returns something like "/mybundle1?v=hzBkDmqVAC8R_Nme4OYZ5qoq5fLBIhAGguKa28lYLfQ1"

Whenever I remove & recreate a bundle with the same alias, absolutely nothing happens: the bundleUrl returned from ResolveBundleUrl is the same as before I removed & recreated the bundle. By "the same" I mean that the content hash is unchanged to reflect the new contents of the bundle.

edit... actually, it's much worse than that. The bundle itself is cached somehow outside of the Bundles collection. If I just generate my own random hash to prevent the browser from caching the script, ASP.NET returns the old script. So, apparently, removing a bundle from BundleTable.Bundles does not actually do anything.

I can simply change the alias to get around this problem, and that is OK for development, but I don't like that idea since it means either I have to deprecate aliases after each page load, or have a BundleCollection that grows in size on every page load. If you left this on in a production environment, it would be a disaster.

So it seems that when a script is served, it gets cached independent of the actual BundleTables.Bundles object. So if you re-use a URL, even if you've removed the bundle that it referred to before reusing it, it responds with whatever's in its cache, and altering the Bundles object does not flush the cache -- so only new items (or rather, new items with a different name) would ever be used.

The behavior seems odd... removing something from the collection should remove it from the cache. But it doesn't. There must be a way to flush this cache and have it use the current contents of the BundleCollection instead of what it cached when that bundle was first accessed.

Any idea how I would do this?

There is this ResetAll method which has an unknown purpose but it just breaks things anyway so that isn't it.

Norrisnorrv

Same problem here. I think I've managed to solve mine. Try and have a look if it works for you. Totally agree. Documentation for System.Web.Optimization is rubbish and all the samples you can find on the internet are outdated. – Honeysucker

+1 for great reference at top combined with biting comment about MS's expectation of trust. And also for asking the question I want an answer to. – Chenay

Answer 1

We hear your pain on documentation, unfortunately this feature is still changing quite fast, and generating documentation has some lag, and can be outdated almost immediately. Rick's blog post is up to date, and I've tried to answer questions here as well to spread current info in the meantime. We are currently in the process of setting up our official codeplex site which will have always current documentation.

Now in regards to your specific issue of how to flush bundles form the cache.

1. We store the bundled response inside of the ASP.NET cache using a key generated off of the bundle url requested, i.e. Context.Cache["System.Web.Optimization.Bundle:~/bundles/jquery"] we also setup cache dependencies against all of the files and directories that were used to generate this bundle. So if any of the underlying files or directories change, the cache entry will get flushed.

2. We don't really support live updating of the BundleTable/BundleCollection on a per request basis. The fully supported scenario is that bundles are configured during app start(this is so everything works properly in the web farm scenario, otherwise some bundle requests would end up being 404's if sent to the wrong server). Looking at your code example, my guess is that you are trying to modify the bundle collection dynamically on a particular request? Any kind of bundle administration/reconfiguration should be accompanied by an appdomain reset to guarantee everything has been setup correctly.

So avoid modifying your bundle definitions without recycling your app domain. You are free to modify the actual files inside of your bundles, that should automatically be detected and generate new hashcodes for your bundle urls.

Ardy

thank you for bringing your direct knowledge to bear here! Yes - I am trying to modify the bundle collection dynamically. The bundles are built based on a set of dependencies described in another script (that is, itself, not necessarily part of the bundle) -- which is why I'm having this problem. Since changing a script that's in a bundle will force a flush, it can be done - is there a possibility of adding a manual flush method? This isn't crucial --this is for convenience during development - but I hate creating code that could cause problems if accidentally used on prod. – Norrisnorrv

Also can you elaborate on the web farm issue? Would adding a new bundle after application start result it in only being available on the server on which it was created -- or just trying to change an existing one? This would be a bit of a dealkiller for what I'm trying to do since it needs to do runtime resolution of dependencies. – Norrisnorrv

Sure, we could add an explicit cache flush equivalent method, it's already there internally. Regarding the web farm issue, basically imagine you have two webservers A and B, your request goes to A who adds the bundle, and sends down the response, your client now goes to fetch the contents of the bundle, but oops the request goes to server B who did not register the bundle, and there's your 404. – Ardy

I mean, is it possible for bundles to be just added after app start? What is the mechanism by which they are propogated initially, before they are cached? – Norrisnorrv

The cache update is lazy, the first time the bundle is used (typically via rendering a reference to the bundle), it is added to the cache. If you have an equivalent app start hook where you setup your bundles on all webservers before you start handling requests, that should be fine. – Ardy

Sorry to keep asking you questions, I really appreciate the feedback. To be sure I understand - it sounds like adding a new bundle dynamically (at the time a particular page is first constructed) WOULD work fine after app start. Its first access would be via a rendered reference - it just happens that the bundle would have been created inline milliseconds before that reference was rendered. Does that make sense? – Norrisnorrv

Going back to the webfarm scenario, the only thing that matters, is if you can guarantee that on all servers, all bundles exist everywhere, irrespective of any requests that have been made to the server. So if some bundles are created on the fly only in response to a particular request, that will not work. – Ardy

As far as I can tell this doesn't work. That is, if I change the constituent file(s), the server cache isn't cleared as stated here. You have to recycle the thing to get any changes out there. Anyone know where that official documentation actually is? – Thordis

see https://mcmap.net/q/243422/-asp-net-bundles-in-regular-html to workaround the "I need to restart the pool" limitation from plain html – Pauiie

How to handle the scenario if there is a database e.g. Getting variable values for less from db then generating css file. – Munos

"we also setup cache dependencies against all of the files and directories that were used to generate this bundle. So if any of the underlying files or directories change, the cache entry will get flushed." Has anyone actually seen this working? I have to cause the AppDomain to reload in order for the bundle to load an included file if it was changed after the first bundle request. – Destinee

Answer 2

I've got a similar problem.
In my class BundleConfig I was trying to see what was the effect of using BundleTable.EnableOptimizations = true.

public class BundleConfig
{
    public static void RegisterBundles(BundleCollection bundles)
    {
        BundleTable.EnableOptimizations = true;

        bundles.Add(...);
    }
}

Everything was working fine.
At some point I was doing some debugging and set the property to false.
I struggled to understand what was happening cause it seemed that the bundle for jquery (the first one) wouldn't be resolved and loaded (/bundles/jquery?v=).

After some swearing I think(?!) I've managed to sort things out. Try to add bundles.Clear() and bundles.ResetAll() at the beginning of the registration and things should start to work again.

public class BundleConfig
{
    public static void RegisterBundles(BundleCollection bundles)
    {
        bundles.Clear();
        bundles.ResetAll();

        BundleTable.EnableOptimizations = false;

        bundles.Add(...);
    }
}

I've realized I need to run these two methods only when I change the EnableOptimizations property.

UPDATE:

Digging deeper I've found out that BundleTable.Bundles.ResolveBundleUrl and @Scripts.Url seem to have problems to resolve the bundle path.

For sake of simplicity I've added a few images:

I have turned off the optimization and bundled a few scripts.

The same bundle is included in the body.

@Scripts.Url gives me the "optimized" path of the bundle while @Scripts.Render generates the proper one.
Same thing happens with BundleTable.Bundles.ResolveBundleUrl.

I am using Visual Studio 2010 + MVC 4 + Framework .Net 4.0.

Honeysucker

Hmm... the thing is I don't actually want to clear out the bundle table, because it will contain lots of other ones from different pages (created from different sets of dependencies). But since this is really just for working in a development environment, I think I could copy the contents of it, then clear it, then add them again, if that would flush the cache. Horrendously inefficient but if it works, its good enough for dev. – Norrisnorrv

Agree but that's the only option I had. I have spent the whole afternoon trying to understand what was the problem. – Honeysucker

I just tried it, STILL not flushing the cache!! I clear it, ResetAll, and have tried setting EnableOptimizations to false both at startup and inline when I need to reset the cache, nothing happening. Argh. – Norrisnorrv

It sure would be nice if the developer could fire off a quick blog post with even a one-liner about the methods in these objects :) – Norrisnorrv

I reckon that problem is in BundleTable.Bundles.ResolveBundleUrl and @Scripts.Url If the optimization is off the path should be NOT the one of the bundle. – Honeysucker

Saw your update... I need to try this without using ResolveBundleUrl too. This is insidious... – Norrisnorrv

Maybe @Hao Kung can try to help us to understand. – Honeysucker

+1'd you because this is very helpful, just not sure it's right yet since my problem isn't completely solved! – Norrisnorrv

So just to explain what these methods do: Scripts.Url is just an alias for BundleTable.Bundles.ResolveBundleUrl, it also will resolve non bundle urls, so its a generic url resolver that happens to know about bundles. Scripts.Render uses the EnableOptimizations flag to determine whether to render out a reference to bundles, or the components that make up the bundle. – Ardy

Answer 3

Bearing in mind Hao Kung's recommendations to not do this because of web farm scenarios, I think there are a lot of scenarios where you might want to do this. Here is a solution:

BundleTable.Bundles.ResetAll(); //or something more specific if neccesary
var bundle = new Bundle("~/bundles/your-bundle-virtual-path");
//add your includes here or load them in from a config file

//this is where the magic happens
var context = new BundleContext(new HttpContextWrapper(HttpContext.Current), BundleTable.Bundles, bundle.Path);
bundle.UpdateCache(context, bundle.GenerateBundleResponse(context));

BundleTable.Bundles.Add(bundle);

You can call the above code at any time and your bundles will get updated. This works both when EnableOptimizations is true or false - in other words, this will throw out the correct markup in debug or live scenarios, with:

@Scripts.Render("~/bundles/your-bundle-virtual-path")

Clangor

Further reading here which talks a little about the caching and GenerateBundleResponse – Clangor

Answer 4

I also ran into issues with updating bundles without rebuilding. Here are the important things to understand:

• The bundle DOES NOT get updated if the file paths change.
• The bundle DOES get updated if the bundle's virtual path changes.
• The bundle DOES get updated if the files on disk change.

So knowing that, if you're doing dynamic bundling, you can write some code to make the bundle's virtual path be based on the file paths. I recommend hashing the file paths and appending that hash to the end of the bundle's virtual path. This way when the file paths change so does the virtual path and the bundle will update.

Here's the code I ended up with that solved the issue for me:

    public static IHtmlString RenderStyleBundle(string bundlePath, string[] filePaths)
    {
        // Add a hash of the files onto the path to ensure that the filepaths have not changed.
        bundlePath = string.Format("{0}{1}", bundlePath, GetBundleHashForFiles(filePaths));

        var bundleIsRegistered = BundleTable
            .Bundles
            .GetRegisteredBundles()
            .Where(bundle => bundle.Path == bundlePath)
            .Any();

        if(!bundleIsRegistered)
        {
            var bundle = new StyleBundle(bundlePath);
            bundle.Include(filePaths);
            BundleTable.Bundles.Add(bundle);
        }

        return Styles.Render(bundlePath);
    }

    static string GetBundleHashForFiles(IEnumerable<string> filePaths)
    {
        // Create a unique hash for this set of files
        var aggregatedPaths = filePaths.Aggregate((pathString, next) => pathString + next);
        var Md5 = MD5.Create();
        var encodedPaths = Encoding.UTF8.GetBytes(aggregatedPaths);
        var hash = Md5.ComputeHash(encodedPaths);
        var bundlePath = hash.Aggregate(string.Empty, (hashString, next) => string.Format("{0}{1:x2}", hashString, next));
        return bundlePath;
    }

Awe

I recommend generally avoiding Aggregate for string concatenation, due to the risk of someone not thinking about the inherent Schlemiel the Painter's algorithm in repeatedly using +. Instead, just do string.Join("", filePaths). This will not have that problem, even for very large inputs. – Fetterlock

Answer 5

Have you tried deriving from (StyleBundle or ScriptBundle), adding no inclusions in your constructor and then overriding

public override IEnumerable<System.IO.FileInfo> EnumerateFiles(BundleContext context)

I do this for dynamic style sheets and EnumerateFiles gets called on every request. It's probably not the greatest solution but it works.

Waxy

Answer 6

Apologies to revive a dead thread, however I ran into a similar issue with Bundle caching in an Umbraco site where I wanted the stylesheets/scripts to automatically minify when the user changed the pretty version in the backend.

The code I already had was (in the onSaved method for the stylesheet):

 BundleTable.Bundles.Add(new StyleBundle("~/bundles/styles.min.css").Include(
                           "~/css/main.css"
                        ));

and (onApplicationStarted):

BundleTable.EnableOptimizations = true;

No matter what I tried, the "~/bundles/styles.min.css" file didn't seem to change. In the head of my page, I was originally loading in the stylesheet like so:

<link rel="stylesheet" href="~/bundles/styles.min.css" />

However, I got it to work by changing this to:

@Styles.Render("~/bundles/styles.min.css")

The Styles.Render method pulls in a query string at the end of the file name which I am guessing is the cache key described by Hao above.

For me, it was as simple as that. Hope this helps anyone else like me who was googling this for hours and could only find several year old posts!

Sybilla