Intermittent "Failed to load viewstate" error

Question

Yes I know this problem has been reported many times, but my case doesn't seem to fit any proposed solutions (unless I miss something).

The ASP.NET page setup (simplified) is: user clicks a link (technically a node of Infragistics WebDataTree) and this action binds a grid (Infragistics WebHierarchicalDataGrid) located in an UpdatePanel.

Now this works fine in a local test system. When deployed to a production server - it also works - most of the times, but sometimes this throws error:

Failed to load viewstate. The control tree into which viewstate is being loaded must match the control tree that was used to save viewstate during the previous request

This seems to happen more if the production system under heavier load (more users accessing it). Again, I am not building any dynamic controls, just a button and a grid that is populated on button click.

What could be causing this? Why this doesn't happen always, but sometimes? Any idea how to fix it?

UPDATE

Here's confirmed scenario

1. If one user accesses and uses the page - error doesn't happen
2. If two user doing the same (can even be 2 browser sessions from the same client) - error happens.

How could one session affect viewstate of another?

UPDATE 2

Application is deployed to a single Windows 2008/IIS7 server (no web farms/web gardens, no load balancers). No updates rolls out during runtime.

Application works fine if a single users accesses it, but when multiple users hit the server - eventually some of them are getting ViewState error (while others still work fine). They're all doing the same thing - clicking tree nodes that trigger grid rebind inside of UpdatePanel. They use different browser - IE(9-11), FF, Chrome - error could happen to a random user/browser

Oh and we are not encrypting ViewState either.

UPDATE 3

Stack Trace:

at System.Web.UI.Control.LoadViewStateRecursive(Object savedState) at System.Web.UI.Control.LoadChildViewStateByIndex(ArrayList childState) at System.Web.UI.Control.LoadViewStateRecursive(Object savedState) at System.Web.UI.Control.LoadChildViewStateByIndex(ArrayList childState) at System.Web.UI.Control.LoadViewStateRecursive(Object savedState at System.Web.UI.Control.LoadChildViewStateByID(ArrayList childState) at System.Web.UI.Control.LoadViewStateRecursive(Object savedState) at System.Web.UI.Control.LoadChildViewStateByIndex(ArrayList childState) at System.Web.UI.Control.LoadViewStateRecursive(Object savedState) at System.Web.UI.Control.LoadChildViewStateByIndex(ArrayList childState) at System.Web.UI.Control.LoadViewStateRecursive(Object savedState) at System.Web.UI.Control.LoadChildViewStateByIndex(ArrayList childState) at System.Web.UI.Control.LoadViewStateRecursive(Object savedState) at System.Web.UI.Control.LoadChildViewStateByIndex(ArrayList childState) at System.Web.UI.Control.LoadViewStateRecursive(Object savedState) at System.Web.UI.Control.LoadChildViewStateByIndex(ArrayList childState) at System.Web.UI.Control.LoadViewStateRecursive(Object savedState) at System.Web.UI.Page.LoadAllState() at System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)

UPDATE 4

It seems if I reduce size of the data returned with each click (e.g. 20 rows instead of 100) the issue either disappears or appears a lot less often.

I've tried manipulating ViewState - e.g. splitting it into chunks, using session as a ViewState storage to reduce actual posted page size - nothing has effect.

Peccant

Check if you are using a custom View State provider – Sunrise

Could you post the code of the page? It sounds like you have some static controls. – Sunrise

entire code is quite complex to post, I could post snips if it helps, but basically in a click event handler grid datasource is assigned. No static (rather "shared" since it's VB.NET controls are used) all controls are declared in ASPX markup – Peccant

You might not have any explicitly dynamic controls, but the grid sounds dynamic, which means it is building a control tree to which viewstate is being applied. Is it possible that one user updates the source of the grid (thus adding/removing records, thus adding/removing stateful controls)? This would cause a subsequent postback by another user to contain viewstate that is now outdated. – Barefaced

@TimMedora would this cause problems in other cases? Standard scenario: 1. User click button to see initial data 2. Data in the DB changes 3. User clicks button again to refresh data in grid – Peccant

It could. Hard to say definitively that it would; depends on the implementation of the grid, what elements are being reconstructed based on viewstate, if the data changed is on the current page of the grid, etc. That error (obviously) means that there is some sort of mismatch between the POSTed viewstate and the control tree to which it is being applied, so the first place to look is at what changed in the control tree being built. Have you tried triggering the problem on a dev machine (multiple sessions, load testing tools, etc)? Source stepping or even a stack trace would be useful. – Barefaced

That's another weird thing. I used to be able to cause the issue on my dev machine (Win7/IIS7) just opening 2 browser sessions, but not anymore. Will try load testers tho, thanks. But I am not sure how stepping thru code may help - error happens outside the code (I've added stack trace to question) and couldn't be caught – Peccant

I suspect the problem lies in your specific code (unless you can reproduce that in a simple project - in this case, post this somewhere). Inspect all involved specific code for anything that could be shared between users/sessions: usage of static variables, cache at all levels, shared memory, whatever... also checks for hardware equipement that would do cache between two users session where it shouldn't (IP address affinity, etc.) – Danner

Can you share with us the Infragistics dll? Also what version of .Net Framework are you using? – Ingham

@Ingham we're using Infragistics35.Web.v12.2.dll which is 12.2.20122.2202 for CLR 3.5 – Peccant

I see, but can you upload the actual dll/dlls somewhere please? – Ingham

I guess I could but can you elaborate how this may help? – Peccant

Since you said that there are no dynamic controls on the page, and the only control on the page is this Grid (and a button) I assume it's something with the grid. So I wanted to take a look, and try to replicate. I believe it's something with your particular setup, and it's not a generic case. – Ingham

@Ingham here it is dropbox.com/s/0ga7x9v4heknoyh/Infragistics35.Web.v12.2.zip but I am not sure if it be of any help – Peccant

Answer 1

After following this advice on how to debug things outside of my code and gaining access to debug symbols for the Infragistics WebHierarchicalDataGrid DLL as well as its source code - turned out exception is thrown by the control - and its not related to the viewstate. Internally in a 1-element array code tries to access second element, throwing "IndexOutOfBound" exception which undergoes magical transformation to "Failed to load viewstate" thrown to the user.

Infragistics is looking into the error and ways to fix it, but since we're really pressed for time we used following workaround: If "view state" error is detected at the end of async postback - we simple reload the page:

Sys.Application.add_load(appLoaded);

function appLoaded(sender, eventArgs) {
   Sys.WebForms.PageRequestManager.getInstance().add_endRequest(EndRequest)
}

function EndRequest(sender, args) {
    if (args.get_error() != undefined) {
        if (args.get_error().message.indexOf('Failed to load viewstate') != -1) {
            setTimeout(function () { location.reload(true) }, 100)
        }
    }
} 

Yes I do realize this is as elegant as hammering a nail with a screwdriver, but since the error is sporadic and refresh gets rid of it - this will have to do.

Peccant

What is the difference if multiple users access the page at once or not? (Btw, the exception is turned into "Failed to load viewstate", it's thrown during loading of viewstate, and Web.UI.Control.LoadViewStateRecursive catches all IndexOutOfRangeExceptions, and throws a HttpException) – Ingham

Thanks for clarifications. And still I am not sure why this only happens under heavier than usual load - we were able to reproduce it with ordinary loadtesters. Hopefully Infragistics will come up with something. – Peccant

Answer 2

How many servers do you have in production? If there is more than one are you setting the machineKey in your web.config so that all servers will be using the same key to sign/validate the ViewState value?

As Michael Liu pointed out in a comment, it's unlikely you would get the control tree error for the machineKey problem. I would still like to know if you have multiple servers in production though because it could still be possible that you are running into an issue where ServerA served up newer version of the code and then the POST back occurred to ServerB which doesn't yet have the new code, so the control trees wouldn't match. This could technically happen even with a single server if you rolled out changes while there are active users on your site.

Sister

A machineKey discrepancy would result in a viewstate MAC validation error, not a changed control tree error. – Maculate

Very good point. I would still like to know if this person has multiple servers though, because that could still indicate that one or more of the servers has diff. code and when you cross those boundaries you could get the control tree error. I will update my answer. – Sister

Updated my question, no, we don't use anything of a kind. – Peccant

Different versions on the servers were the cause for our errors #6572747 – Devolve

Answer 3

The viewstate are not related with the session state. But, if you try the websiste with 2 instance of the same browser software, the browsers have the same session. This appends for IE>=8, firefox and chrome. Try with 2 different browsers (sorry for my bad english)

Shortlived

Thank you for the response, but in my tests I am using IE9 and explicitly select "New Session" from menu. In real scenario this does happen from multiple clients – Peccant

Answer 4

It seems the UpdatePanel is causing a problem with the viewstate.

Update:

To debug, I would try to log the view states that are being sent out to each client and those that are being received back from the client during the partial post back.

You can store them temporariliy on your db server associated with a session, and clear them when the session is over if there was no exception. Then if you encounter this exception mark it in the db and you can compare what your server sent to the client and what was received back on postback. You can also manually load these values in your browser and replay them to see what happens.

Either the view state is being currupted client side or your server is sending out invalid view states.

At least this will help you get some data to debug.

---

Here is an article about update panels and some alternative ajax methods that might help solve the problem:

http://msdn.microsoft.com/en-us/magazine/cc163413.aspx

Some other possible helps:

http://lachlankeown.blogspot.com/2008/04/firefox-refresh-viewstate-updatepanel.html?m=1

http://weblog.west-wind.com/posts/2005/Jul/16/Invalid-ViewState

http://support.microsoft.com/default.aspx?scid=kb;EN-US;829743

Tortious

Thanks for the links, I will review them. Meanwhile we do have to use the UpdatePanel and unfortunately cannot disable the ViewState. And the maddening thing - the page does work if multiple users aren't hitting it – Peccant

Answer 5

If you host this in production using Load balancing, the problem could occur when a user changes from one server to another. The solution here is often to specify the machineKey in web.config. Read more about it here at msdn.

<machineKey  
    validationKey="21F090935F6E49C2C797F69BBAAD8402ABD2EE0B667A8B44EA7DD4374267A75D7AD972A119482D15A4127461DB1DC347C1A63AE5F1CCFAACFF1B72A7F0A281B"           
    decryptionKey="ABAA84D7EC4BB56D75D217CECFFB9628809BDB8BF91CFCD64568A145BE59719F"
    validation="SHA1" decryption="AES" />

Academy

Thanks, I updated my question - no we don't use load balancers – Peccant

Answer 6

Are your users actually reporting the problem or are you just seeing it in error logs? I've seen this error before when the user cancelled a request on the browser, causing ASP.NET to error when attempting to parse a partially transmitted request. If the viewstate happened to be partially transmitted, you could see this error. The larger your viewstate & the more users you have the more likely you are to see this error. You can safely ignore this error.

You could test this by loading up a page with lots of viewstate and cancelling a postback.

Impressment

Users actually reporting this error so unfortunately we cannot ignore it – Peccant

Answer 7

I experienced similar issue sometime back in one of my project. The reason for the problem was that we were adding dynamic controls to a table control, and then on submitting the page the table would be populated by different set of controls. On looking deeply into the issue we found that mostly for the dropdown controls on the previous page if there is some other control at the same location the viewstate would fail.

The solution was extremely simple for us, since we were creating the controls dynamically they were anyways being created everytime, so we simply turned the ViewState=false for the control (in your case a grid control).

Based on your comment I think you need to identify what for you is spanning across the sessions. It may very well be the data that changes for the first user when another user is logged in. In that case also you should consider setting the ViewState=false, or binding the grid at page_Oninit, that way the ViewState would not conflict.

Hope this helps.

Brunelleschi

Thanks for reply but I'd like to reiterate that we do not use dynamic controls, just a grid, declared in ASPX markup which is bound on the postback. Did your issue also happen only under heavy server load and didn't happen under normal circumnstances? – Peccant

There is definitely some other reason for what you are experiencing but it may be rendered irrelevant if the viewstate does not conflict. If I am not mistaken for you if two users open your site they see or may load data in the grid and then edit it. The data is common and changes affect all. If that is the case you should consider either of the two suggestions I mentioned above. View state conflict happens when a control say grid was initially loaded and when a post back happens on binding it is found the structure of a previously existing row (eg. DataTable row and thus control) has changed. – Brunelleschi

No edits take place the grid is read-only. When users click different tree nodes - grid is bound to different resultset (rows-wise, columns remain the same). So every time a user clicks - he does see different rows in the grid, but normally that causes no problems. Otherwise wouldn't any bound control cause issues on postback if every bind brings new data? – Peccant

The viewstate fails for very specific scenerio and not with every bound. For me it was failing when a dropdown was being replaced with a textbox on binding secon time. (This does not happen for all controls as well [ie. replacing radio button with a text box does not cause any issue.]). – Brunelleschi

Answer 8

Try decompiling the dll with ILSpy, save the project, build it, than link that project instead of the dll. This way you will get the full stack trace, and maybe get closer to the exception.

Ingham

Thanks, I've already considered that. There's an easier way https://mcmap.net/q/713985/-failed-to-load-viewstate-happening-only-occasionally-tough-to-recreate that I am trying now – Peccant

Did you had any luck? Usually stepping into asp.net does not work. However Red Gate's .Net Reflector could help you, that tool can step into any code. – Ingham

I did get 2 stack trace steps further; they led into "IndexOutOfBounds" error inside of Infragistics DLL, still investigating. – Peccant

Answer 9

I had the same problems when for example: clicking on a button that would redirect me, then press the back button on IE with the "old" viewstate, while some content had changed. On IE, Chrome, Firefox, i had that problem due to caching.

To avoid browsers from caching your pages, you can use a random parameter in your url : for ex : "&ran=54921356" that would randomely be generatad. See if this helps

I don't know how your WebHierarchicalDataGrid is binded, your updatepanel wraping it shouldn't be viewstate enabled if data can change between a post-back and then going back.

I hope this helps !

PS: Also, i think that storing the viewstate in session is a bad idea...

Inclement

Agree about viewstate in session, tried this only as a test to see if it makes a difference. Still trying other approaches – Peccant