ASP.NET MVC 3 Razor performance

Question

Important Update: See update 5 at the bottom there is no performance issue in asp.net mvc 3, this is a benchmark issue

I've made a simple hello world project in asp.net mvc2,3 aspx and 3 razor and benchmarked them. What I see is:

System                  Requests per second
-------------------------------------------
asp.net mvc 2 ASPX                     4200
asp.net mvc 3 Beta 1 ASPX              3200
asp.net mvc 3 Beta 1 Razor             1700

What's wrong with razor, it's so slow?

Update: I've redone the test. All 4 test virtual directories uses the same .net 4 integrated mode app pool. All projects are done with add new empty x project and adding 1 page with 1 line text and with no code. all sites are compiled in release mode. My system is windows 7, 4 gb i7 4 cores. I've run the test 2 times to warm up iis and these are second run results. apache bench parameters : ab -n100000 -c1000 results:

System         Requests per second   CPU Utilization
----------------------------------------------------
asp.net 4                     4780               43%
mcv 2                         4322               58%
mvc 3 beta 1 aspx             2324               54%
mvc 3 beta 1 razor            1615               54%

Update 2 Scott Guthrie answered in his blog:

We haven't fully optimized MVC3 yet (there is usually a lot of cache tuning we do). We expect razor to be the same performance as the .aspx view engine before it is finally released.

System         Requests per second   CPU Utilization
----------------------------------------------------
mvc 3 rc1 razor               1960               54%
mvc 3 rc2 razor               2187               54%
mvc 3 rc2 aspx                4014               58%

Update 5 all tests done in release mode but, the problem was debug="true" in my web.config file (that also effects release builds), after change it to false, issue fixed. And it's interesting how it's effecting only razor templating at this scale. This should be in our mind on deployments.

System         Requests per second   CPU Utilization
----------------------------------------------------
mvc 3 rc2 razor               3940               58%
mvc 3 rc2 aspx                4100               58%

Thanks to asp.net mvc team, excellent job!

Mere

How did you perform the benchmark? Was your site deployed on IIS in mode Release? Did you use the <deployment retail="true" /> section in your machine.config? Also remember that ASP.NET MVC 3 is still under heavy development so you cannot expect it to be fully optimized yet. At least wait until it hits RTM. – Messmate

+1 Darin - can't expect a product in alpha/beta stages to be fully optimized. – Referendum

That's a perfectly suitable answer, why would you submit it as a comment? – Celle

Can you disable the old ASPX ViewEngine in MVC3? I think I read somewhere that multiple ViewEngines are a major performance hog in MVC2, perhaps that problem still exists in MVC3 and that overhead is part of the problem? Would be very interesting to find that out! – Giguere

I saw you asked this question on Phil Haack's blog in relation to RC2. We did some signifficant perf work and so your numbers should improve. Could you give the new build a try? – Ropedancer

@marcind: I've updated the benchmark. There is a problem when using razor templating. I have a razor page with only 1 line string. In the same test when I rename the file extesion with aspx, the performance gain is twice as you can see in my update, interesting. – Mere

Posting response as a new answer since the RC2 bits are pretty much what RTM is going to be. – Ropedancer

@upthecreek:all tests done in release mode and I didn't expect a default setting in web.config that can also effects release builds. It's interesting how it's effecting only razor templating (not aspx) at this scale. – Mere

Answer 1

(new answer to respond to your RC2 numbers)

Thanks for the updated numbers. A few points:

1. Your Aspx numbers look good, in the sense that we expect MVC3 Aspx to be on par with MVC2 Aspx (a bit slower in such a Hello World example is expected)
2. Your Razor numbers look suspect. We know Razor is a bit slower than the equivalent Aspx, however, the difference should be no larger than 5%-7%. Your numbers indicate 50% slower, which simply does not match up with our results. Check if the project compiles in Release and you have debug="false" set in web.config.
3. Your CPU utilization is a bit suspect. With 1000 concurrent requests the CPU should be utlized 100%. (Even only 8 concurrent requests should be enough since you have 8 virtual cores)
4. Your tests are running for about 20-25 seconds. That's a bit on the low side because a short (1-2 seconds) burst of activity elsewhere in the system could throw off your results quite significantly.
5. Related to point 4, did you run each scenario once or a few times? Are you seeing much variance in the results? Since your OS is doing other things in the background it's typical to see different results between runs.

Ropedancer

thank you very much Marcind, #2 solved the problem, I've compiled in release mode in all tests but didn't set debug=true in web.config. Setting it false resolved the issue. I'm updating my question about this and also post a comment to haacked's blog. rpc in razor is 3940 after that chage. – Mere

No problem. You have to be careful when measuring perf. – Ropedancer

Answer 2

How did you perform the benchmark? Was your site deployed on IIS in mode Release? Did you use the <deployment retail="true" /> section in your machine.config? Also remember that ASP.NET MVC 3 is still under heavy development so you cannot expect it to be fully optimized yet. At least wait until it hits RTM.

Messmate

yes in release mode on iis 7.5. I've tested with ab (Apache benchmark) and run 50.000 requests with 500 - 1000 clients. I've tested it two times and these numbers are the second results (first ones are lower). I know it's under development but not sure 3x performance gain is possible after optimization. – Mere

Why are you sure that 3x optimsation is not possible? The ASP.NET team have already stated that the Razor parser is not optimised yet. – Consubstantiate

Beta 1 released and razor rps dropped to 1550. – Mere

@Mere we are aware of certain performance issues with the current Beta bits. Did you rerun your tests for all 3 configurations or just Razor on MVC 3 (if the machine is under a different load then runs might not be comparable). Is your test app really just a simple Hello World page? I find it surprising that the difference is so big between Razor and ASPX, considering that when you have a very simple page the time spent rendering is relatively small. – Ropedancer

it was jus a simple hello world test and cpu usage was approx. %40 on a 4 core i7 for razor. I couldn't remember aspx cpu usages. I'm running all tests in the same conditions on a windows 7 and sample applications were same empty projects. I'll make a fresh new test for all and return to you with more detail. – Mere

I might suggest running some more complex examples, too. It may make things look more even or less, but it'd at least be more realistic – Ragouzis

@Michael: an empty project with a single controller and a page shows the minimum overhead of the system, there is no problem with my benchmark in my opinion. – Mere

@sirmak, I'm suggesting you add to this by having a complex example, too. The view engines may not scale as you'd expect – Ragouzis