Benchmark Linq2SQL, Subsonic2, Subsonic3 - Any other ideas to make them faster?
Asked Answered
T

5

7

I am working with Subsonic 2 more than 3 years now...

After Linq appears and then Subsonic 3, I start thinking about moving to the new Linq futures that are connected to sql.

I must say that I start move and port my subsonic 2 with SubSonic 3, and very soon I discover that the speed was so slow thats I didn't believe it - and starts all that tests.

Then I test Linq2Sql and see also a delay - compare it with Subsonic 2.

My question here is, especial for the linq2sql, and the up-coming dotnet version 4, what else can I do to speed it up ? What else on linq2sql settings, or classes, not on this code that I have used for my messures

I place here the project that I make the tests, also the screen shots of the results.

How I make the tests - and the accurate of my measures.

I use only for my question Google chrome, because its difficult for me to show here a lot of other measures that I have done with more complex programs. This is the most simple one, I just measure the Data Read. How can I prove that. I make a simple Thread.Sleep(10 seconds) and see if I see that 10 seconds on Google Chrome Measure, and yes I see it.


(source: planethost.gr)

here are more test with this Sleep thead to see whats actually Chrome gives.

10 seconds delay
100 ms delay
Zero delay

There is only a small 15ms thats get on messure, is so small compare it with the rest of my tests that I do not care about.

So what I measure

I measure just the data read via each method - did not count the data or database delay, or any disk read or anything like that. Later on the image with the result I show that no disk activity exist on the measures
See this image to see what really I measure and if this is correct

Why I chose this kind of test

Its simple, it's real, and it's near my real problem that I found the delay of subsonic 3 in real program with real data.

Now lets tests the dals

Start by see this image I have 4-5 calls on every method, the one after the other.

The results are. For a loop of 100 times, ask for 5 Rows, one not exist, approximatively..

Simple adonet:81ms
SubSonic 2 :210ms
linq2sql :1.70sec
linq2sql using CompiledQuery.Compile :239ms
Subsonic 3 :15.00sec (wow - extreme slow)

The project http://www.planethost.gr/DalSpeedTests.rar


Can any one confirm this benchmark, or make any optimizations to help me out ?

Other tests

Some one publish here this link http://ormbattle.net/ (and then remove it - don not know why) In this page you can find a really useful advanced tests for all, except subsonic 2 and subsonic 3 that I have here !

Optimizing

What I really ask here is if some one can now any trick how to optimize the DALs, not by changing the test code, but by changing the code and the settings on each dal. For example...

Optimizing Linq2SQL

I start search how to optimize Linq2sql and found this article, and maybe more exist.

Finally I make the tricks from that page to run, and optimize the code using them all. The speed was near 1.50sec from 1.70.... big improvement, but still slow.

Then I found a different way - same idea article, and wow ! the speed is blow up. Using this trick with CompiledQuery.Compile, the time from 1.5sec is now 239ms. Here is the code for the precompiled...

Func<DataClassesDataContext, int, IQueryable<Product>> compiledQuery =
    CompiledQuery.Compile((DataClassesDataContext meta, int IdToFind) =>
                          (from myData in meta.Products
                           where myData.ProductID.Equals(IdToFind)
                           select myData));

StringBuilder Test = new StringBuilder();
int[] MiaSeira = { 5, 6, 10, 100, 7 };

using (DataClassesDataContext context = new DataClassesDataContext())
{
    context.ObjectTrackingEnabled = false;

    for (int i = 0; i < 100; i++)
    {
        foreach (int EnaID in MiaSeira)
        {
            var oFindThat2P = compiledQuery(context, EnaID);

            foreach (Product One in oFindThat2P)
            {
                Test.Append("<br />");
                Test.Append(One.ProductName);
            }
        }
    }
}

Optimizing SubSonic 3 and problems

I make many performance profiling, and start change the one after the other and the speed is better but still too slow. I post them on subsonic group but they ignore the problem, they say that everything is fast...

Here is some capture of my profiling and delay points inside subsonic source code

I have end up that subsonic3 make more call on the structure of the database rather than on data itself. Needs to reconsider the hole way of asking for data, and follow the subsonic2 idea if this is possible.

Try to make precompile to subsonic 3 like I did in linq2Sql but fail for the moment...

Optimizing SubSonic 2

After I discover that subsonic 3 is extreme slow, I start my checks on subsonic 2 - that I have never done before believing that is fast. (and it is)

So its come up with some points that can be faster. For example there are many loops like this ones that actually is slow because of string manipulation and compares inside the loop. I must say to you that this code called million of times ! on a period of few minutes ! of data asking from the program.

On small amount of tables and small fields maybe this is not a big think for some people,

but on large amount of tables, the delay is even more. So I decide and optimize the subsonic 2 by my self, by replacing the string compares, with number compares! Simple. I do that almost on every point that profiler say that is slow. I change also all small points that can be even a little faster, and disable some not so used thinks.

The results, 5% faster on NorthWind database, near 20% faster on my database with 250 tables. That is count with 500ms less in 10 seconds process on northwind, 100ms faster on my database on 500ms process time. I do not have captures to show you for that because I have made them with different code, different time, and track them down on paper.

Anyway this is my story and my question on all that, what else do you know to make them even faster...

For this measures I have use Subsonic 2.2 optimized by me, Subsonic 3.0.0.3 a little optimized by me, and Dot.Net 3.5

Thorne answered 2/3, 2010 at 14:24 Comment(12)
Seems pretty clear that there is some problem with the Subsonic 3 test. A misconfiguration, perhaps?Thriftless
Any help appreciate for subsonic 3 ! - I have ask also the creator but did not wish to comment that speed.Thorne
Your question is far too vague to get any helpful answers. You need to post a simple example of what exact piece of your code is running slowly. Preferably one that others can take and run themselves for comparison. Until you can do that I can't see how anyone can answer this question helpfully for you.Digamma
Dera Adam, I have upload the project, the source code is inside and its very simple, please take the time and download it to see it by your self.Thorne
If you can't post the exact code into the question then I can't help you. Your benchmark isn't a benchmark and you have no idea where the bottleneck is in your code. Surprisingly I don't have the time to download setup and debug your project for you.Digamma
Adam its ok I post the soure code. I like to say you that this is a simple test, I have work with that more than 2 weeks, and make speed profiling everything try to find the bottleneck. Actually you have right is not a benchmark, but a compare betwing them.Thorne
i use subsonic ALOT with mysql, i used to use my own class and stored procedures and one of the first thing i noticed with subsonic and mysql is the extra time it takes i did approach this a number of times myself however rob assured me that the code itself is quick, so have strugged it off, but i can only speak of working with mysql. my DB is on its own servers and the web are on the same switch just 1 hop its strange i did a step through once and it to and from database was fast but seemed to build twice when running queries etc but im not experience enough to actually comment on qualityUphill
Dear minus4, you use subsonic 2 or 3 ?. "rob assured me that the code itself is quick" Very soon I write a more detailed article, and say how can make Subsonic 2 really super fast !, rob avoid even to discuss this, do not know why. Subsonic 3 is super slow, and for the moment can not make it fast enough.Thorne
i use subsonic 3.0.0.3 most latest version but with a few amends of my own but nothing superfast i spend most of my time on optomising the DB server and using Cache on my MVC projects now to speed things upUphill
Minus4, just make a test by your self, change the subsonic 3 with subsonic 2, or linq and precompile thinks like my example above, and check again the speed.Thorne
i dont have the time to be that worried or bothered really too much work on, clients are happy so i leave it at that if clients compain then it would become an issueUphill
have you tried this with a remote or local DB ??????????????Uphill
M
3

Aristos, here's the thing I'm having an issue with. You posted a question to our groups and we had a nice, 23-long email exchange. You insisted that SubSonic 3 has problems and you asked me why I haven't fixed this "slow, broken tool".

I tried to explain to you that, with 233 tables, SubSonic has to read the schema of EACH ONE on load up. This isn't typical usage for SubSonic 3, but we do it this way so it's actually FASTER than SubSonic 2 on subsequent loadups.

You're ingoring this. I'm answering you. RBarry left a comment above that I was beind "denigrating". I'm quite frustrated. You seem to think that I haven't benchmarked this thing and I have, MANY MANY times. I can't turn out an ORM with problems like the one you suggested, and I didn't.

You have to understand that if it took SubSonic 10 seconds to run a query, then it wouldn't be used.

So - your answer: 233 tables, scripted out as classes, need to load to Provider memory on first run.

That's too many. SubSonic isn't your tool.

Middlebreaker answered 2/3, 2010 at 14:24 Comment(4)
Dear Rob, I have say to you a lot of times, I have 4 database, only one have 233 tables the other have 10 tables only. The delay is even in those with 10 tables. All my work and my reports is to help out to find out why the delay, and how to fix it, or tell to me how to correct it. But please I say to you again, the tests here is with Northwind database.Thorne
Rob, I must give you credits and my gratitude for both subsonic 2 and 3. Subsonic 2 is very good, fast, easy to use, and after some more optimizes is even faster (why you do not won the source code of this extra optimizes ?). I start work with subsonic 3 that it is a lot of easier and very cool and very nice, I like it - how ever need work with the speed, accept it and think how to fix it. Actually I like to ask you, what tool do you have use to measure the speed of both of them ? Do you see the screen captures of my speed profiling ? Do you see the actual delays inside subsonic 3 ?Thorne
Rob, The very interesting think here is that you still denied the problem, and you do not try to see what you do to improve your program. What do you think ? that this delay is not exist ? this is a rare case ? is my fault ?. Why not accept it and try to fix it ? Why not try to improve it ? You never going to make a perfect tool this way by avoid the critic, or say to anyone, "the subsonic isn't your tool" just because they find bugs and improvements thats needs. Rob, I do not have anything with you, as I say Subsonic 2 is super cool and recommended, but subsonic 3 needs work.Thorne
Submit a patch or go away. I'm growing a bit weary of your obsessive brow-beating.Middlebreaker
I
1

I think to get a better Benchmark on these, you should check the time in sql profiler not in google chrome, because in the browser it may be a lot of things affect the speed on the page load.

Inkhorn answered 2/3, 2010 at 14:29 Comment(6)
I have disable all google chrome extensions, how ever if this extra time exist, then is the same for all tests.Thorne
this means that if you refreshed the page for one ORM you will get the same number, is this the result?Inkhorn
yes ! you get the same number - I have refresh the page 1 time for compile, then 3 times to see the number is the same, then I get the capture.Thorne
it's ok to asking. For me its very critical the speed of the tool that I am going to use, so I need to know, and share this experience with others, and maybe found any mistakes that I have made. The right tool, is critical because I start learn it, spend time with it, and after months and years of developing if I discover that I have made mistake, then to go back and use something else is painful.Thorne
Dear Amr, I have update the article, to prove that google chrome actually count what I ask for, also I have include more measures and more details.Thorne
thanks you for supporting us with these info, really helpful.Inkhorn
M
1

Your speed test is a web page that, to my eyes, looks like your testing how long it takes to load. SubSonic isn't that slow and testing load time in a web page is rather ridiculous.

If you want to truly benchmark something you need to use a Console and run read loops that work against indexed data. What you've done is create a web page and say "let's see what happens on load".

This is flawed for a number of reasons. First - all of the code written needs to be compiled from IL do ML. Because SubSonic generates your code for you, and because I know you have a lot of tables here (300+ if I remember right), you can imagine there's some work going on under the covers on your first load.

To be perfectly honest here - your inexperience is undoing a lot of work I've put in for free, by posting things like "it's slow and not ready". I don't care if people use SubSonic - I do care when people do dumb things (like benchmark web tests) and blame me for it.

Middlebreaker answered 2/3, 2010 at 17:43 Comment(9)
Dear Rob, the speed I measure is the TIME TO TAKE TO GET THE DATA. The time to load the page is the next one thats follows. The example is here, please get it and feel free to make a new example, or improve this one.Thorne
Rob, take a moment to make some tests by your self, download chrome to see how it works and what its shows. There is a very easy way to show what is slow and what is not - how ever some one not need chrome to measure the speed, because the different are very big and you can feel it right way - that the way I understand that something going slow and start make tests.Thorne
This example is using the Northwind database ! If you ever take the time to download and tell me whats really wrong !Thorne
your inexperience is undoing a lot of work I've put in for free - the free is not an excuse to get anything that is not work. You are very experience, why not take it run faster ? I like to point here that you are not talk about the subject here that is the slow speed, you attact to me say that I am inexperience, and your second argument is that is free. Both argument even if they are right, they not change the speed I have post here !!!Thorne
Dear Rob, I am not blame you. Now you have a site, you speak a lot there, why not make yours benchmark, placing the source code like I did and post the results thats shows something different ? And if you do that and prove the I am wrong, I am take all back, and give you credits for that. After all I have all ready give you for subsonic 2 !!! that is super fast super good, and I work with it more than 3 years now - and I recommender and support it. I have also optimize subsonic 2 and make it run faster - if you wish you can get the source code. How ever until now you ignore me.Thorne
-1: denigrating an honest, intelligent and well-researched question because of your personal association with its subject. Inappropriate and unprofessional and the posters language difficulties are no excuse.Siffre
@Siffre Sorry but, this question is neither intelligent nor well-researched it's an ill conceived rant. Basically this guy's saying I've found a specific situation where x is slower than y therefore y is too slow and sucks.Digamma
Adam, I am practical person, I have see this delays on real program, real data, with real users... its not only this situation, I only have place here this test to give it as an idea.Thorne
Rob is correct here, SubSonic does have signifant startup time on lots of tables. This should really be a non issue for any web application as far as I see it. 15 second delay to start a web app that runs on the server for who knows how long. You really should adjust the benchmark to count after each provider returns it's first query.Commodious
D
1

Based on your example you can improve the performance by using the following code:

 StringBuilder Test = new StringBuilder();
 int[] MiaSeira = { 5, 6, 10, 100, 7 };
 for (int i = 0; i < 100; i++)
 {
     foreach (int EnaID in MiaSeira)
     {
         var Products = (from product in Product.
             where MiaSeira.Contains(product.ProductID)
             select product).ToList();

         if (Products == null || Products.Count == 0)
             continue;

         foreach (Product product in Products)
         {
            Test.Append("<br />");
            Test.Append(product.ProductName);
         }
     }
 }

 txtDebug.Text = Test.ToString();
Digamma answered 3/3, 2010 at 21:1 Comment(3)
Adam, thank you for your time to give me this but actually something else I have on my mind. So I update the article giving a better idea.Thorne
So I offer you a solution and you immediately change your question, what do you have in mind now?Digamma
I think Products will never be null when used like this, and instead of using Products.Count you could use Products.Any() which returns true of there are records in the list, false otherwise.Gizzard
A
0

I don't have firsthand experience with subsonic, but if it is doing a bunch of code generation, you might want to post separate results for the first request (cold), and an average of subsequent requests (warm). You should also use a Stopwatch to time just the data extraction part, so you don't conflate other timings into your data.

Amatory answered 4/3, 2010 at 17:11 Comment(2)
Ok, I will do that, please give me some time.Thorne
Jimmy I have update the article, including now capture of 4-5 requests in a row, and many other infosThorne

© 2022 - 2024 — McMap. All rights reserved.