Varying OleDb performance?
Asked Answered
L

3

7

I'm currently trying to use OleDb/JET to export SQL Server data to an Excel 2003 file. I initially coded the export in a console application, and it worked fast enough. However, running the same code during an ASP.Net request takes roughly three times longer. Specifically, it's slowing down during a call to OleDbCommand.ExecuteQuery, which is inserting the record to the Excel file.

The only difference between the two is literally that one is running in a console app and the other is running in IIS. Other than that, both applications are:

  • running on my development machine
  • running the same code as below from a shared dll that both applications reference
  • connecting to the same database with the same connection string
  • using the same exact select statement with the same exact parameter values
  • creating a brand new file in the same location on my development machine and writing to it
  • generating byte-for-byte identical files

Is there something inherently slow when using OleDb in ASP.Net that I'm unaware of?

UPDATE: This is the code in question. The import command used in both the console application and ASP.Net sites are identical. They are both connecting to a database that is local to my machine, and both applications are running on my machine.

public void Convert(IDbCommand importCommand, string savePath, string sheetName)
{
    var excelConnString = new OleDbConnectionStringBuilder();
    excelConnString.Provider = "Microsoft.ACE.OLEDB.12.0";
    excelConnString.DataSource = savePath;
    excelConnString["Extended Properties"] = "Excel 8.0;HDR=Yes";

    using (var dr = importCommand.ExecuteReader())
    {
        var columnCount = dr.FieldCount;

        using (var oleConn = new OleDbConnection(excelConnString.ToString()))
        {
            oleConn.Open();
            var headers = new string[columnCount];
            var formattedHeaders = new List<string>();
            var qs = new List<string>();

            var insertCmd = oleConn.CreateCommand();

            for (var curCol = 0; curCol < dr.FieldCount; curCol++)
            {
                var name = dr.GetName(curCol);
                headers[curCol] = name;
                formattedHeaders.Add("[" + name + "]");
                qs.Add("?");
                insertCmd.Parameters.Add(name, OleDbType.LongVarChar, 20000);
            }

            using (var cmd = oleConn.CreateCommand())
            {
                cmd.CommandText = string.Format("create table {0}({1})", sheetName,
                                                string.Join(",", formattedHeaders.Select(x => x + " longtext")));
                cmd.ExecuteNonQuery();
            }

            insertCmd.CommandText = string.Format("insert into {0} ({1}) values ({2})", sheetName, string.Join(",", formattedHeaders), string.Join(",", qs));
            insertCmd.Prepare();

            var values = new object[columnCount];

            while (dr.Read())
            {
                dr.GetValues(values);
                for (var i = 0; i < columnCount; i++)
                {
                    insertCmd.Parameters[headers[i]].Value = values[i];
                }
                insertCmd.ExecuteNonQuery();
            }
        }
    }
}
Lalittah answered 9/3, 2012 at 22:24 Comment(14)
a wild guess - if the console app only writes to the Excel file, but ASP.Net opens it in such a mode that there are recalculations / refreshes inside Excel...Sokil
@deathApril: In both cases, they're creating a new file each time. The files are identical. I'll update my question to mention that.Lalittah
do they use the same COM method with the same parameters to create and write to an Excel file?Sokil
As I mentioned in the question, they literally are running identical code with identical data. The code is in a third C# dll that's referenced by each project.Lalittah
evidently they are not running identical code, if there is performance difference.. but sorry i don't know how to help you more :(Sokil
I appreciate the help, but "evidently"? No need to be glibLalittah
evidently = "according to the evidence available"Sokil
Just to be sure: how do you measure execution time? There are various overhead layers in a http request - have you excluded those?Mince
Olaf: In the code example, I had used a bunch of Stopwatches that measured and averaged a few things(how long it took to complete the whole conversion, how long it took to import a single row from sql to excel, how long it took to run ExecuteNonQuery, etc). I removed them just to keep the code short. The HttpRequest and what not weren't part of the measurement.Lalittah
@Lalittah where are you calling this from ? from an aspx page - from a handler ? or from a thread on background. If you called from a page then the issue is your session - disable it test it, and tell me to make this an answer.Personable
@Aristos: It's coming from a button handler on an aspx page. I can't test it at the moment(not at work), but why would the Session object have any impact on it when I'm not calling the Session for anything?Lalittah
Are the build configurations identical for both? Regarding DEBUG or TRACE constants, conditional compilation symbols, platform target, code optimization, arithmetic overflow release/debug mode, unmanaged code allowed or not, both 32bit (the dll should be because of OleDb, but is the calling web?)... let's look for an overlooked difference.Mince
@Olaf: Checking just now, the only difference was the web project being set to Any CPU and having the DEBUG constant checked. Switching it to match the other project didn't have any effect. Good thought, though!Lalittah
FWIW, I had a similar issue in doing batches of Excel imports via ASP.Net and resolved to the fact that ACE appears to have to load quite a bit of out-of-process code. So what? Except for that the security context of my console app was me - IIS was anon, when I changed the app pool to run as me there was a significant change. Not equal, but a noticable difference.Decurrent
L
3

Sounds like you need a profiler.

It's probably worth noting that Microsoft tell you not to use ACE in ASP .NET. Perhaps they know something about the implementation that's not documented?

The Access Database Engine 2010 Redistributable is not intended ... (to) be used by a system service or server-side program where the code will run under a system account, or will deal with multiple users identities concurrently, or is highly reentrant and expects stateless behavior. Examples would include a program that is run from task scheduler when no user is logged in, or a program called from server-side web application such as ASP.NET, or a distributed component running under COM+ services.

But if you're going to continue down this road, I would suggest getting rid of your application as the middleman and having the INSERT query the source data directly. ACE should support this syntax:

SELECT *
FROM ExternalTable IN '' [ODBC;Driver={SQL Server}; Server=ServerName; Database=DatabaseName; Trusted_Connection=Yes]
Lottery answered 21/3, 2012 at 11:4 Comment(1)
I'm awarding you the bounty cause it seems like it's the most likely culprit, though I've been unable to test this out.Lalittah
P
0

After your comment that this is called from a button handler on an aspx page I think that the extra delay come from the session lock.

Even if you not use session the session lock entire all request until this work finish and the page ends. This lock can delay the request and if you make two or three call to the same routing the one lock the other, but also if you see other pages the session lock add extra delay.

To avoid this session lock just disabled on the page declaration by set EnableSessionState="false"

Look also at

call aspx page to return an image randomly slow

Replacing ASP.Net's session entirely

Personable answered 17/3, 2012 at 15:13 Comment(2)
I get what you mean about session lock, but I don't see how that pertains to my issue? I'm not measuring the time for the request, only the amount of time it takes to create the excel file itself. The session, or even other people making requests, should have no baring on this, especially since I'm the only one making requests to the server, and there's only ever one going on at a time.Lalittah
@Lalittah set EnableSessionState=false, make the test, and then we talk again...Personable
B
0

Guess this might clear your thoughts

performance pitfalls

asp-net-slower-than-console-application

Barbel answered 21/3, 2012 at 10:56 Comment(0)

© 2022 - 2024 — McMap. All rights reserved.