Using SQLite with async in C#
Asked Answered
D

4

6

I am trying to get my head around the async/await keywords and usage, and I think I got the basics. But something isn't working right in my SQLite code.

I am using the SQLite.core NuGet package in a simple project I've been working on. I noticed that the async code I wrote does not behave asynchronously (like I expected), so I created a simpler test project to test my understanding.

In my test code, I open a connection to an in-memory database (I have the same problem with a file-based database. In-memory was just easier in the test code), and issue a single "create table" command, using ExecuteNonQueryAsync. I don't immediately await for the result, but instead write something to the console, before finally using the await keyword.

I expect the console command to be executed before the ExecuteNonQueryAsync finishes, so in my test I should see "1 2 3 4". But instead I get "1 3 2 4"

I run an identical test using an SQL Server LocalDB connection (running the same code, only the DbConnection is different), and get the expected "1 2 3 4". So I guess my basic understanding of the async is not that far off from the mark.

What am I missing? Do I need to use a special connection string with the SQLite in order to support the async methods? Does it even support it?

My full test project can be found here.

And here is the main program itself:

 namespace DatabaseTest
   {
    using System;
    using System.Data.Common;
    using System.Data.SqlClient;
    using System.Data.SQLite;
    using System.Threading.Tasks;
class Program
{
    static void Main(string[] args)
    {
        Task.WaitAll(TestDatabase(true), TestDatabase(false));
    }

    private static async Task TestDatabase(bool sqLite)
    {
        Console.WriteLine("Testing database, sqLite: {0}", sqLite);
        using (var connection = CreateConnection(sqLite))
        {
            connection.Open();
            var task = ExecuteNonQueryAsync(connection);
            Console.WriteLine("2");
            await task;
            Console.WriteLine("4");
        }
    }

    private static DbConnection CreateConnection(bool sqLite)
    {
        return sqLite ?
            (DbConnection)new SQLiteConnection(string.Format("Data Source=:memory:;")) :
            new SqlConnection(@"Data Source=(LocalDB)\MSSQLLocalDB;AttachDbFilename=|DataDirectory|\DatabaseTest.mdf;Integrated Security=True;Connect Timeout=30");
    }

    private static async Task ExecuteNonQueryAsync(DbConnection connection)
    {
        var command = connection.CreateCommand();
        command.CommandText = "CREATE TABLE test (col1 integer);";
        Console.WriteLine("1");
        await command.ExecuteNonQueryAsync();
        Console.WriteLine("3");
    }
}

And the output:

Testing database, sqLite: True
1
3
2
4
Testing database, sqLite: False
1
2
3
4
Divisionism answered 27/12, 2015 at 6:20 Comment(0)
U
4

Once you have started an asynchronous task, both that task and the main thread can continue to run. So there is no guarantee which one runs faster.

SQLite is an embedded database with no client/server communication overhead, and, as a library, runs on the same CPU. So this implementation might have decided that actually supporting asynchronous execution would not make sense.

Umlaut answered 27/12, 2015 at 8:4 Comment(4)
Can you plz give me more information about this? really i didn't know why it happened .Fadge
"as a library, runs on the same CPU" How exactly is it guaranteed to run on the exact same CPU? a managed library has no such control.Cheri
@YuvalItzchakov There is no guarantee. But from the OS's point of view, a function call into a library is no reason to change this process's scheduling. (With SQL Server, the database code always runs asynchronously at the other end of a named pipe.)Umlaut
So basically, I "shouldn't worry about it", and write my code using await. If the SQLite decides it needs to use a different thread, I will get the benefits. If it runs "fast enough" I shouldn't care.Divisionism
D
16

System.Data.SQLite implementation is 100% synchronous. They don't have any async overloads, and that's Microsoft to blame for this misconception, as SQLiteCommand extends System.Data.Common.DbCommand with default implementations of *Async methods that just call synchronous versions:

/// <summary>This is the asynchronous version of <see cref="M:System.Data.Common.DbCommand.ExecuteNonQuery" />. Providers should override with an appropriate implementation. The cancellation token may optionally be ignored.The default implementation invokes the synchronous <see cref="M:System.Data.Common.DbCommand.ExecuteNonQuery" /> method and returns a completed task, blocking the calling thread. The default implementation will return a cancelled task if passed an already cancelled cancellation token.  Exceptions thrown by <see cref="M:System.Data.Common.DbCommand.ExecuteNonQuery" /> will be communicated via the returned Task Exception property.Do not invoke other methods and properties of the <see langword="DbCommand" /> object until the returned Task is complete.</summary>
/// <param name="cancellationToken">The token to monitor for cancellation requests.</param>
/// <returns>A task representing the asynchronous operation.</returns>
/// <exception cref="T:System.Data.Common.DbException">An error occurred while executing the command text.</exception>
public virtual Task<int> ExecuteNonQueryAsync(CancellationToken cancellationToken)
{
  ...
  return Task.FromResult<int>(this.ExecuteNonQuery());
  ...
}

I just figured that the same hard way, and I'm not happy with approach they took, but that's what we got. Just for the record, I think that there should be NotSupportedException.

Daguerre answered 19/9, 2018 at 17:11 Comment(3)
Thanks for this, I was wondering why ExecuteReaderAsync returned a DbDataReader.Underclassman
Same with Microsoft.Data.Sqlite implementation, async ADO.NET methods will execute synchronously learn.microsoft.com/en-us/dotnet/standard/data/sqlite/asyncBraeunig
Almost all ADO.NET database provider implementations are synchronous except the one for MS SQL Server, which is fully asynchronous! System.Data.SQLite is also synchronous because you can use Write Ahead Logging (sqlite.org/wal.html) which may be advised here also. Always use a transaction block to minimize thread blocking!Pesach
U
4

Once you have started an asynchronous task, both that task and the main thread can continue to run. So there is no guarantee which one runs faster.

SQLite is an embedded database with no client/server communication overhead, and, as a library, runs on the same CPU. So this implementation might have decided that actually supporting asynchronous execution would not make sense.

Umlaut answered 27/12, 2015 at 8:4 Comment(4)
Can you plz give me more information about this? really i didn't know why it happened .Fadge
"as a library, runs on the same CPU" How exactly is it guaranteed to run on the exact same CPU? a managed library has no such control.Cheri
@YuvalItzchakov There is no guarantee. But from the OS's point of view, a function call into a library is no reason to change this process's scheduling. (With SQL Server, the database code always runs asynchronously at the other end of a named pipe.)Umlaut
So basically, I "shouldn't worry about it", and write my code using await. If the SQLite decides it needs to use a different thread, I will get the benefits. If it runs "fast enough" I shouldn't care.Divisionism
P
2

Almost all ADO.NET database provider implementations are synchronous (asynchronous -> internal route to synchronous implementations) except the one for MS SQL Server, which is fully asynchronous!

The System.Data.SQLite .NET data provider is also synchronous, because you can use Write Ahead Logging (https://www.sqlite.org/wal.html) which may be advised here also. Always use a transaction block to minimize thread blocking (speeds up enormously)!

AFAIK synchronous .NET data providers:

  • all SQLite providers
  • Sybase (SQL Anywhere)
  • PostgreSQL
  • Oracle MySQL data provider

Asynchronous are:

  • MS SQL Server data provider living in System.Data
  • MySqlConnector for MySQL
  • DevArt commercial providers, but they don't have the TAP (task asynchronous pattern)

Maybe write your own extensions and put the method into Task.Run(() => Func()).

TIP: When you want inspect whether your ".NET data provider code" is running in a different thread than the main process thread, display the thread window in debug menu in Visual Studio or VS code.

Regards

Pesach answered 7/4, 2023 at 7:16 Comment(10)
"Maybe write your own extensions and put the method into Task.Run" -- No!Canso
@TheodorZoulias, from that article - " If a developer needs to achieve responsiveness or parallelism with synchronous APIs, they can simply wrap the invocation with a method like Task.Run.".. if it were OP's code, then I agree with you - however with a synchronous Sqlite API, then if you want app responsiveness, you haven't got much choice!Hereinbefore
@Hereinbefore I am not against using Task.Run. I am against exposing an API that pretends to be genuinely asynchronous when it's actually not. The Task.Run should be used in application code, not in library code. Don't hide it, make it visible. For example use it directly inside your async void button_Click event handler, not in async methods that are called by the handler. That's what Microsoft's article recommends.Canso
@TheodorZoulias, aha, that's an important distinction and I agree with you entirely!Hereinbefore
@TheodorZoulias: Nobody talked about exposing an API. AFAI understood the guy asking just wanted to use provider code client side!Pesach
Martin you said "Maybe write your own extensions and put the method into Task.Run(() => Func())." This means exposing an API. A method is an API. Microsoft advises against hiding the Task.Run inside methods. At least that's how I understand it.Canso
NO! Exposing an API (application binary interface) means to make it public for others! Did is say write a library and publish it? Extensions don't necessarily mean to make them public (you can write private or internal once)! Don't read things who don't stand there! It' a very common mistake of humans!Pesach
@Theodor Zoulias: And do me a favor! Don't link to others ppls opinion: build your your own and make it here public. Linking to other ppls opinions mean "i don't really think myself"!Pesach
Martin I've linked to a blog post in Microsoft's site, written by a top Microsoft engineer (Stephen Toub), who is writing on a topic of his expertise. So it's not just an opinion, it's a guideline. I have read this blog post multiple times, and I agree with everything that is written there. Having my own opinion doesn't mean that I must have a different opinion. I am allowed to have the same opinion with someone else.Canso
I suggest to edit your answer and expand the "Maybe write your own extensions and put the method into Task.Run(() => Func())." suggestion to a complete example of such an extension, so that people have a clear view of what you are actually suggesting here, and evaluate (vote) your answer accordingly.Canso
V
1

This is an old post with a lot of views so I want to clarify something here. Tasks in C# NEVER guarantee parallelism on their own. The task scheduler or synchronization context might run that work in the same thread, for example. Besides, the method call will run synchronously until a call to Task.Run or some IO completion handler is registered within it, which could be many levels deep in a call stack. Don’t depend on it - if you require the code to be backgrounded, use Task.Run. Generally you don’t want to do that in library code but you know your situation best.

Regardless, Microsoft documents that their SQLite library is synchronous (see the remarks of their documentation.

Vauntcourier answered 19/1 at 7:21 Comment(0)

© 2022 - 2024 — McMap. All rights reserved.