Is tokio multithreaded?

Question

I know tokio allows to write concurrent code. But I'm not sure if it runs in parallel. My computer has eight cores. So ideally I would run no more than eight threads. If I needed more concurrency I would run coroutines on top of those threads (using tokio).

Unless of course, tokio was already multithreaded. In that case, creating those eight threads in the beginning would be counterproductive. So what I am trying to ask is, is tokio multithreaded by default, or is that something I should implement myself?

Cris

Answer 1

Yes. Tokio is multi-threaded. By default, it creates as many worker threads as there are cores. You can customize how many worker threads the runtime creates via the tokio::main macro. Example:

#[tokio::main(flavor = "multi_thread", worker_threads = 10)]
async fn main() {
    // your code here
}

Gametocyte

Answer 2

Tokio does not directly use multiple threads for concurrent execution. Instead it relies on the operating system's support for asychronous I/O, such as the epoll, kqueue and IOCompletionPort APIs on Linux, macOS and Windows, respectively. These APIs allow the operating system to multiplex the execution asynchronous tasks over a single thread over an event loop.

However, Tokio provides abstraction for spawning tasks (threads) automatically or manually that execute concurrently. These taskes are useful to handle blocking IO-bound tasks - the tasks will block the caller from returning until the IO has completed.

Note: Asynchronous tasks are non-blocking, i.e., the tasks will return immediately even the function is not yet completed. An underlying event loop is used to handle completed asynchronous tasks.

Tokio provides multiple variations of the runtime:

1. Multithreaded

#[tokio::main(flavor = "multi_thread", worker_threads = 10)]
async fn main() {
     // Your code here
}

In this example, the runtime will create 10 worker threads in addition to the main thread, and it will use these worker threads to execute tasks concurrently.

2. Single-threaded (i.e., current thread)

#[tokio::main(flavor = "current_thread")]
async fn main() {
     // Your code here
}

The main() function executes asynchronous code over an event loop. However, it is possible to spawn new asynchronous tasks that will be executed concurrently with the main task.

Spawning new tasks are useful to distribute blocking IO-bound tasks (spending most of its time waiting for IO to complete) over several tasks manually. For examples:

use tokio::net::TcpListener;

#[tokio::main]
async fn main() {
    let listener = TcpListener::bind("127.0.0.1:6379").await.unwrap();

    loop {
        // Asynchronous code for the main task goes here
        let (socket, _) = listener.accept().await.unwrap();

        // A new task is spawned for each inbound socket.
        // The socket is moved to the new task and processed there.
        tokio::spawn(async move {
            process(socket).await;
        });
    }
}

Latinalatinate

Would your last example run the tasks in separate thread? Is there a way to specify with each spawn whether you want it on a new thread or concurrently on the current thread? – Madlin

Isn't the statement "Tokio does not directly use multiple threads for concurrent execution" contradicted by the first example? The example shows how "Tokio uses multiple threads for concurrent execution" – Aeromedical