Tokioruntime

Why Tokio

Async Rust done right: the runtime that powers the ecosystem

v1.1·10 min read·Kenneth Pernyér

tokiorustasyncruntimeconcurrency

The Problem

Modern services are I/O-bound. They wait for databases, external APIs, file systems, and network responses. Traditional threading models allocate a thread per connection—expensive and limited.

Async is the answer, but Rust requires a runtime.

Rust's async/await syntax produces state machines, not threads. But those state machines need an executor to poll them and a reactor to handle I/O events. Rust's standard library doesn't include these.

We needed an async runtime that:

Scales to thousands of concurrent connections
Integrates with the ecosystem (Axum, Tonic, sqlx)
Provides work-stealing for balanced CPU utilization
Is production-proven at scale

Current Options

Option	Pros	Cons
TokioThe de facto standard. Multi-threaded, work-stealing, battle-tested.	Ecosystem standard Work-stealing scheduler Excellent documentation Production-proven at massive scale	Runtime overhead vs sync code Colored function problem (async is viral) Debugging can be complex
async-stdMirrors standard library APIs.	Familiar API Simpler for standard library users Good documentation	Smaller ecosystem Less adoption Fewer libraries support it
smolMinimal async runtime.	Tiny dependency footprint Simple implementation Easy to understand	Fewer features No work-stealing by default Manual ecosystem integration

Future Outlook

Tokio has won the async Rust runtime race.

Ecosystem convergence is complete.

Axum, Tonic, sqlx, reqwest, hyper—the major async libraries are built on Tokio. Using anything else means fighting the ecosystem.

Work-stealing scales.

Tokio's scheduler distributes work across cores automatically. No manual thread pool tuning. Tasks migrate to idle threads, maximizing utilization.

The future is structured concurrency.

Tokio's task spawning model, combined with libraries like tokio-util, enables structured concurrency patterns that make complex async code manageable.

Our Decision

✓Why we chose this

Ecosystem standardAxum, Tonic, sqlx, hyper, reqwest—all built on Tokio. Integration is seamless.
Work-stealing schedulerTasks automatically balance across CPU cores. No manual tuning needed.
Production-provenCloudflare, Discord, AWS—Tokio runs at massive scale.
Rich utilitiesChannels, sync primitives, time utilities, I/O helpers—batteries included.

×Trade-offs we accept

Async is viralOnce you go async, callers must be async. The coloring spreads.
Debugging complexityAsync stack traces are harder to read than sync ones.
Runtime overheadFor CPU-bound work, async adds overhead without benefit.

Motivation

Our services handle thousands of concurrent connections. Each request involves database queries, external API calls, and internal service communication. Thread-per-connection would exhaust resources.

Tokio lets us handle 10,000 connections with a small thread pool. Each await point releases the thread for other work. Memory usage stays constant regardless of connection count.

The ecosystem alignment sealed the decision. Every library we use—Axum for HTTP, Tonic for gRPC, sqlx for databases—is built on Tokio. There is no alternative that integrates as well.

Recommendation

Use Tokio as your async runtime. The ecosystem alignment makes this the only practical choice.

Configuration:

Multi-threaded runtime for services (default)
Current-thread runtime for simple scripts or tests
Work-stealing enabled by default

Patterns to adopt:

Spawn tasks for concurrent independent work
Use channels (mpsc, broadcast, watch) for communication
Leverage tokio::select! for racing futures
Use tower for middleware on services

Avoid spawning tasks for everything. Many operations work fine with sequential awaits. Spawn when you need true concurrency or background work.

Examples

src/main.rsrust

use tokio::net::TcpListener;
use tokio::io::{AsyncReadExt, AsyncWriteExt};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let listener = TcpListener::bind("127.0.0.1:8080").await?;
    println!("Listening on :8080");

    loop {
        let (mut socket, addr) = listener.accept().await?;
        println!("Connection from {}", addr);

        // Spawn task for each connection
        tokio::spawn(async move {
            let mut buf = [0; 1024];

            loop {
                let n = match socket.read(&mut buf).await {
                    Ok(0) => return, // Connection closed
                    Ok(n) => n,
                    Err(_) => return,
                };

                // Echo back
                if socket.write_all(&buf[..n]).await.is_err() {
                    return;
                }
            }
        });
    }
}

#[tokio::main] sets up the runtime. Each connection gets a spawned task. Thousands of connections share a small thread pool.

RustWhy Rust for Antifragile Systems Axum & TonicWhy Axum & Tonic ElixirWhy Elixir for the Ledger