Asynchronous IO

In any IO-bound application, CPU execution speed far outpaces disk and network IO operations. When code execution encounters blocking operations like file reads/writes or network requests, the thread must pause and wait for the IO operation to complete before proceeding.

This blocking behavior presents a significant problem: a single IO operation can halt the entire thread, preventing other code from executing. Traditional solutions involve multithreading or multiprocessing to handle concurrent requests—each user gets a dedicated thread, and when one thread blocks on IO, others continue processing.

However, thread-based concurrency has fundamental limitations. Systems cannot spawn unlimited threads, and context switching between threads carries substantial overhead. Excessive thread count leads to performance degradation as the CPU spends more time switching contexts than executing actual application code.

The Async IO Alternative

Asynchronous IO offers an elegant solution to this problem. Instead of blocking the current thread during an IO operation, the code issues the IO request and immediately continues to the next task. The system notifies the application when the IO operation completes, at which point it can process the result.

Consider this synchronous pattern:

do_some_code()
file_handle = open('/path/to/file', 'r')
content = file_handle.read()  # Thread blocks here
process_content(content)

The synchronous model cannot achieve true async behavior because execution waits for each IO operation.

An async model employs an event loop that continuously processes messages:

event_loop = get_event_loop()
while True:
    event = event_loop.get_event()
    process_event(event)

This message-driven approach mirrors how desktop applications handle UI events. A GUI thread processes keyboard and mouse events from a queue. When one event takes too long to process, the entire application appears frozen because the thread cannot attend to other queued events.

The async IO model solves the blocking problem diffferently: when encountering an IO request, the code sends the request without waiting, then immediately returns to process the next message. When the IO completes, a completion message arrives, and the handler processes the result. During the wait period, the main thread handles other messages instead of idling—this enables a single thread to manage multiple concurrent IO operations efficiently.

Coroutines

Before diving deeper into async IO, understanding coroutines is essential.

A coroutine (also called a micro-thread or fiber) is a computer program component that generalizes subroutines for non-preemptive multitasking. Traditional function calls form a clear hierarchy: function A calls B, B calls C, and execution unwinds back through the call stack.

Coroutine execution differs fundamentally. While a coroutine resembles a subroutine, it can suspend itself mid-execution to run other coroutines, resuming later from where it paused.

def routine_a():
    print('alpha')
    print('beta')
    print('gamma')

def routine_b():
    print('first')
    print('second')
    print('third')

When managed by a coroutine scheduler, execution might produce:

alpha
beta
first
second
gamma
third

Notice that routine_a never explicitly calls routine_b—the scheduler handles the context switches, creating execution flow that differs from traditional function calls.

Coroutine Advantages

Coroutines offer two significant benefits over multithreading:

Superior Efficiency: Since coroutine switching is program-controlled rather than OS-mediated, it avoids the expensive context switches required by threads. This advantage scales dramatically as concurrency increases.

No Locking Required: With only one thread executing, there's no race condition on shared resources. Coroutines can manage shared state through simple state checks rather than mutex locks, dramatically improving performance.

To leverage multiple CPU cores with coroutines, combine multiprocessing with coroutines—processes handle CPU-bound work while coroutines handle IO-bound concurrency.

Coroutines via Generators

Python implements coroutines through generators. Beyond iteration, generators can receive values from callers via the yield statement.

A classic producer-consumer pattern demonstrates this elegantly:

def consumer():
    status = ''
    while True:
        item = yield status
        if item is None:
            return
        print(f'[CONSUMER] Processing item: {item}')
        status = 'COMPLETE'

def producer(worker):
    worker.send(None)  # Start the generator
    count = 0
    while count < 5:
        count += 1
        print(f'[PRODUCER] Generating item {count}')
        result = worker.send(count)
        print(f'[PRODUCER] Worker response: {result}')
    worker.close()

consumer_instance = consumer()
producer(consumer_instance)

Output:

[PRODUCER] Generating item 1
[CONSUMER] Processing item: 1
[PRODUCER] Worker response: COMPLETE
[PRODUCER] Generating item 2
[CONSUMER] Processing item: 2
[PRODUCER] Worker response: COMPLETE
...

The workflow:

1. producer initiates the generator with send(None)
2. Each iteration sends items to consumer via send()
3. consumer processes items and returns results via yield
4. producer receives results and continues its loop
5. When producer finishes, it closes the generator

This cooperative multitasking model requires no locks—producer and consumer coordinate voluntarily.

Donald Knuth famously described coroutines as "subroutines on steroids":

"Subroutines are coroutines in disguise."

The asyncio Module

Python 3.4 introduced asyncio as a standard library for asynchronous programming. It provides an event loop and cooperative multitasking primitives.

The programming model centers on an event loop managing coroutine execution:

import asyncio

@asyncio.coroutine
def greet():
    print("Hello world!")
    # Non-blocking sleep simulates async IO
    await asyncio.sleep(1)
    print("Greetings complete!")

# Obtain the event loop
loop = asyncio.get_event_loop()
# Execute until the coroutine finishes
loop.run_until_complete(greet())
loop.close()

The @asyncio.coroutine decorator marks a generator as a coroutine. The yield from syntax allows calling another coroutine without blocking—execution yields to the event loop, which processes other ready coroutines.

Concurrent Coroutine Execution

Wrap multiple coroutines in tasks for concurrent execution:

import asyncio
import threading

@asyncio.coroutine
def delayed_greeting():
    print(f'Hello world! (thread: {threading.current_thread().name})')
    yield from asyncio.sleep(1)
    print(f'Goodbye! (thread: {threading.current_thread().name})')

loop = asyncio.get_event_loop()
coroutines = [delayed_greeting(), delayed_greeting()]
loop.run_until_complete(asyncio.wait(coroutines))
loop.close()

Output demonstrates same-thread concurrency:

Hello world! (MainThread)
Hello world! (MainThread)
(1 second pause)
Goodbye! (MainThread)
Goodbye! (MainThread)

Both coroutines execute on the main thread, interleaving their execution without blocking.

Network IO Example

Fetch multiple HTTP pages concurrently:

import asyncio

@asyncio.coroutine
def fetch_page(host):
    print(f'Fetching from {host}')
    connection = asyncio.open_connection(host, 80)
    reader, writer = yield from connection
    
    http_request = f'GET / HTTP/1.0\r\nHost: {host}\r\n\r\n'
    writer.write(http_request.encode('utf-8'))
    yield from writer.drain()
    
    lines = []
    while True:
        line = yield from reader.readline()
        if line == b'\r\n':
            break
        lines.append(line)
    
    writer.close()
    return lines

loop = asyncio.get_event_loop()
hosts = ['www.sina.com.cn', 'www.sohu.com', 'www.163.com']
tasks = [fetch_page(host) for host in hosts]
loop.run_until_complete(asyncio.wait(tasks))
loop.close()

The three network requests proceed concurrently on a single thread.

Modern Syntax: async/await

Python 3.5 introduced async and await keywords to simplify async code. The syntax changes are minimal but significantly improve readability:

Transformation rules:

• Replace @asyncio.coroutine with async def
• Replace yield from with await

Compare the same coroutine in both styles:

Legacy style:

@asyncio.coroutine
def greeting():
    print("Hello world!")
    result = yield from asyncio.sleep(1)
    print("Hello again!")

Modern style:

async def greeting():
    print("Hello world!")
    result = await asyncio.sleep(1)
    print("Hello again!")

The async/await syntax clearly communicates asynchronous intent, making code easier to read and maintain. Note that this syntax requires Python 3.5 or later—older versions must use the decorator-based approach.

Building HTTP Servers with aiohttp

While asyncio handles concurrent IO efficiently on the client side, its real power emerges in server applications. HTTP connections are IO-bound, so asyncio enables single-threaded servers to handle massive concurrent connections.

The aiohttp library provides an HTTP framework built on asyncio:

pip install aiohttp

A basic HTTP server handling two endpoints:

import asyncio
from aiohttp import web

async def home(request):
    await asyncio.sleep(0.5)  # Simulate processing delay
    return web.Response(text='<h1>Welcome Home</h1>')

async def greet_user(request):
    await asyncio.sleep(0.5)
    username = request.match_info['name']
    message = f'<h1>Hello, {username}!</h1>'
    return web.Response(text=message)

async def startup(server_loop):
    app = web.Application()
    app.router.add_get('/', home)
    app.router.add_get('/greet/{name}', greet_user)
    
    handler = app.make_handler()
    server = await server_loop.create_server(handler, '127.0.0.1', 8080)
    print('Server running at http://127.0.0.1:8080')
    return server

loop = asyncio.get_event_loop()
loop.run_until_complete(startup(loop))
loop.run_forever()

The startup function is itself a coroutine, demonstrating that async functions can await other async operations. The server binds to port 8080 and processes requests concurrently without threading.