Python Coroutines and Asynchronous I/O Explained

Concurrency vs Parallelism: Concurrency refers to multiple tasks sharing a single CPU during a time period, while parallelism means multiple tasks running simultaneously on different CPUs.

Synchronous vs Asynchronous: Synchronous calls wait for I/O completion before returning, whereas asynchronous cals return immediately without waiting.

Blocking vs Non-blocking: Blocking calls suspend the current thread, while non-blocking calls return immediately.

I/O Multiplexing Techniques

Key models for handling multiple connections:

  • Blocking I/O - Waits idle for data
  • Non-blocking I/O - Constantly polls for readiness
  • I/O Multiplexing (select/poll/epoll) - Efficiently monitors multiple sockets
  • Asynchronous I/O - Returns immediately and notifies when complete

Epoll excels with many low-activity connections, while select performs better with fewer highly-active connections.

Non-blocking HTTP Request Example


import socket
from urllib.parse import urlparse

def fetch_webpage(url):
    parsed = urlparse(url)
    conn = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    conn.setblocking(False)
    
    try:
        conn.connect((parsed.netloc, 80))
    except BlockingIOError:
        pass
        
    while True:
        try:
            conn.send(f"GET {parsed.path or '/'} HTTP/1.1\r\nHost:{parsed.netloc}\r\n\r\n".encode())
            break
        except OSError:
            pass
            
    response = b""
    while True:
        try:
            chunk = conn.recv(1024)
            if not chunk: break
            response += chunk
        except BlockingIOError:
            continue
            
    return response.decode().split("\r\n\r\n")[1]

Event Loop Implementation


from selectors import DefaultSelector, EVENT_READ, EVENT_WRITE

selector = DefaultSelector()
active_urls = []

class AsyncFetcher:
    def __init__(self, url):
        self.url = url
        self.conn = None
        self.response = b""
        
    def handle_connect(self, key):
        selector.unregister(key.fd)
        self.conn.send(f"GET {self.path} HTTP/1.1\r\nHost:{self.host}\r\n\r\n".encode())
        selector.register(self.conn.fileno(), EVENT_READ, self.handle_read)

    def handle_read(self, key):
        data = self.conn.recv(1024)
        if data:
            self.response += data
        else:
            selector.unregister(key.fd)
            active_urls.remove(self.url)
            print(self.response.decode().split("\r\n\r\n")[1])
            self.conn.close()

    def start(self):
        parsed = urlparse(self.url)
        self.host = parsed.netloc
        self.path = parsed.path or "/"
        self.conn = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
        self.conn.setblocking(False)
        
        try:
            self.conn.connect((self.host, 80))
        except BlockingIOError:
            pass
            
        selector.register(self.conn.fileno(), EVENT_WRITE, self.handle_connect)

def event_loop():
    while active_urls:
        events = selector.select()
        for key, _ in events:
            callback = key.data
            callback(key)

Coroutine Fundamentals

Coroutines address callback complexity by providing:

  • Better readability
  • Easier state management
  • Simpler error handling

Generator methods for coroutine control:


def data_processor():
    try:
        received = yield "READY"
        print(f"Processing: {received}")
    except Exception as e:
        print(f"Error: {e}")
        
gen = data_processor()
status = next(gen)  # Initialize
gen.send("Sample Data")  # Send data
gen.throw(Exception("Test Error"))  # Raise exception

Modern Async/Await Syntax


async def fetch_data(url):
    response = await make_async_request(url)
    return response

async def make_async_request(url):
    return f"Fake response from {url}"

coro = fetch_data("example.com")
try:
    result = coro.send(None)
    print(result)
except StopIteration:
    pass

Tags: python Coroutines async-io selectors generators

Posted on Tue, 08 Sep 2026 16:20:02 +0000 by Frederick