I/O multiplexing allows a program to monitor multiple file descriptors simultaneously, significantly enhancing performance in network applications.

Common scenarios requiring I/O multiplexing include:

Client-side:

• Handling multiple sockets simultaneously, such as in non-blocking connect implementations.
• Managing user input alongside network connections, for example, in chat room applications.

Server-side:

• TCP servers handling both listening sockets and connection sockets (the most common use case).
• Servers processing both TCP and UDP requests concurrently.
• Servers monitoring multiple ports or handling multiple services.

Blocking Nature: Its important to note that while I/O multiplexing watches multiple file descriptors, the mechanism itself is blocking. When multiple descriptors become ready, the program typically processes them sequentially unless additional concurrency measures are taken. To achieve true parallelism, multi-processing or multi-threading is required.

Linux provides several system calls for I/O multiplexing: select, poll, and epoll.

The select System Call

The select system call monitors specific file descriptors for readability, writability, or exceptional conditions over a specified timeout period (handled by the kernel).

select API

#include <sys/select.h>

int select(int nfds, fd_set *readfds, fd_set *writefds, fd_set *exceptfds, struct timeval *timeout);

1. nfds: The maximum file descriptor value plus one. Since descriptors start at 0, this tells the kernel how many bits to scan.
2. readfds, writefds, exceptfds: Pointers to descriptor sets for read, write, and exception events. The application passes its interested descriptors in; on return, the kernel modifies these sets to indicate which descriptors are ready. These are of type fd_set.

The fd_set structure is essentially a bitmask array where each bit represents a file descriptor. The size is limited by FD_SETSIZE, typically 1024.

#include <typesizes.h>
#define __FD_SETSIZE 1024
#include <sys/select.h>
#define FD_SETSIZE __FD_SETSIZE

// Simplified representation of the internal bit array
typedef struct {
    long int __fds_bits[FD_SETSIZE / (8 * sizeof(long int))];
} fd_set;

Instead of manual bit manipulation, use these macros:

FD_ZERO(fd_set *set);       // Clear all bits
FD_SET(int fd, fd_set *set);   // Set bit for fd
FD_CLR(int fd, fd_set *set);   // Clear bit for fd
FD_ISSET(int fd, fd_set *set); // Check if bit for fd is set

3. timeout: Specifies the waiting interval. A NULL pointer causes indefinite blocking. A zero-timeval causes immediate return (polling).

struct timeval {
    long tv_sec;  // seconds
    long tv_usec; // microseconds
};

Return Value:

• On success: Number of ready descriptors.
• If timeout expires: Returns 0.
• On error: Returns -1 (e.g., interrupted by a signal EINTR).

Descriptor Ready Conditions

A socket is considered readable if:

• Receive buffer data >= SO_RCVLOWAT.
• The peer closes the connection (read returns 0).
• A listening socket has pending connection requests.
• An unread error is present.

A socket is considered writable if:

• Send buffer space >= SO_SNDLOWAT.
• The write side is shut down (writing triggers SIGPIPE).
• A non-blocking connect completes (success or failure).
• An unread error is present.

Exceptional conditions are primarily used for receiving out-of-band (OOB) data.

Implementation Workflow

1. Initialize an fd_set and add target file descriptors.
2. Invoke select to block until events occur or timeout.
3. Upon return, iterate through descriptors to check status using FD_ISSET.

Server Example

#include <stdio.h>
#include <arpa/inet.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>
#include <sys/select.h>

#define SVR_IP "127.0.0.1"
#define SVR_PORT 6789

int main() {
    int listener = socket(AF_INET, SOCK_STREAM, 0);
    if (listener == -1) { perror("socket"); exit(1); }

    struct sockaddr_in srv_addr;
    srv_addr.sin_family = AF_INET;
    inet_pton(AF_INET, SVR_IP, &srv_addr.sin_addr);
    srv_addr.sin_port = htons(SVR_PORT);

    if (bind(listener, (struct sockaddr*)&srv_addr, sizeof(srv_addr)) == -1) { perror("bind"); exit(1); }
    if (listen(listener, 8) == -1) { perror("listen"); exit(1); }

    fd_set active_fds, read_fds;
    FD_ZERO(&active_fds);
    FD_SET(listener, &active_fds);
    int max_fd = listener;

    while (1) {
        read_fds = active_fds;
        int ready = select(max_fd + 1, &read_fds, NULL, NULL, NULL);
        
        if (ready == -1) { perror("select"); break; }
        
        if (FD_ISSET(listener, &read_fds)) {
            struct sockaddr_in cli_addr;
            socklen_t len = sizeof(cli_addr);
            int client_fd = accept(listener, (struct sockaddr*)&cli_addr, &len);
            if (client_fd != -1) {
                char ip[16];
                inet_ntop(AF_INET, &cli_addr.sin_addr, ip, sizeof(ip));
                printf("New client: %s:%d\n", ip, ntohs(cli_addr.sin_port));
                FD_SET(client_fd, &active_fds);
                if (client_fd > max_fd) max_fd = client_fd;
            }
        }

        for (int i = listener + 1; i <= max_fd; i++) {
            if (FD_ISSET(i, &read_fds)) {
                char buffer[1024] = {0};
                int n = read(i, buffer, sizeof(buffer));
                if (n <= 0) {
                    if (n == 0) printf("Client disconnected.\n");
                    close(i);
                    FD_CLR(i, &active_fds);
                } else {
                    printf("Echo: %s\n", buffer);
                    write(i, buffer, strlen(buffer));
                }
            }
        }
    }
    close(listener);
    return 0;
}

Client Example

#include <stdio.h>
#include <arpa/inet.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>

#define SVR_IP "127.0.0.1"
#define SVR_PORT 6789

int main() {
    int sock = socket(AF_INET, SOCK_STREAM, 0);
    struct sockaddr_in srv_addr;
    srv_addr.sin_family = AF_INET;
    inet_pton(AF_INET, SVR_IP, &srv_addr.sin_addr);
    srv_addr.sin_port = htons(SVR_PORT);

    if (connect(sock, (struct sockaddr*)&srv_addr, sizeof(srv_addr)) == -1) { perror("connect"); exit(1); }

    while (1) {
        char *msg = "ping from client";
        write(sock, msg, strlen(msg));
        char buf[1024] = {0};
        int r = read(sock, buf, sizeof(buf));
        if (r > 0) printf("Server reply: %s\n", buf);
        sleep(1);
    }
    close(sock);
    return 0;
}

Limitations and Optimization

• Performence: select copies the set between user and kernel space on every call and performs linear scans.
• Limit: Default limit of 1024 file descriptors (FD_SETSIZE).
• State Reset: The descriptor sets are modified in place, requiring re-initialization before each call.
• Optimization: Maintain an array of client sockets to avoid scanning invalid descriptors after disconnects.