Process Fundamentals in Operating Systems

Processes are among the most crucial concepts in operating systems, representing the fundamental unit of execution. Threads are also important components within this context.

Both processes and threads are managed by the operating system through scheduling algorithms, which programmers cannot directly control.

What are processes and threads? What is a program?
# Processes and programs are distinct concepts
Program: Essentially a static entity - a collection of code without a lifecycle
Process: Has a lifecycle, existing only while the task is being executed
	"""
	Cooking analogy: A recipe represents the program, the cooking process is the execution, and the chef is the thread. When cooking completes, the process terminates. Threads perform the actual work within a process.
	"""
    
# A process can contain multiple threads, though it may have only one
# Every process must have at least one thread
Both processes and threads are scheduled by the operating system. Programmers cannot control this scheduling directly. Coroutines, however, are programmer-controlled and don't exist as a concept in the operating system - they're an abstraction created by developers.

Process >>> Thread >>> Coroutine
Resource consumption comparison: Process >>> Thread >>> Coroutine
# A single CPU can only run one task at a time.
http://www.ruanyifeng.com/blog/2013/04/processes_and_threads.html

Process Scheduling Algorithms (Four Types)

CPU operation mechanism:
	1. When encountering I/O operations, the CPU yields execution control
    2. When encountering time-consuming operations, the CPU automatically yields control to switch to other tasks

I/O-bound tasks
	"""These are blocked by time and don't consume significant CPU resources, like sleep(3)"""
CPU-bound tasks
	"""Don't consume much time but use substantial CPU resources"""
    for i in range(10000000):
        i+=1
        
1. First-Come, First-Served scheduling
2. Shortest Job First scheduling
3. Round Robin scheduling
4. Multilevel Feedback Queue

Parallelism vs Concurrency

Parallelism: Simultaneous execution at the same moment
	# With a single-core CPU, true simultaneous execution of multiple tasks is impossible
    # With multi-core CPUs, simultaneous execution becomes possible
    # With 2 cores, maximum 2 tasks can run simultaneously
    # With 4 cores, maximum 4 tasks can run simultaneously
    
Concurrency: Appears to run simultaneously over a period
	# Even with a single-core CPU, you can perform multiple operations like listening to music, watching videos, and browsing the web simultaneously through CPU context switching
    
Real-world concurrency examples:
	Alibaba's Double 11 shopping festival

"""Interview question: What are your insights on high concurrency? How would you address it?"""

Synchronous/Asynchronous and Blocking/Non-blocking Concepts

Synchronous: Each operation depends on the result of the previous one
Asynchronous: Operations don't depend on previous results
"""Asynchronous operations are more efficient than synchronous ones"""

Blocking
Non-blocking

Asynchronous + Non-blocking # Highest efficiency
Synchronous + Blocking   # Lowest efficiency

Creating Processes in Python

from multiprocessing import Process


def worker_task():
    with open('output.txt', 'w', encoding='utf-8') as file_handle:
        file_handle.write('hello world')


"""This implementation doesn't actually start a new process yet"""
"""This conditional is mandatory on Windows platforms"""
if __name__ == '__main__':
    process_instance = Process(target=worker_task)  # Create a process object to execute worker_task
    process_instance.start()  # Actually start the process

Process Class Parameters

from multiprocessing import Process


def worker(name, age, gender):
    print(name, age, gender)
    with open('output.txt', 'w', encoding='utf-8') as file_handle:
        file_handle.write('hello world')

"""This implementation doesn't actually start a process yet"""
"""This conditional is mandatory on Windows platforms"""
if __name__ == '__main__':
    """
    group=None, target=None, name=None, args=(), kwargs={},
                 *, daemon=None
    """
    process = Process(target=worker, name='worker-2', args=(), kwargs={'name':'alice', 'age':25, 'gender':'female'})
    process.start()
    # The operating system is responsible for launching this process
    # Launching a process to execute worker_task - the actual work is done by threads within the process
    """Process attributes: 1. Process name 2. Process ID (pid)"""
    # How to check process name
    print(process.name) # Process-1
    # How to change process name
    # process.name = 'new_process_name'
    # print(process.name)  # new_process_name

Process Class Methods

from multiprocessing import Process

import time
def worker(name, age, gender):
    print(name, age, gender)
    time.sleep(3)
    print("Child process execution complete")

"""This implementation doesn't actually start a process yet"""
"""This conditional is mandatory on Windows platforms"""
if __name__ == '__main__':
    """
    group=None, target=None, name=None, args=(), kwargs={},
                 *, daemon=None
    """
    # Child process vs main process
    """The start() method only notifies the OS to create a process, not immediately launches it - this reflects the significant overhead of process creation"""
    process = Process(target=worker, name='worker-2', args=(), kwargs={'name':'alice', 'age':25, 'gender':'female'})
    process.start()
    # The operating system is responsible for launching this process
    # Launching a process to execute worker_task - the actual work is done by threads within the process
    """Process attributes: 1. Process name 2. Process ID (pid)"""
    # How to check process name
    # print(process.name) # Process-1
    # How to change process name
    # process.name = 'new_process_name'
    # print(process.name)  # new_process_name

    ## How to check process ID
    # print(process.pid) # process id

    # print(process.is_alive()) # True
    # process.terminate() # Kill process
    # import time
    # time.sleep(1)
    # print(process.is_alive())
    process.join() # Wait for child process to complete before continuing with main process
    print("Main process execution complete")

Implementing Multi-process Applications

Multi-process architecture enables simultaneous execution of multiple tasks - each process handles one task, so multiple processes handle multiple tasks
from multiprocessing import Process

import time
def worker_task(identifier):
    print(f"Process {identifier} working")
    time.sleep(1)
if __name__ == '__main__':
    """Theoretically, you can create unlimited processes, but you must consider resource consumption"""
    start_timestamp = time.time()
    process_list = []
    for i in range(10):
        p = Process(target=worker_task, kwargs={'identifier':i})
        p.start()
        process_list.append(p)

    for j in process_list:
        j.join()

    print(f"Main process completed. Total time: {time.time() - start_timestamp}")

TCP-based High-Concurrency Application

A single server cannot simultaneously communicate with multiple clients
import socket  # Python's socket module

def handle_client(connection):
    while True:
        try:
            # Handling the sticky packet issue
            data = connection.recv(1024)  # Maximum 1024 bytes
            if len(data) == 0:
                continue
            print(data)  # Still in bytes format

            # Server sends response back to client
            connection.send(data.upper())
        except Exception as error:
            print(error)
            break

    connection.close()


from multiprocessing import Process

if __name__ == '__main__':
    # 1. Create socket (phone)
    # SOCK_STREAM ====> Represents TCP protocol
    # socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    # socket.socket(socket.AF_INET, socket.SOCK_DGRAM) # UDP protocol
    server_socket = socket.socket()  # Defaults to TCP

    # 2. Bind to address (SIM card)
    # '0.0.0.0'  =====> Allows connections from any IP
    # server_socket.bind(('0.0.0.0', 8000))
    server_socket.bind(('127.0.0.1', 8001))

    # 3. Start listening
    server_socket.listen(5)  # Listen with a backlog of 5
    print('Server ready to accept client connections:')
    while True:
        client_conn, client_address = server_socket.accept()  # Accept connection (blocks until connection arrives)
        p = Process(target=handle_client, args=(client_conn,))
        p.start()