Process-Level Data Isolation in Python
When working with Python's multiprocessing module, each process operates with its own independent copy of global variables, regardless of synchronization mechanisms like locks. This behavior demonstrates the fundamental isolation between processes.
Demonstration Code
The following example illustrates how two separate processes maintain independent copies of a shared list:
import time
from multiprocessing import Process
shared_data = [10, 20, 30, 40]
class ReaderProcess(Process):
def run(self):
while True:
print(f"Reader sees: {shared_data}")
time.sleep(1.5)
class WriterProcess(Process):
def run(self):
while True:
print(f"Writer sees: {shared_data}")
shared_data.append("new_item")
time.sleep(1.5)
if __name__ == '__main__':
ReaderProcess().start()
WriterProcess().start()
Observations
Running this code reveals that:
- The reader process consistently displays the original list [10, 20, 30, 40]
- The writer process shows its own modified version with appended items
- Changes made by one process are not visible to the other
Enhanced Test with Mutual Modification
Modifying both processes to alter the data confirms the isolation:
class ReaderProcess(Process):
def run(self):
while True:
print(f"Reader sees: {shared_data}")
shared_data.append("reader_item")
time.sleep(1.5)
This results in each process maintaining completely separate lists with their own appended values.
Synchronization Limitations
Even with synchronziation primitives, the isolation persists because each process starts with a copy of the parent process's memory space:
from multiprocessing import Process, Lock
data_lock = Lock()
def modify_data(value):
with data_lock:
shared_data.append(value)
The lock only provides synchronization within each process's context, not across process boundaries.
Technical Explanation
This behavior occurs because Python multiprocessing uses separate memory spaces for each process. When a new process is spawned, it receives an initial copy of the parent's data, but subsequent modifications remain isolated within each process's address space.
For actual inter-process communication, Python provides mechanisms like:
- multiprocessing.Queue
- multiprocessing.Pipe
- multiprocessing.Manager
- Shared memory objects