Processes and isolation

Separate interpreters

Each process has its own interpreter and memory. Processes can run independent CPU work across cores without sharing one interpreter's GIL. Startup and data transfer can outweigh small calculations, so measure the complete workload.

Importable workers

Process pools serialize function references, arguments, and results. With spawn, a fresh interpreter imports the worker module. Define workers at module scope, use serializable values, and guard pool creation with if __name__ == "__main__"::

from concurrent.futures import ProcessPoolExecutor
import multiprocessing

def cube(value):
    return value ** 3

if __name__ == "__main__":
    context = multiprocessing.get_context("spawn")
    with ProcessPoolExecutor(max_workers=2, mp_context=context) as executor:
        print(list(executor.map(cube, [2, 3])))  # [8, 27]

map preserves input order. An explicit local context selects the start method without changing the process-wide default. Run this as a saved .py file; notebook-local or interactive definitions are not reliably importable spawn targets.

Data ownership

Ordinary lists and dictionaries are transferred values, not shared Python containers:

from concurrent.futures import ProcessPoolExecutor
from multiprocessing import get_context

def append_marker(values):
    values.append("child")
    return values

if __name__ == "__main__":
    parent_values = ["parent"]
    with ProcessPoolExecutor(max_workers=1, mp_context=get_context("spawn")) as pool:
        child_values = pool.submit(append_marker, parent_values).result()
    print(parent_values)  # ['parent']
    print(child_values)  # ['parent', 'child']

Child-local mutations do not update the parent. Finish constructing submitted data before handoff: serialization may happen after submit() returns, so concurrent input mutation is not a reliable snapshot policy.

Open external resources in their owning process, using an initializer when appropriate. Connections, locks, descriptors, and library state are not portable just because one start method can inherit them.

Gotchas

• Lambdas and nested functions generally cannot serve as portable pool targets. A main guard does not make them importable.
• Start-method defaults vary by platform and Python version. Select a context when the method affects the design.
• Callable failures surface during result retrieval; an abrupt worker crash may break the pool. Context-manager shutdown waits for workers.
• Measure serialization, transfer, peak memory, and startup as well as computation. Batching can amortize overhead; shared memory reduces some copies but adds lifetime and synchronization duties.

Return structured results rather than relying on interleaved worker output. This lesson requires native processes and cannot run in the browser runtime.