Alle boeken

Professioneel

Apps Over Coach Inloggen Begin met lezen

Quantitative Finance · Begrippenlijst

Wat is Process-based parallelism, serialisation cost?

Ook bekend als: process-based parallelism · serialisation cost

Definition 8.8 Research, Data and Risk Platforms · Hoofdstuk 8 — Python at Scale

Process-based parallelism runs work in several operating-system processes, each with its own interpreter and lock, so that Python code runs on several cores at once. Its serialisation cost is the time spent converting data to bytes and back to move them between processes (pickling, copying through a pipe), which shared memory or files mapped by both processes avoid.

Moving a 64 MB array to a second process (pickled through a pipe, or placed in shared memory), and running a pure-Python loop and a numpy sort on one thread and on two (each thread doing the whole task). Measured on a laptop (Intel Core Ultra 7 155H) under WSL2, machine otherwise idle. Data: bench_pyscale.py.
Figure 8.4. Moving a 64 MB array to a second process (pickled through a pipe, or placed in shared memory), and running a pure-Python loop and a numpy sort on one thread and on two (each thread doing the whole task). Measured on a laptop (Intel Core Ultra 7 155H) under WSL2, machine otherwise idle. Data: bench_pyscale.py.
Lees in het hoofdstuk →