Parallelize
Parallelize[expr]
evaluates expr using automatic parallelization.
Details and Options
 Parallelize[expr] automatically distributes different parts of the evaluation of expr among different available kernels and processors.
 Parallelize[expr] normally gives the same result as evaluating expr, except for side effects during the computation.
 Parallelize has attribute HoldFirst, so that expressions are not evaluated before parallelization.
 The Method option for Parallelize specifies the parallelization method to use. Possible settings include:

"CoarsestGrained" break the computation into as many pieces as there are available kernels "FinestGrained" break the computation into the smallest possible subunits "EvaluationsPerKernel">e break the computation into at most e pieces per kernel "ItemsPerEvaluation">m break the computation into evaluations of at most m subunits each Automatic compromise between overhead and loadbalancing  Method>"CoarsestGrained" is suitable for computations involving many subunits, all of which take the same amount of time. It minimizes overhead, but does not provide any load balancing.
 Method>"FinestGrained" is suitable for computations involving few subunits whose evaluations take different amounts of time. It leads to higher overhead, but maximizes load balancing.
 The DistributedContexts option for Parallelize specifies which symbols appearing in expr have their definitions automatically distributed to all available kernels before the computation.
 The default value is DistributedContexts:>$Context, which distributes definitions of all symbols in the current context, but does not distribute definitions of symbols from packages.
Examples
open allclose allBasic Examples (3)
Scope (13)
Listable Functions (1)
StructurePreserving Functions (3)
Many functional programming constructs that preserve list structure parallelize:
The result need not have the same length as the input:
Without a function, Parallelize simply evaluates the elements in parallel:
Reductions (3)
Count the number of primes up to one million:
Check whether 93 occurs in a list of the first 100 primes:
Check whether a list is free of 5:
The argument does not have to be an explicit List:
Inner and Outer Products (2)
Iterators (4)
Evaluate a table in parallel, with or without an iterator variable:
Generate an array in parallel:
Evaluate sums and products in parallel:
The evaluation of the function happens in parallel:
The list of file names is expanded locally on the subkernels:
Associative Functions (1)
Functions with the attribute Flat automatically parallelize:
Generalizations & Extensions (3)
Options (11)
Method (6)
Break the computation into the smallest possible subunits:
Break the computation into as many pieces as there are available kernels:
Break the computation into at most 2 evaluations per kernel for the entire job:
Break the computation into evaluations of at most 5 elements each:
The default option setting balances evaluation size and number of evaluations:
Calculations with vastly differing runtimes should be parallelized as finely as possible:
A large number of simple calculations should be distributed into as few batches as possible:
DistributedContexts (5)
By default, definitions in the current context are distributed automatically:
Do not distribute any definitions of functions:
Distribute definitions for all symbols in all contexts appearing in a parallel computation:
Distribute only definitions in the given contexts:
Restore the value of the DistributedContexts option to its default:
Applications (4)
Properties & Relations (7)
For data parallel functions, Parallelize is implemented in terms of ParallelCombine:
Parallel speedup can be measured with a calculation that takes a known amount of time:
Define a number of tasks with known runtimes:
The time for a sequential execution is the sum of the individual times:
Measure the speedup for parallel execution:
Finestgrained scheduling gives better load balancing and higher speedup:
Scheduling large tasks first gives even better results:
Form the arithmetic expression 1⊗2⊗3⊗4⊗5⊗6⊗7⊗8⊗9 for ⊗ chosen from +, –, *, /:
Each list of arithmetic operations gives a simple calculation:
Find all sequences of arithmetic operations that give 0:
Display the corresponding expressions:
Functions defined interactively are automatically distributed to all kernels when needed:
Distribute definitions manually and disable automatic distribution:
For functions from a package, use ParallelNeeds rather than DistributeDefinitions:
Set up a random number generator that is suitable for parallel use and initialize each kernel:
Possible Issues (8)
Expressions that cannot be parallelized are evaluated normally:
Side effects cannot be used in the function mapped in parallel:
Use a shared variable to support side effects:
If no subkernels are available, the result is computed on the master kernel:
If a function used is not distributed first, the result may still appear to be correct:
Only if the function is distributed is the result actually calculated on the available kernels:
Definitions of functions in the current context are distributed automatically:
Definitions from contexts other than the default context are not distributed automatically:
Use DistributeDefinitions to distribute such definitions:
Alternatively, set the DistributedContexts option to include all contexts:
Explicitly distribute the definition of a function:
The modified definition is automatically distributed:
Suppress the automatic distribution of definitions:
Symbols defined only on the subkernels are not distributed automatically:
The value of $DistributedContexts is not used in Parallelize:
Set the value of the DistributedContexts option of Parallelize: