ParallelMap
ParallelMap[f,expr]
applies f in parallel to each element on the first level in expr.
ParallelMap[f,expr,levelspec]
applies f in parallel to parts of expr specified by levelspec.
Details and Options
 ParallelMap is a parallel version of Map, which automatically distributes different applications of f among different kernels and processors.
 ParallelMap will give the same results as Map, except for side effects during the computation.
 ParallelMap uses the same level specifications as Map. Not all level specifications can be parallelized.
 Parallelize[Map[f,expr]] is equivalent to ParallelMap[f,expr].
 If an instance of ParallelMap cannot be parallelized, it is evaluated using Map.
 The following options can be given:

Method Automatic granularity of parallelization DistributedContexts $DistributedContexts contexts used to distribute symbols to parallel computations ProgressReporting $ProgressReporting whether to report the progress of the computation  The Method option specifies the parallelization method to use. Possible settings include:

"CoarsestGrained" break the computation into as many pieces as there are available kernels "FinestGrained" break the computation into the smallest possible subunits "EvaluationsPerKernel">e break the computation into at most e pieces per kernel "ItemsPerEvaluation">m break the computation into evaluations of at most m subunits each Automatic compromise between overhead and load balancing  Method>"CoarsestGrained" is suitable for computations involving many subunits, all of which take the same amount of time. It minimizes overhead, but does not provide any load balancing.
 Method>"FinestGrained" is suitable for computations involving few subunits whose evaluations take different amounts of time. It leads to higher overhead, but maximizes load balancing.
 The DistributedContexts option specifies which symbols appearing in expr have their definitions automatically distributed to all available kernels before the computation.
 The default value is DistributedContexts:>$DistributedContexts with $DistributedContexts:=$Context, which distributes definitions of all symbols in the current context, but does not distribute definitions of symbols from packages.
 The ProgressReporting option specifies whether to report the progress of the parallel computation.
 The default value is ProgressReporting:>$ProgressReporting.
Examples
open allclose allBasic Examples (4)
ParallelMap works like Map, but in parallel:
ParallelMap can work with any function:
Functions defined interactively can immediately be used in parallel:
Longer computations display information about their progress and estimated time to completion:
Scope (6)
Generalizations & Extensions (2)
ParallelMap can be used on expressions with any head:
Functions with attribute Listable are mapped automatically:
Options (13)
DistributedContexts (5)
By default, definitions in the current context are distributed automatically:
Do not distribute any definitions of functions:
Distribute definitions for all symbols in all contexts appearing in a parallel computation:
Distribute only definitions in the given contexts:
Restore the value of the DistributedContexts option to its default:
Method (6)
Break the computation into the smallest possible subunits:
Break the computation into as many pieces as there are available kernels:
Break the computation into at most 2 evaluations per kernel for the entire job:
Break the computation into evaluations of at most 5 elements each:
The default option setting balances evaluation size and number of evaluations:
Calculations with vastly differing runtimes should be parallelized as finely as possible:
A large number of simple calculations should be distributed into as few batches as possible:
ProgressReporting (2)
Do not show a temporary progress report:
Use Method"FinestGrained" for the most accurate progress report:
Applications (2)
Watch the results appear as they are found:
Assign a singledigit number to each letter:
And interpret each word as a number in base 10:
Automate the checking for a particular digit assignment:
To systematically solve this assignment, first get the list of letters:
The puzzle can be solved by considering all permutations of all subsets of eight digits:
Only the solutions with are usually considered:
The search can be stopped as soon as a nontrivial solution is found using ParallelTry:
Properties & Relations (10)
The parallelization happens at the outermost level used:
Map can be parallelized automatically, effectively using ParallelMap:
Show the effect of load balancing with tasks of known size:
Define a number of tasks with known runtimes:
Measure the time for parallel execution:
The speedup obtained (more is better):
Finestgrained scheduling gives better load balancing and higher speedup:
Scheduling large tasks first gives even better results:
A function of several arguments can be mapped with MapThread:
Get a parallel version by using Parallelize:
MapIndexed passes the index of an element to the mapped function:
Get a parallel version by using Parallelize:
Scan does the same as Map, but without returning a result:
Functions defined interactively are automatically distributed to all kernels when needed:
Distribute definitions manually and disable automatic distribution:
For functions from a package, use ParallelNeeds rather than DistributeDefinitions:
Get the size of all Wolfram Language files in an installation of Mathematica in parallel:
Possible Issues (7)
If a level specification prevents parallelization, ParallelMap evaluates like Map:
Side effects cannot be used in the function mapped in parallel:
Use a shared variable to support side effects:
A function used that is not known on the parallel kernels may lead to sequential evaluation:
Define the function on all parallel kernels:
The function is now evaluated on the parallel kernels:
Definitions of functions in the current context are distributed automatically:
Definitions from contexts other than the default context are not distributed automatically:
Use DistributeDefinitions to distribute such definitions:
Alternatively, set the DistributedContexts option to include all contexts:
Trivial operations may take longer when parallelized:
Measure the minimum parallel communication overhead:
Compare with a simple calculation done on the master kernel itself:
Functions may simplify short argument lists, but not longer ones:
Such simplification of partial expressions may make parallel mapping impossible:
Prevent simplification of partial expressions and apply the desired function only at the end:
If the larger computations are near the end of the input list, timing estimates will be inaccurate:
Text
Wolfram Research (2008), ParallelMap, Wolfram Language function, https://reference.wolfram.com/language/ref/ParallelMap.html (updated 2010).
CMS
Wolfram Language. 2008. "ParallelMap." Wolfram Language & System Documentation Center. Wolfram Research. Last Modified 2010. https://reference.wolfram.com/language/ref/ParallelMap.html.
APA
Wolfram Language. (2008). ParallelMap. Wolfram Language & System Documentation Center. Retrieved from https://reference.wolfram.com/language/ref/ParallelMap.html