ParallelMap

ParallelMap[f,expr]

applies f in parallel to each element on the first level in expr.

ParallelMap[f,expr,levelspec]

applies f in parallel to parts of expr specified by levelspec.

Details and Options

  • ParallelMap is a parallel version of Map, which automatically distributes different applications of f among different kernels and processors.
  • ParallelMap will give the same results as Map, except for side effects during the computation.
  • ParallelMap uses the same level specifications as Map. Not all level specifications can be parallelized.
  • Parallelize[Map[f,expr]] is equivalent to ParallelMap[f,expr].
  • If an instance of ParallelMap cannot be parallelized, it is evaluated using Map.
  • The following options can be given:
  • Method Automaticgranularity of parallelization
    DistributedContexts $DistributedContextscontexts used to distribute symbols to parallel computations
    ProgressReporting $ProgressReportingwhether to report the progress of the computation
  • The Method option specifies the parallelization method to use. Possible settings include:
  • "CoarsestGrained"break the computation into as many pieces as there are available kernels
    "FinestGrained"break the computation into the smallest possible subunits
    "EvaluationsPerKernel"->ebreak the computation into at most e pieces per kernel
    "ItemsPerEvaluation"->mbreak the computation into evaluations of at most m subunits each
    Automaticcompromise between overhead and load balancing
  • Method->"CoarsestGrained" is suitable for computations involving many subunits, all of which take the same amount of time. It minimizes overhead, but does not provide any load balancing.
  • Method->"FinestGrained" is suitable for computations involving few subunits whose evaluations take different amounts of time. It leads to higher overhead, but maximizes load balancing.
  • The DistributedContexts option specifies which symbols appearing in expr have their definitions automatically distributed to all available kernels before the computation.
  • The default value is DistributedContexts:>$DistributedContexts with $DistributedContexts:=$Context, which distributes definitions of all symbols in the current context, but does not distribute definitions of symbols from packages.
  • The ProgressReporting option specifies whether to report the progress of the parallel computation.
  • The default value is ProgressReporting:>$ProgressReporting.

Examples

open allclose all

Basic Examples  (4)

ParallelMap works like Map, but in parallel:

ParallelMap can work with any function:

Use explicit pure functions:

Functions defined interactively can immediately be used in parallel:

Longer computations display information about their progress and estimated time to completion:

Scope  (6)

Map at level 1 (default):

Map down to level 2:

Map at level 2:

Map down to level 3:

Map on all levels, starting at level 1:

Negative levels:

Positive and negative levels can be mixed:

Different heads at each level:

Not all levels can be parallelized:

Generalizations & Extensions  (2)

ParallelMap can be used on expressions with any head:

Functions with attribute Listable are mapped automatically:

Options  (13)

DistributedContexts  (5)

By default, definitions in the current context are distributed automatically:

Do not distribute any definitions of functions:

Distribute definitions for all symbols in all contexts appearing in a parallel computation:

Distribute only definitions in the given contexts:

Restore the value of the DistributedContexts option to its default:

Method  (6)

Break the computation into the smallest possible subunits:

Break the computation into as many pieces as there are available kernels:

Break the computation into at most 2 evaluations per kernel for the entire job:

Break the computation into evaluations of at most 5 elements each:

The default option setting balances evaluation size and number of evaluations:

Calculations with vastly differing runtimes should be parallelized as finely as possible:

A large number of simple calculations should be distributed into as few batches as possible:

ProgressReporting  (2)

Do not show a temporary progress report:

Use Method"FinestGrained" for the most accurate progress report:

Applications  (2)

Watch the results appear as they are found:

Solve the arithmetic puzzle:

Write it in symbolic form:

Assign a single-digit number to each letter:

And interpret each word as a number in base 10:

Automate the checking for a particular digit assignment:

To systematically solve this assignment, first get the list of letters:

The puzzle can be solved by considering all permutations of all subsets of eight digits:

Parallelize it:

Only the solutions with are usually considered:

The search can be stopped as soon as a nontrivial solution is found using ParallelTry:

Properties & Relations  (10)

The parallelization happens at the outermost level used:

Map can be parallelized automatically, effectively using ParallelMap:

Show the effect of load balancing with tasks of known size:

Define a number of tasks with known runtimes:

Measure the time for parallel execution:

The speedup obtained (more is better):

Finest-grained scheduling gives better load balancing and higher speedup:

Scheduling large tasks first gives even better results:

A function of several arguments can be mapped with MapThread:

Get a parallel version by using Parallelize:

MapIndexed passes the index of an element to the mapped function:

Get a parallel version by using Parallelize:

Scan does the same as Map, but without returning a result:

Functions defined interactively are automatically distributed to all kernels when needed:

Distribute definitions manually and disable automatic distribution:

For functions from a package, use ParallelNeeds rather than DistributeDefinitions:

Get the size of all Wolfram Language files in an installation of Mathematica in parallel:

Possible Issues  (7)

If a level specification prevents parallelization, ParallelMap evaluates like Map:

Side effects cannot be used in the function mapped in parallel:

Use a shared variable to support side effects:

A function used that is not known on the parallel kernels may lead to sequential evaluation:

Define the function on all parallel kernels:

The function is now evaluated on the parallel kernels:

Definitions of functions in the current context are distributed automatically:

Definitions from contexts other than the default context are not distributed automatically:

Use DistributeDefinitions to distribute such definitions:

Alternatively, set the DistributedContexts option to include all contexts:

Trivial operations may take longer when parallelized:

Measure the minimum parallel communication overhead:

Compare with a simple calculation done on the master kernel itself:

Functions may simplify short argument lists, but not longer ones:

Such simplification of partial expressions may make parallel mapping impossible:

Prevent simplification of partial expressions and apply the desired function only at the end:

If the larger computations are near the end of the input list, timing estimates will be inaccurate:

Wolfram Research (2008), ParallelMap, Wolfram Language function, https://reference.wolfram.com/language/ref/ParallelMap.html (updated 2010).

Text

Wolfram Research (2008), ParallelMap, Wolfram Language function, https://reference.wolfram.com/language/ref/ParallelMap.html (updated 2010).

CMS

Wolfram Language. 2008. "ParallelMap." Wolfram Language & System Documentation Center. Wolfram Research. Last Modified 2010. https://reference.wolfram.com/language/ref/ParallelMap.html.

APA

Wolfram Language. (2008). ParallelMap. Wolfram Language & System Documentation Center. Retrieved from https://reference.wolfram.com/language/ref/ParallelMap.html

BibTeX

@misc{reference.wolfram_2023_parallelmap, author="Wolfram Research", title="{ParallelMap}", year="2010", howpublished="\url{https://reference.wolfram.com/language/ref/ParallelMap.html}", note=[Accessed: 18-March-2024 ]}

BibLaTeX

@online{reference.wolfram_2023_parallelmap, organization={Wolfram Research}, title={ParallelMap}, year={2010}, url={https://reference.wolfram.com/language/ref/ParallelMap.html}, note=[Accessed: 18-March-2024 ]}