Introduction
Parallel Computing with Mathematica
Parallel computing in
Mathematica is based on launching and controlling
multiple
Mathematica kernel (worker) processes from within a single master
Mathematica, providing a distributed-memory environment for parallel programming. Every copy of
Mathematica comes with all the components and tools to run and create parallel applications.
The parallel computing features are written almost entirely in the
Mathematica language and are therefore machine-independent. They have been tested on Unix, Linux, Windows, and Macintosh platforms and are well suited to working locally on a multi-core machine, in a grid of machines, or in a heterogeneous network. All client and application code is distributed
, so no common file system is necessary.
Parallel computing is now provided as a standard part of
Mathematica, thereby facilitating the use of parallel programming techniques. These can benefit from many of the advances in technology, for example, multi-core machines, usage of specialized hardware for parallel computation and, and fast network technology that helps running a grid of connected computers.
To perform computations in parallel, you need to be able to perform the following tasks:
- start processes and wait for processes to finish
- schedule processes on available processors
MathLink
Underlying parallel computation in
Mathematica is
MathLink. This is a general and flexible interface for programs to communicate with
Mathematica. It is a fundamental component of
Mathematica -- and is used extensively in
Mathematica itself. One common mode operation is for different components of
Mathematica to communicate using
MathLink, exactly how parallel computation uses
MathLink.
MathLink provides some core features, it is platform and architecture independent, it works both locally and across the network, it can transmit anything that
Mathematica can represent, and it provides an interface for controlling
Mathematica. These are all features that are key to parallel computation, allowing local and network communication between both homogeneous and heterogeneous networks, and allowing transmission of both data and programs.
Connection Methods
Mathematica can run parallel workers in a number of different ways, locally on the same machine or remote on a network. In addition, the network might be a homogeneous grid controlled by some dedicated management application, or it might be a heterogenous grid. To run in these situations
Mathematica needs to launch parallel workers and then communicate with them.
To carry out parallel computation in these different situations
Mathematica provides a number of different connection methods, each of which is specialized to a particular style of usage. Some of the main connection methods are described here.
Local Kernels
The Local Kernels connection method is used to run parallel workers on the same computer as the master
Mathematica. It is suitable for a multi-core environment, and is the easiest way to get up and running with parallel computation.
Whenever your processor has more than one core, you can use local kernels for parallel computations.
| Out[25]= |  |
The Lightweight Grid
The Lightweight Grid connection method is used to run parallel workers on different computers from the master
Mathematica. It uses Wolfram Lightweight Grid technology to launch
Mathematica on the remote machines. It is suitable for a heterogeneous network and where there is no management technology.
Cluster Integration
The Cluster Integration connection method is used to run parallel workers on different computers from the master
Mathematica. It integrates with a large number of third party cluster management technologies.
Remote Kernels
The Remote Kernels connection method is used to run parallel workers on different computers from the master
Mathematica. It relies on using a remote shell invocation technology for launching, and is typically harder to configure and maintain.
Features
The main features of parallel computing in
Mathematica are:
- distributed memory, master/slave parallelism
- MathLink communication with remote kernels
- exchange of symbolic expressions and programs with remote kernels, not only numbers and arrays
- heterogeneous network, multiprocessor machines, LAN and WAN
- virtual process scheduling or explicit process distribution to available processors
- virtual shared memory, synchronization, locking
- parallel functional programming and automatic parallelization support
- failure recovery, automatic reassignment of stranded processes on failed remote computers
Lower-Level Features
Additional parallel programming features are available in the
Parallel`Developer` context. They allow you to get and set properties of parallel kernels, and implement your own scheduler for concurrency.
The lower-level features are best made available by adding their context to your
$ContextPath.
For a single use, it is sufficient to refer to developer functions with their full name, such as
Parallel`Developer`ClearKernels[].
