Parallel computing in Mathematica is based on launching and controlling multiple Mathematica kernel (worker) processes from within a single master Mathematica, providing a distributed-memory environment for parallel programming. Every copy of Mathematica comes with all the components and tools to run and create parallel applications.
The parallel computing features are written almost entirely in the Mathematica language and are therefore machine independent. They have been tested on Unix, Linux, Windows, and Macintosh platforms and are well suited to working locally on a multicore machine, in a grid of machines, or in a heterogeneous network. All client and application code is distributed, so no common file system is necessary.
Parallel computing is now provided as a standard part of Mathematica, thereby facilitating the use of parallel programming techniques. These can benefit from many of the advances in technology, for example, multicore machines, usage of specialized hardware for parallel computation, and fast network technology that helps with running a grid of connected computers.
Underlying parallel computation in Mathematica is MathLink. This is a general and flexible interface for programs to communicate with Mathematica. It is a fundamental component of Mathematica and is used extensively in Mathematica itself. One common mode of operation is for different components of Mathematica to communicate using MathLink, which is exactly how parallel computation uses MathLink.
MathLink provides some core features, it is platform and architecture independent, it works both locally and across the network, it can transmit anything that Mathematica can represent, and it provides an interface for controlling Mathematica. These are all features that are key to parallel computation, allowing local and network communication between both homogeneous and heterogeneous networks, and allowing transmission of both data and programs.
Mathematica can run parallel workers in a number of different ways, locally on the same machine or remotely on a network. In addition, the network might be a homogeneous grid controlled by some dedicated management application, or it might be a heterogeneous grid. To run in these situations, Mathematica needs to launch parallel workers and then communicate with them.
To carry out parallel computation in these different situations, Mathematica provides a number of different connection methods, each of which is specialized to a particular style of usage. Some of the main connection methods are described here.
The local kernels connection method is used to run parallel workers on the same computer as the master Mathematica. It is suitable for a multicore environment and is the easiest way to get up and running with parallel computation.
The Lightweight Grid connection method is used to run parallel workers on different computers from the master Mathematica. It uses Wolfram Lightweight Grid technology to launch Mathematica on the remote machines. It is suitable for a heterogeneous network and where there is no management technology.
The Cluster Integration connection method is used to run parallel workers on different computers from the master Mathematica. It integrates with a large number of third-party cluster management technologies.
The remote kernels connection method is used to run parallel workers on different computers from the master Mathematica. It relies on using a remote shell invocation technology for launching and is typically harder to configure and maintain.
Additional parallel programming features are available in the Parallel`Developer` context. They allow you to get and set properties of parallel kernels and implement your own scheduler for concurrency.