LIGHTWEIGHT GRID MANAGER TUTORIAL

Introduction

The Wolfram Lightweight Grid is a system for launching, managing, and using Mathematica kernels across a network. It is particularly useful for creating an ad hoc grid built out of a collection of different types of computers. An important application is to enable parallel computing in Mathematica to run over a collection of computers; each copy of Mathematica comes with a Lightweight Grid Client, which has tools to find and then connect to the Lightweight Grid.

Each node runs a copy of the Lightweight Grid Manager, which is a server that broadcasts itself and waits for a request to launch Mathematica kernels for parallel computation. The manager also allows connections from a web browser, and this lets an administrator monitor usage and make configuration changes.

This document describes the operation of the Lightweight Grid Manager, the server component of the Lightweight Grid. It covers issues relating to installation, configuring, and operation, as well as troubleshooting.

gridMathematica Server

Parallel computing as a standard feature was introduced in Mathematica 7. This means that Mathematica running on a multicore computer can run multiple parts of a computation concurrently.

gridMathematica Server lets you go beyond a single multicore machine and add kernels that run on other machines across the network to drive parallel computation. Since it works to extend Mathematica, each unit is priced more competitively than a full version of Mathematica. In addition, when you install gridMathematica Server you have the option of configuring the Wolfram Lightweight Grid Manager. Note that gridMathematica Server only provides parallel or subkernels; it does not contain a controlling or master kernel. This means that gridMathematica Server is an extension to Mathematica.

Alternatives to the Lightweight Grid

The Wolfram Lightweight Grid is a convenient way for Mathematica to use gridMathematica Server for parallel computation. However, there are alternative ways to use gridMathematica Server.

Cluster Integration Package

The Cluster Integration Package allows Mathematica to connect across the network to gridMathematica when you have a standard cluster management system. At present, the following systems are supported:

  • Windows Compute Cluster Server
  • Windows HPC Server 2008
  • Platform LSF®
  • Altair® PBS Professional®
  • Sun Grid Engine

If you want to use one of these, you should install gridMathematica Server, but not activate the Lightweight Grid. When you run the master version of Mathematica, you can then use the built-in tools to connect to the cluster.

More information is available in the documentation about the Cluster Integration Package.

Remote Kernels

The Remote Kernels connection method provides another way for Mathematica to connect across the network to gridMathematica. The Remote Kernels method requires that you configure your own remote shell invocation tools, perhaps using ssh, and is more involved to set up.

You would install gridMathematica Server, but not activate the Lightweight Grid, and then configure your network to allow remote shell invocation launching of Mathematica. When you run the master version of Mathematica, you can then use the built-in tools to connect to the cluster.

More information is available in the "Remote Kernels" section of the Parallel Tools tutorial "Connection Methods".

Lightweight Grid Technology

The Lightweight Grid consists of a network of servers, each of which knows how to launch Mathematica kernels for parallel computation. The server is a version of Apache Tomcat, with suitable extensions. When the server runs it broadcasts itself on the network with a multicast DNS service discovery technology.

The server also accepts HTTP connections. You can connect to it with a web interface, which is useful for various management purposes. The server also accepts HTTP requests from a Mathematica kernel that wishes to obtain parallel kernels for computation. When it gets a request, the server launches a kernel (this will be a parallel kernel) and responds with the name of a link that can be used to connect to this parallel kernel. The master Mathematica kernel can use this link to set up parallel computation. When the link is finished the parallel kernel will exit.

Each Lightweight Grid server needs an account to run. This account (defined by a username and password) is known by the computer on which the server runs. The account is also used to run parallel kernels launched by the server. In addition, to carry out administration of the server using the web interface, a password is required. This web interface administrator password is managed by the server and is used only in the web interface.

It is good to understand that the password for the user account that runs the server is distinct from the web interface administrator password, which is the password that allows configuration changes to the server. When you install the Lightweight Grid you can choose to let the installer activate the server. In this case the account to run the server is set up and the administrator password is entered.

The multicast-based service discovery that the server provides allows other computers to find it on the local network. This is built into Mathematica, which will automatically find any servers in the Lightweight Grid on the local subnet, when you enable the Lightweight Grid tab in the parallel preferences.

If Mathematica is not running on the same network as computers running the Lightweight Grid, you will have to enter the name of one computer that runs the Lightweight Grid. When you do this, all the other computers that this one computer knows about will be made available to Mathematica. This works because each Lightweight Grid server keeps track of any other servers that it finds with the service discovery technology.

New to Mathematica? Find your learning path »
Have a question? Ask support »