Introduction to the Wolfram Lightweight Grid System

The Wolfram Lightweight Grid System provides functions for launching and managing remote kernels, one part of a complete Mathematica® parallel computing environment. With the Lightweight Grid, you can:

Quick Start

This section describes how to start using the Lightweight Grid with the Parallel Tools.

First, load the Lightweight Grid package:

Then find out what machines are on the network. If you already know which machine or machines you want to use, you can skip this step:

You can open a kernel on each of these machines:

LaunchKernels will use the Lightweight Grid connection method when given URL strings.

Now perform a parallel calculation, such as factoring integers of the form 111...1:

Continue with your parallel computing. When you want to shut down the kernels, call CloseKernels:

Hostnames, URLs, and Short Forms

There are many places in the Lightweight Grid system where you need to identify a the Lightweight Grid manager running on a particular computer. This section describes the different ways to identify an agent.

The canonical way to identify a the Lightweight Grid manager, which is a web application running within a web server, is by using its published contact URL. Each of the shorter specifications takes advantage of the DNS Service Discovery (DNS-SD) directory to narrow down which agent is intended, or is used to build a URL assuming the agent uses defaults for the URL components not specified. An agent that does not use default values and is not known to DNS-SD cannot be referred to by any short form.

Ways to Identify an Agent

The four accepted agent specification forms are:

1.  Simple hostname

Example: galaxy

The simple hostname is the first label of a fully qualified domain name, before the first dot. For example, galaxy is the hostname obtained from the fully qualified domain name galaxy.example.com. This form can be used if either of the following is true: (1) the agent has been found by DNS-SD, the hostname is unique, and there is only one agent running on that host, or (2) the agent is contactable using the default values for the components of its URL (port number, context path, and servlet name).

2.  Fully qualified domain name or IP address

Example: galaxy.example.com

Example: 10.202.100.37

This form can be used if either of the following is true: (1) the agent has been found by DNS-SD and there is only one agent running on that host, or (2) the agent is contactable using the default values for the components of its URL (port number, context path, and servlet name).

3.  Host:port

Example: galaxy:8080

Example: galaxy.example.com:8080

Example: 192.168.1.16:8080

This form can be used if either of the following is true: (1) the agent has been found by DNS-SD and there is an agent matching the given host and port, or (2) the agent uses default values for the context path and servlet name.

4.  URL

Example: http://galaxy.example.com:3737/WolframLightweightGrid/Manager

This does not have to be the same as the canonical URL, the one published by the agent through DNS-SD as the URL to use in contacting it, as long as the agent is reachable by ordinary network connectivity mechanisms.

For agent specifications 1 through 3, the Lightweight Grid will use the given hostname and optional port number to index its list of known agents. If a unique match is found, the published contact URL for the matching agent will be used to contact it. If no match is found, a contact URL will be constructed using default values. It is an error if multiple agents match the given specification. This could happen, for example, when a hostname such as galaxy is used when the same hostname exists in more than one domain, such as galaxy.example.com and galaxy.example.edu.

Services

The Lightweight Grid manager has one or more services. A service is a set of configuration properties, such as the command for launching the kernel. The agent always has one service designated as the default service, initially named General. Functions that can take a service as an argument, such as RemoteKernelOpen, will use the default service if no service is specified.

One significant use of the service feature is to provide support for more than one version of Mathematica. For example, if you have installed Mathematica 12.0 and 12.1, Lightweight Grid can launch either of these versions by associating a different service with each one.

In general, if you find that you have different service configuration needs, you should consider creating new services to match your different configuration needs rather than repeatedly reconfiguring the default service.