Handling Lists, Arrays, and Other Expressions

The Wolfram Symbolic Transfer Protocol (WSTP) allows you to exchange data of any type with external programs. For more common types of data, you simply need to give appropriate or specifications in your WSTP template file.

Wolfram Language specification
C specification
Integerintegerint
Realfloatingpoint numberdouble
IntegerListlist of integersint*,long
RealListlist of floatingpoint numbersdouble*,long
Stringcharacter stringchar*
Symbolsymbol namechar*
Manualcall WSTP routines directlyvoid

Basic type specifications.

Here is the WSTP template for a function that takes a list of integers as its argument.
:Begin:
:Function: h
:Pattern: h[a_List]
:Arguments: {a}
:ArgumentTypes: {IntegerList}
:ReturnType: Integer
:End:
Here is the C source code for the function. Note the extra argument which is used to pass the length of the list.
int h(int *a, long alen) {

int i, tot=0;

for(i=0; i<alen; i++)
tot += a[i];

return tot;
}
This installs an external program containing the specifications for the function .
In[1]:=
Click for copyable input
Out[1]=
This calls the external code.
In[2]:=
Click for copyable input
Out[2]=
This does not match the pattern so does not call the external code.
In[3]:=
Click for copyable input
Out[3]=
The pattern is matched, but the elements in the list are of the wrong type for the external code, so $Failed is returned.
In[4]:=
Click for copyable input
Out[4]=

You can mix basic types of arguments in any way you want. Whenever you use or , however, you have to include an extra argument in your C program to represent the length of the list.

Here is an specification.
:ArgumentTypes:  {IntegerList, RealList, Integer}
Here is a possible corresponding C function declaration.
void f(int *a, long alen, double *b, long blen, int c)

Note that when a list is passed to a C program by WSTP its first element is assumed to be at position 0, as is standard in C, rather than at position 1, as is standard in the Wolfram Language.

In addition, following C standards, character strings specified by String are passed as objects, terminated by null bytes. "Portability of WSTP Programs" discusses how to handle special characters.

WSPutInteger32(stdlink,int i)put a single integer
WSPutReal64(stdlink,double x)put a single floatingpoint number
WSPutInteger32List(stdlink,int*a,int n)
put a list of n integers starting from location a
WSPutReal64List(stdlink,double*a,int n)
put a list of n floatingpoint numbers starting from location a
WSPutInteger32Array(stdlink,int*a,int*dims,NULL,int d)
put an array of integers to form a depth d list with dimensions dims
WSPutReal64Array(stdlink,double*a,int*dims,NULL,int d)
put an array of floatingpoint numbers
WSPutString(stdlink,char*s)put a character string
WSPutSymbol(stdlink,char*s)put a character string as a symbol name
WSPutFunction(stdlink,char*s,int n)
begin putting a function with head s and n arguments

WSTP functions for sending data to the Wolfram Language.

When you use a WSTP template file, what mprep and mcc actually do is to create a C program that includes explicit calls to WSTP library functions. If you want to see an example of how to use the WSTP library functions directly, you can look at the source code of this program. Note when you use mcc, you typically need to give a option, otherwise the source code that is generated is automatically deleted.

If your external function just returns a single integer or floatingpoint number, then you can specify this just by giving Integer or Real as the in your WSTP template file. But because of the way memory allocation and deallocation work in C, you cannot directly give specifications such as or . And instead, to return such structures, you must explicitly call WSTP library functions within your C program, and give Manual as the specification.

Here is the WSTP template for a function that takes an integer as an argument, and returns a list of the digits in its binary representation using explicit WSTP functions.
:Begin:
:Function: bits
:Pattern: bits[i_Integer]
:Arguments: {i}
:ArgumentTypes: {Integer}
:ReturnType: Manual
:End:
The return type of the function is declared as .
void bits(int i) {

int a[32], k;
This puts values into the C array .
   for(k=0; k<32; k++) {
a[k] = i%2;
i >>= 1;
if (i==0) break;
}

if (k<32) k++;
This sends elements of the array back to the Wolfram Language.
    WSPutInteger32List(stdlink, a, k);
return ;
}
This installs the program containing the external function .
In[5]:=
Click for copyable input
Out[5]=
The external function now returns a list of bits.
In[6]:=
Click for copyable input
Out[6]=

If you declare an array in C as , then you can use WSPutInteger32Array() to send it to the Wolfram Language as a depth 3 list.

Here is a declaration for a 3dimensional C array.
   int a[8][16][100];
This sets up the array and initializes it to the dimensions of .
   int dims[] = {8, 16, 100};
This sends the 3dimensional array to the Wolfram Language, creating a depth 3 list.
    WSPutInteger32Array(stdlink, a, dims, NULL, 3);

You can use WSTP functions to create absolutely any Wolfram Language expression. The basic idea is to call a sequence of WSTP functions that correspond directly to the FullForm representation of the Wolfram Language expression.

This sets up the Wolfram Language function Plus with 2 arguments.
WSPutFunction(stdlink, "Plus", 2);
This specifies that the first argument is the integer .
WSPutInteger32(stdlink, 77);
And this specifies that the second argument is the symbol .
WSPutSymbol(stdlink, "x");

In general, you first call WSPutFunction(), giving the head of the Wolfram Language function you want to create, and the number of arguments it has. Then you call other WSTP functions to fill in each of these arguments in turn. "Expressions" discusses the general structure of Wolfram Language expressions and the notion of heads.

This creates a Wolfram Language list with 2 elements.
WSPutFunction(stdlink, "List", 2);
The first element of the list is a list of 10 integers from the C array .
WSPutInteger32List(stdlink, r, 10);
The second element of the main list is itself a list with 2 elements.
WSPutFunction(stdlink, "List", 2);
The first element of this sublist is a floatingpoint number.
WSPutReal64(stdlink, 4.5);
The second element is an integer.
WSPutInteger32(stdlink, 11);

WSPutInteger32Array() and WSPutReal64Array() allow you to send arrays which are laid out in memory in the onedimensional way that C preallocates them. But if you create arrays during the execution of a C program, it is more common to set them up as nested collections of pointers. You can send such arrays to the Wolfram Language by using a sequence of WSPutFunction() calls, ending with an WSPutInteger32List() call.

This declares to be a nested list of lists of lists of integers.
int ***a;
This creates a Wolfram Language list with elements.
WSPutFunction(stdlink, "List", n1);
for (i=0; i<n1; i++) {
This creates a sublist with elements.
    WSPutFunction(stdlink, "List", n2);
    for (j=0; j<n2; j++) {
This writes out lists of integers.
        WSPutInteger32List(stdlink, a[i][j], n3);
    }
}

It is important to realize that any expression you create using WSTP functions will be evaluated as soon as it is sent to the Wolfram Language. This means, for example, that if you wanted to transpose an array that you were sending back to the Wolfram Language, all you would need to do is to wrap a Transpose around the expression representing the array. You can then do this simply by calling WSPutFunction(stdlink,"Transpose",1); just before you start creating the expression that represents the array.

The idea of postprocessing data that you send back to the Wolfram Language has many uses. One example is as a way of sending lists whose length you do not know in advance.

This creates a list in the Wolfram Language by explicitly appending successive elements.
In[7]:=
Click for copyable input
Out[7]=
This creates a list in which each successive element is in a nested sublist.
In[8]:=
Click for copyable input
Out[8]=
Flatten flattens out the list.
In[9]:=
Click for copyable input
Out[9]=
Sequence automatically flattens itself.
In[10]:=
Click for copyable input
Out[10]=

In order to call WSPutInteger32List(), you need to know the length of the list you want to send. But by creating a sequence of nested Sequence objects, you can avoid having to know the length of your whole list in advance.

This sets up the List around your result.
WSPutFunction(stdlink, "List", 1);
while( condition ) {
/* generate an element */
Create the next level Sequence object.
    WSPutFunction(stdlink, "Sequence", 2);
Put the element.
    WSPutInteger32(stdlink,  i );
}
This closes off your last Sequence object.
WSPutFunction(stdlink, "Sequence", 0);
WSGetInteger32(stdlink,int*i)get an integer, storing it at address
WSGetReal64(stdlink,double*x)get a floatingpoint number, storing it at address

Basic functions for explicitly getting data from the Wolfram Language.

WSTP provides functions like WSPutInteger32() to send data from an external program into the Wolfram Language. WSTP also provides functions like WSGetInteger32() that allow you to get data from the Wolfram Language into an external program.

The list that you give for in a WSTP template can end with Manual, indicating that after other arguments have been received, you will call WSTP functions to get additional expressions.

:Begin:
:Function: f
The function in the Wolfram Language takes 3 arguments.
:Pattern:        f[i_Integer, x_Real, y_Real]
All these arguments are passed directly to the external program.
:Arguments:      {i, x, y}
Only the first argument is sent directly to the external function.
:ArgumentTypes:  {Integer, Manual}
:ReturnType:     Real
:End:
The external function only takes one explicit argument.
double f(int i) {
This declares the variables and .
   double x, y;
WSGetReal64() explicitly gets data from the link.
   WSGetReal64(stdlink, &x);
WSGetReal64(stdlink, &y);
   return i+x+y;
}

WSTP functions such as WSGetInteger32(link,pi) work much like standard C library functions such as . The first argument specifies the link from which to get data. The last argument gives the address at which the data that is obtained should be stored.

WSCheckFunction(stdlink,"name",int*n)
check the head of a function and store how many arguments it has

Getting a function via WSTP.

:Begin:
:Function: f
The function in the Wolfram Language takes a list of integers as an argument.
:Pattern:        f[a:{___Integer}]
The list is passed directly to the external program.
:Arguments:      {a}
The argument is to be retrieved manually by the external program.
:ArgumentTypes:  {Manual}
:ReturnType:     Integer
:End:
The external function takes no explicit arguments.
int f(void) {
This declares local variables.
    int n, i;
int a[MAX];
This checks that the function being sent is a list, and stores how many elements it has in .
    WSCheckFunction(stdlink, "List", &n);
This gets each element in the list, storing it in .
   for (i=0; i<n; i++)
WSGetInteger32(stdlink, a+i);

In simple cases, it is usually possible to ensure on the Wolfram Language side that the data you send to an external program has the structure that is expected. But in general the return value from WSCheckFunction() will be MLSUCCESS only if the data consists of a function with the name you specify.

Note that if you want to get a nested collection of lists or other objects, you can do this by making an appropriate sequence of calls to WSCheckFunction().

WSGetInteger32List(stdlink,int**a,int*n)
get a list of integers, allocating the memory needed to store it
WSGetReal64List(stdlink,double**a,int*n)
get a list of floatingpoint numbers
WSReleaseInteger32List(stdlink,int*a,int n)
release the memory associated with a list of integers
WSReleaseReal64List(stdlink,double*a,int n)
release the memory associated with a list of floatingpoint numbers

Getting lists of numbers.

When an external program gets data from the Wolfram Language, it must set up a place to store the data. If the data consists of a single integer, as in WSGetInteger32(stdlink,&n), then it suffices just to have declared this integer using .

But when the data consists of a list of integers of potentially any length, memory must be allocated to store this list at the time when the external program is actually called.

WSGetInteger32List(stdlink,&a,&n) will automatically do this allocation, setting a to be a pointer to the result. Note that memory allocated by functions like WSGetInteger32List() is always in a special reserved area, so you cannot modify or free it directly.

Here is an external program that will be sent a list of integers.
int f(void) {
This declares local variables. is an array of integers.
    int n;
int *a;
This gets a list of integers, making be a pointer to the result.
    WSGetInteger32List(stdlink, &a, &n);
This releases the memory used to store the list of integers.
    WSReleaseInteger32List(stdlink, a, n);
...
}

If you use as an specification, then WSTP will automatically release the memory used for the list after your external function exits. But if you get a list of integers explicitly using WSGetInteger32List(), then you must not forget to release the memory used to store the list after you have finished with it.

WSGetInteger32Array(stdlink,int**a,int**dims,char***heads,int*d)
get an array of integers of any depth
WSGetReal64Array(stdlink,double**a,int**dims,char***heads,int*d)
get an array of floatingpoint numbers of any depth
WSReleaseInteger32Array(stdlink,int*a,int*dims,char**heads,int d)
release memory associated with an integer array
WSReleaseReal64Array(stdlink,double*a,int*dims,char**heads,int d)
release memory associated with a floatingpoint array

Getting arrays of numbers.

WSGetInteger32List() extracts a onedimensional array of integers from a single Wolfram Language list. WSGetInteger32Array() extracts an array of integers from a collection of lists or other Wolfram Language functions nested to any depth.

The name of the Wolfram Language function at level i in the structure is stored as a string in . The size of the structure at level i is stored in , while the total depth is stored in d.

If you pass a list of complex numbers to your external program, then WSGetReal64Array() will create a twodimensional array containing a sequence of pairs of real and imaginary parts. In this case, heads[0] will be "List" while heads[1] will be .

Note that you can conveniently exchange arbitraryprecision numbers with external programs by converting them to lists of digits in the Wolfram Language using IntegerDigits and RealDigits.

WSGetString(stdlink,char**s)get a character string
WSGetSymbol(stdlink,char**s)get a symbol name
WSReleaseString(stdlink,char*s)release memory associated with a character string
WSReleaseSymbol(stdlink,char*s)release memory associated with a symbol name

Getting character strings and symbol names.

If you use String as an specification, then WSTP will automatically release the memory that is used to store the string after your function exits. This means that if you want to continue to refer to the string, you must allocate memory for it, and explicitly copy each character in it.

If you get a string using WSGetString(), however, then WSTP will not automatically release the memory used for the string when your function exits. As a result, you can continue referring to the string. Be careful not to modify the contents of the string by writing to the memory that is returned by WSGetString(). When you no longer need the string, you must nevertheless explicitly call WSReleaseString() in order to release the memory associated with it.

WSGetFunction(stdlink,char**s,int*n)
begin getting a function, storing the name of the head in s and the number of arguments in n
WSReleaseSymbol(stdlink,char*s)release memory associated with a function name

Getting an arbitrary function.

If you know what function to expect in your external program, then it is usually simpler to call WSCheckFunction(). But if you do not know what function to expect, you have no choice but to call WSGetFunction(). If you do this, you need to be sure to call WSReleaseSymbol() to release the memory associated with the name of the function that is found by WSGetFunction().