Interaction with the Wolfram Language

Wolfram LibraryLink allows dynamic libraries to be directly loaded into the Wolfram Language kernel so that functions in the libraries can be immediately called from the Wolfram Language. You can exchange not only C-like data types such as integers, reals, packed arrays, and strings, but also arbitrary Wolfram Language expressions. In addition, there are useful functions such as sending errors and calling back to the Wolfram Language.

This section describes the functions that the Wolfram Language provides for working with Wolfram Libraries.

Wolfram Language functions for working with libraries.

LibraryFunctionLoad loads a function from a library and returns a LibraryFunction.

Here the LibraryFunction is called with an integer argument.

LibraryFunctionInformation returns information about the LibraryFunction, such as the library from which it was loaded, as well as the argument and return types.

Library Specification

The first argument to LibraryFunctionLoad is the library to load. This can be given as an absolute file name such as /Library/myLibrary.dylib. However, it is often more convenient to give a relative specification to the library, finding it from a path. Another problem is that different platforms use different file name extensions for libraries. The conventions are summarized in the following table.

Extensions for dynamic libraries on different systems.

The Wolfram Language provides a solution to this with FindLibrary and $LibraryPath. FindLibrary can take a platform-independent specification of a library, look for it on $LibraryPath, and, if it finds the library, return the actual file for your system.

The following searches on $LibraryPath and finds the library suitable for your platform; in this example it is Windows.

More information on how $LibraryPath is set up is found in "Locating Libraries".

Function Name

The second argument to LibraryFunctionLoad gives the name of the function to load. This function should be exported from the library, as described in "Library Structure". Note that if you compile the library as C++, you should probably export it with a C naming convention; this is also described in "Library Structure".

Type Specification

The third and fourth arguments to LibraryFunctionLoad specify the types of the arguments and the return type.

Argument and return type specification.

The types that specify tensors are designed to map directly to Wolfram Language packed arrays and also to use tensors in the Wolfram Language compiler. This gives the system great efficiency and allows libraries access to many tensor operations. Tensors can specify the type of each element and the rank explicitly, or they can be left unspecified. Unspecified types and rank give a lot of flexibility for applications that work with tensors. Note that tensors are only supported for Integer, Real, and Complex.

Tensor type specification.

When a tensor is passed as an argument, you can also specify how its memory is handled. This is discussed in detail in the next section.

In the code of your library function, you collect the data from each type from the argument array. Macros for MArgument have been provided to make this collection easy and straightforward. Sample code is shown below.

  mint I0 = MArgument_getInteger(Args[0]);
  double R0 = MArgument_getReal(Args[1]);
  mcomplex C0 = MArgument_getComplex(Args[2]);
  MTensor T0 = MArgument_getMTensor(Args[3]);

When you return from the library function, you must assign to the result. A sample that stores a double is shown below.

  MArgument_setReal(Res, R1);

If you specify tensor input but with arbitrary type and rank, you can find the actual type and rank with the callback functions, as in the following.

  type = libData->MTensor_getType( T0);
  rank = libData->MTensor_getRank( T0);

MTensor_getType returns an integral value that represents the type of the result.

Results for MTensor_getType.

If the library function takes no arguments, then you simply pass an empty list for the input specification.

If the library function does not return a result, you can use a return specification of "Void".

Memory Management of MTensors

Types such as mint, double, and mcomplex are passed to and returned from libraries by value, as is common for calling C functions. The usage in the library is really independent of that in the Wolfram Language.

By contrast, an MTensor is a pointer to a data structure, and thus it is passed by reference. A consequence is that you have to think about how its memory is managed. The Wolfram Language chooses a default technique that is safe and simple, but if you want to pass in large amounts of data or save it for future usage, then you need to think about its management.

MTensor Input Arguments

When you pass an MTensor to a library function, you have a number of options that determine how this is done.

Memory management possibilities for MTensor arguments to library functions.

Automatic Passing

If you select automatic passing, this means that the MTensor is copied before the function is called and it will be cleaned when the function returns. This would be suitable for a function such as the following.

DLLEXPORT int demo_TI_R( WolframLibraryData libData, mint Argc, MArgument *Args, MArgument Res) {
    MTensor T0;
    mint I0;
    mreal R0;
    int err = LIBRARY_NO_ERROR;
    T0 = MArgument_getMTensor(Args[0]);
    I0 = MArgument_getInteger(Args[1]);
    err = libData->MTensor_getReal(T0, &I0, &R0);
    if (err) return err;
    MArgument_setReal(Res, R0);
    return LIBRARY_NO_ERROR;
}

In this function there is no need to free the MTensor. However, you could not save a reference to the MTensor and use it once the function has finished.

An MTensor passed with automatic passing is owned for both read and write access by the library function only as long as the function call is active.

Constant Passing

If you select constant passing, this means that a reference to the MTensor is passed in and it is assumed that your function will not modify the MTensor in any way. This effectively gives you very fast read-only access to the MTensor data. If your code breaks this assumption and does modify the data, a grave error could result in your Wolfram Language session.

An MTensor passed with constant passing is owned by the library function for read access only as long as the function call is active.

Manual Passing

If you select manual passing, this means that the MTensor is copied before the function is called and it is not cleaned when the function returns. This would be suitable for a function such as the following.

DLLEXPORT int demo1_TI_R(WolframLibraryData libData, mint Argc, MArgument *Args, MArgument Res) {
    MTensor T0;
    mint I0;
    mreal R0;
    int err = LIBRARY_NO_ERROR;
    T0 = MArgument_getMTensor(Args[0]);
    I0 = MArgument_getInteger(Args[1]);
    
    err = libData->MTensor_getReal( T0, &I0, &R0);
    libData->MTensor_free(T0);
    if (err) return err;

    MArgument_setReal(Res, R0);
    return LIBRARY_NO_ERROR;
}

Note how the function frees the MTensor. If it were not freed, the memory would be lost. However, instead of freeing the tensor, you could save the tensor and use it in some other part of the library. Finally, if you finish with the tensor, you should call MTensor_free. An alternative would be to return it from a library function to the Wolfram Language in which ownership would pass back to the Wolfram Language.

An MTensor passed with manual passing is completely owned by the library; this continues until the MTensor is freed or passed back to the Wolfram Language.

Shared Passing

If you select shared passing, this means that the MTensor is not copied before the function is called; it is just passed directly to the library function. This would be suitable for a function such as the following.

DLLEXPORT int demo2_TI_R(WolframLibraryData libData, mint Argc, MArgument *Args, MArgument Res) {
    MTensor T0;
    mint I0;
    mreal R0;
    int err = LIBRARY_NO_ERROR;
    T0 = MArgument_getMTensor(Args[0]);
    I0 = MArgument_getInteger(Args[1]);
    
    err = libData->MTensor_getReal( T0, &I0, &R0);
    libData->MTensor_disown( T0);
    if (err) return err;

    MArgument_setReal(Res, R0);
    return LIBRARY_NO_ERROR;
}

When you use shared pass in arguments, the MTensor is shared between the Wolfram Language and your library. The Wolfram Language keeps the MTensor in a table to make sure that its memory does not get collected. When your library no longer wants to use the MTensor, you should call MTensor_disown.

An MTensor passed with shared passing is shared between the library and the Wolfram Language. It will be kept alive until the library and the Wolfram Language have finished all usages.

This loads a sample function using "Shared" to pass in the arguments.

This creates a packed array suitable for using as an argument for the function.

Here you can call the function passing in the array.

However, if you call the function with an argument that is not a packed array, the call will work, but you will get a warning message. This is because the Wolfram Language had to convert the input into a packed array, thereby copying the data and losing one of the advantages of using shared passing.

MTensor Return

When you return an MTensor from a library function, you also control how its memory is managed.

Memory management possibilities for MTensor results from library functions.

Automatic Return

If you select automatic return, this means that the MTensor goes directly back from the library to the Wolfram Language, which will use it as the result of the library function.

DLLEXPORT int demo_I_T(WolframLibraryData libData, 
            mint Argc, MArgument *Args, MArgument Res) {
    MTensor T0;
    mint i, I0, dims[1];
    int err = LIBRARY_NO_ERROR;

    I0 = MArgument_getInteger(Args[0]);
    dims[0] = I0;
    
    err = libData->MTensor_new(MType_Integer, 1, dims, &T0);
    for ( i = 1; i <= I0 && !err; i++) {
        err = libData->MTensor_setInteger( T0, &i, i*2);
    }
    MArgument_setMTensor(Res, T0);
    return err;
}

If the MTensor is owned by the library, i.e. it was created in the library or passed in with manual passing, then once it has been returned to the Wolfram Language it is no longer owned by the library, which should not make any usage of it whatsoever. This is shown in the function above.

If the MTensor is shared between the library and the Wolfram Language, then automatic return does not change anything in the ownership of the MTensor. However, it is rather strange to return a shared MTensor from a library, since the Wolfram Language already has a reference to it.

Shared Return

If you select shared return this means that the MTensor will be shared between the library and the Wolfram Language. Technically this is implemented by adding it to the sharing table when the function returns. This makes sure the MTensor does not get collected. When your library no longer wants to use the MTensor, you should call MTensor_disown (or MTensor_disownAll).

Note that if you call MTensor_disown (or MTensor_disownAll) before the MTensor has been added to the sharing table, it will have no effect and a warning message will be issued.

MTensor Memory Management Summary

One way to understand the three different types of memory management for an MTensor is to consider how the MTensor is "owned" between the different components.

For automatic passing, the MTensor is owned by the library function only when that call to the library is active. The function can return the MTensor whether it has been modified or not, but whatever happens, the MTensor cannot be used after the function has returned.

For manual passing, the MTensor is owned by the library after the function has been called. The MTensor can be used at any time after the function, for example, in another function. The MTensor will continue to be owned by the library until either it is returned by a library function or it is passed to a call to MTensor_free. If you want to return an MTensor that was passed in with manual passing but keep ownership in the library, you will have to copy it with a call to MTensor_clone or return it from a function that uses shared return.

For shared passing and return, ownership of the MTensor is shared between the Wolfram Language and the library. If the Wolfram Language expression that holds the packed array for the MTensor can no longer be reached, the Wolfram Language will no longer have ownership. The library will also keep ownership until you have called MTensor_disown on the MTensor. Note that you have to call MTensor_disown on an MTensor as many times as you have passed it into and returned from a function. For example, if you pass the same packed array into a library three times, then you need to call MTensor_disown three times. The function MTensor_disownAll can be useful to remove all references, and the function MTensor_shareCount gives the actual number of times the MTensor is shared.

A final thing to remember is that if any errors arise, these might transfer control away from the parts of your library function that free memory. In this case you might want to insert your own error handler; this is discussed in the section on errors.

String Arguments

String arguments are passed in with characters encoded using UTF8. When a string is passed as an argument, memory management for the string is left completely up to the program, similarly to the manual passing for an MTensor.

DLLEXPORT int countSubstring(WolframLibraryData libData,
            mint Argc, MArgument *Args, MArgument Res)
{
    char *instring = MArgument_getUTF8String(Args[0]);
    char *substring = MArgument_getUTF8String(Args[1]);
    mint i, n = strlen(instring);
    mint slen = strlen(substring);
    mint c = 0;
    if (n > slen) {
        n -= slen;
        for (i = 0; i <= n; i++) {
            if (!strncmp(instring + i, substring, slen)) {
                c++;
            }
        }
    }
    MArgument_setInteger(Res, c);
    libData->UTF8String_disown(instring);
    libData->UTF8String_disown(substring);
    return LIBRARY_NO_ERROR;
}

Note how the memory for both string arguments is released using UTF8String_disown. If this is not done and you do not otherwise keep a reference to the string argument in your program, the memory will simply be lost. When a string is returned as a result, the Wolfram Language accesses the memory to convert it to its own internal string format, but does not attempt to free the memory. Thus, if your program allocates a string for a result, it also needs to free that memory, but in a separate function from the one setting the string result, since the Wolfram Language needs to access the string memory after the library function has returned.

MSparseArray

A SparseArray in the Wolfram Language can be passed to or from a LibraryFunction where it appears as an argument or result of type MSparseArray. MSparseArray is a pointer to a data structure, and just like with MTensor they are passed by reference. A consequence is that you have to think about how its memory is managed. The Wolfram Language chooses a default technique that is safe and simple, but if you want to pass in large amounts of data or save it for future usage then you need to think about its management. For the most part, memory management for MSparseArray is very much like memory management for MTensor.

SparseArray Type Specification

Arguments or results of type MSparseArray are specified using LibraryDataType.

LibraryFunction type specification for MSparseArray arguments or results.

An argument of type MSparseArray is managed by converting the actual argument passed to a LibraryFunction into a Wolfram Language SparseArray object if it is not already one. If the conversion is successful and the rank is allowed by the argument specification, then type coercion is done. If no type is specified, coercion is done so that the explicit and implicit values are of the same machine number type. If type is specified, then coercion of both explicit and implicit values to the specified type is attempted.

A result of type MSparseArray will always be returned to the Wolfram Language as a SparseArray object unless the rank or one of the dimensions is 0. In these cases, the result is converted to a number or an empty list, respectively.

MSparseArray Data Structure

An MSparseArray is a reflection of the internal structure of SparseArray in the Wolfram Language. While you typically do not need to know much about this structure within the Wolfram Language, it can sometimes be very useful to have an understanding of it for uses in functions.

MSparseArray is stored as an extension of the compressed sparse row (CSR) matrix storage format. In an × matrix with positions that are explicitly stored, values corresponding to each position are stored in a rank 1 MTensor of length . The positions are encoded as follows. A rank 2 MTensor with dimensions {nz,1} and integer type is used to store the column indices starting from row 1 through row . A rank 1 MTensor of length and integer type is used to store the cumulative number of positions in each row, or "row pointers". The last element of this MTensor is always equal to .

For example, consider the matrix where all nonzero values are stored explicitly.

The nonzero values are {1., 2., 1., 4., 3., 1.}.

The column MTensor contains {{1}, {1}, {2}, {1}, {3}, {4}}.

The row pointers MTensor contains {0, 1, 3, 5, 6}.

The format is extended to arrays of any rank as follows. If the rank is 1, a vector of length is effectively stored as a 1× matrix. If the rank is greater than 2, the columns are stored as an MTensor with dimensions {nz,r-1}.

You can see the storage format by looking at the last two parts of the InputForm of a SparseArray object in the Wolfram Language.

In most cases, the implicit value is 0 and this is the default for SparseArray. However, any value can be used, as follows.

Even though the implicit value is 1 in the above example, the data structure is effectively the same. In MSparseArray the implicit value is always stored as a rank 0 MTensor. The type of this MTensor is the type of the data in the MSparseArray; the values, if any, have type consistent with the implicit values.

The MTensors used in the data structure can be accessed using the library callback functions.

Callback functions for accessing the CSR data in MSparseArray.

The MSparseArray callback functions are all in a field sparseLibraryFunctions of WolframLibraryData and they are declared in WolframSparseLibrary.h, so a typical use looks like

#include "WolframLibrary.h"
#include "WolframSparseLibrary.h"
....
MTensor *t;
MSparseArray s;

...
t = (*libData->sparseLibraryFunctions->MSparseArray_getImplicitValue)(s);
...

Note that all of these callback functions return pointers to MTensor. They belong to the MSparseArray data structure and should not be freed using MTensor_free. They will be freed when the MSparseArray is freed. Accessing the data inside of the MTensor allows you to change in place the values in the data structure. The explicit values can be changed fairly safely as long as the memory for the MSparseArray is owned by the library. Changing the implicit value can be done, but often a change in the implicit value implies a different sparse structure, as in the example above, so there is a callback function MSparseArray_resetImplicitValue that will recompute the CSR data structure for the new value. Changing the column indices or the row pointers needs to be done with extreme care—these need to be consistent and any incorrect change could lead to severe problems down the line.

It can be difficult to construct the or modify CSR storage directly, and it is typically easier to work with the actual positions in the array that are represented.

Callback functions for working with explicit positions.

The MTensor constructed by MSparseArray_getExplicitPositions belongs to the library and so should be freed when you are done using it.

In the Wolfram Language, the SparseArray[{pos₁->val₁,pos₂->val₂,…}] syntax for constructing SparseArray objects corresponds to the callback MSparseArray_fromExplicitPositions, and the function ArrayRules corresponds to the callback MSparseArray_getExplicitPositions combined with MSparseArray_getExplicitValues.

MSparseArray Input Argument Memory Management

When you pass an MSparseArray to a library function you have a number of options that determine how this is done.

Memory management possibilities for MSparseArray arguments to library functions.

Automatic Passing

If you select automatic passing, this means that the MSparseArray is copied before the function is called and it will be cleaned when the function returns.

An MSparseArray passed with automatic passing is owned for both read and write access by the library function only as long as the function call is active.

Constant Passing

If you select constant passing, this means that a reference to the MSparseArray is passed in and it is assumed that your function will not modify the MSparseArray in any way. This effectively gives you very fast read-only access to the MSparseArray data. If your code breaks this assumption and does modify the data, a grave error could result in your Wolfram Language session.

An MSparseArray passed with constant passing is available to the library function for read access only as long as the function call is active.

Manual Passing

If you select manual passing, this means that the MSparseArray is copied before the function is called and it is not cleaned when the function returns.

An MSparseArray passed with manual passing is completely owned by the library; this continues until the MSparseArray is freed or passed back to the Wolfram Language.

Shared Passing

If you select shared passing, this means that the MSparseArray is not copied before the function is called; it is just passed directly to the library function.

When you use shared pass in arguments, the MTensor is shared between the Wolfram Language and your library. The Wolfram Language keeps the MSparseArray in a table to make sure that its memory does not get collected. When your library no longer wants to use the MSparseArray, you should call MSparseArray_disown.

An MSparseArray passed with shared passing is shared between the library and the Wolfram Language. It will be kept alive until the library and the Wolfram Language have finished all usages.

Similarly to the case with MTensor, the data is only shared if the Wolfram Language argument given is a SparseArray and no coercion of either the explicit or implicit values is necessary to make an MSparseArray.

MSparseArray Return

When you return an MSparseArray from a library function, you also control how its memory is managed.

Memory management possibilities for MSparseArray results from library functions.

Automatic Return

If you select automatic return, this means that the MSparseArray goes directly back from the library to the Wolfram Language, which will use it as the result of the library function.

If the MSparseArray is owned by the library, i.e. it was created in the library or passed in with manual passing, then once it has been returned to the Wolfram Language it is no longer owned by the library, which should not make any usage of it whatsoever.

If the MSparseArray is shared between the library and the Wolfram Language, then automatic return does not change anything in the ownership of the MSparseArray. However, it is rather strange to return a shared MSparseArray from a library, since the Wolfram Language already has a reference to it.

Shared Return

If you select shared return, this means that the MSparseArray will be shared between the library and the Wolfram Language. Technically this is implemented by adding it to the sharing table when the function returns. This makes sure the MSparseArray does not get collected. When your library no longer wants to use the MSparseArray, you should call MSparseArray_disown (or MSparseArray_disownAll).

Note that if you call MSparseArray_disown (or MSparseArray_disownAll) before the MSparseArray has been added to the sharing table, it will have no effect and a warning message will be issued.

MNumericArray [Experimental]

A NumericArray and ByteArray objects in the Wolfram Language can be passed to or from a LibraryFunction where it appears as an argument or result of type MNumericArray. MNumericArray is a pointer to a data structure, and just like with MTensor, it is passed by reference. A consequence is that you have to think about how its memory is managed. The Wolfram Language chooses a default technique that is safe and simple, but if you want to pass in large amounts of data or save it for future usage then you need to think about its management. For the most part, memory management for MNumericArray is very much like memory management for MTensor.

Type Specification

Arguments or results of type MNumericArray are specified using LibraryDataType.

LibraryFunction type specification for MNumericArray arguments or results.

An argument arg of expression type MNumericArray is managed by checking if arg is a Wolfram Language NumericArray or ByteArray object. Then a reference to the MNumericArray data structure contained in the expression is passed to the function.

A result of type MNumericArray will always be returned to the Wolfram Language as a NumericArray or a ByteArray object.

MNumericArray Data Structure

An MNumericArray is a reflection of the internal structure of NumericArray and ByteArray objects in the Wolfram Language. The MNumericArray data structure holds information about the array, and much of this can be accessed through callback functions.

The most important part of the MNumericArray data structure is an array of data of type corresponding to the NumericArray type. ByteArray is always of type "UnsignedInteger8".

Type	C type	description
"Integer8"	MNumericArray_Type_Bit8	signed 8-bit integers from through
"UnsignedInteger8"	MNumericArray_Type_UBit8	integer 0 through 255
"Integer16"	MNumericArray_Type_Bit16	signed 16-bit integers from through
"UnsignedInteger16"	MNumericArray_Type_UBit16	integer 0 through 65535
"Integer32"	MNumericArray_Type_Bit32	signed 32-bit integers from through
"UnsignedInteger32"	MNumericArray_Type_UBit32	integer 0 through 232-1
"Integer64"	MNumericArray_Type_Bit64	signed 32-bit integers from through
"UnsignedInteger64"	MNumericArray_Type_UBit64	integer 0 through 264-1
"Real32"	MNumericArray_Type_Real32	single-precision reals (32-bit)
"Real64"	MNumericArray_Type_Real64	double-precision reals (64-bit)
"ComplexReal32"	MNumericArray_Type_Complex_Real32	single-precision complex numbers
"ComplexReal64"	MNumericArray_Type_Complex_Real64	double-precision complex numbers

NumericArray types and the corresponding C types.

A callback function is provided that gives access to the array data.

A function to get a pointer to the data.

There are MNumericArray_getType and MNumericArray_convertType functions for getting array type and converting between all supported data types. In addition, there are MNumericArray_getRank, MNumericArray_getDimensions, and MNumericArray_getFlattenedLength that return information about the rank and dimensions of MNumericArray.

The MNumericArray callback functions are all in a field numericarrayLibraryFunctions of WolframLibraryData, and they are declared in WolframNumericArrayLibrary.h, so a typical use looks as follows.

#include "WolframLibrary.h"
#include "WolframNumericArrayLibrary.h"
....
MNumericArray na_in = NULL, na_out = NULL;
numericarray_data_t type = MNumericArray_Type_Undef;
WolframNumericArrayLibrary_Functions naFuns = libData->numericarrayLibraryFunctions;

...
na_in = MArgument_getMNumericArray(Args[0]);
type = naFuns->MNumericArray_getType(na_in);
if(type != MNumericArray_Type_Real32) {
    na_out = naFuns->MNumericArray_convertType(na_in, MNumericArray_Type_Real32,     
             MNumericArray_Convert_Clip_Round);    
}
...

MNumericArray Input Argument Memory Management

When you pass an MNumericArray to a library function, you have a number of options that determine how to do memory management.

Memory management possibilities for NumericArray arguments to library functions.

The same options can be used with a ByteArray object.

Automatic Passing

If you select automatic passing, this means that the MNumericArray is copied before the function is called and it will be cleaned when the function returns.

An MNumericArray passed with automatic passing is owned for both read and write access by the library function only as long as the function call is active.

Constant Passing

If you select constant passing, this means that a reference to the MNumericArray is passed in and it is assumed that your function will not modify the MNumericArray in any way. This effectively gives you very fast read-only access to the MNumericArray data. If your code breaks this assumption and does modify the data, a grave error could result in your Wolfram Language session.

An MNumericArray passed with constant passing is available to the library function for read access only as long as the function call is active.

Manual Passing

If you select manual passing, this means that the MNumericArray is copied before the function is called and it is not cleaned when the function returns.

An MNumericArray passed with manual passing is completely owned by the library; this continues until the MNumericArray is freed or passed back to the Wolfram Language.

Shared Passing

If you select shared passing, this means that the MNumericArray is not copied before the function is called; it is just passed directly to the library function.

When you use shared pass in arguments, the MNumericArray is shared between the Wolfram Language and your library. The Wolfram Language keeps the MNumericArray in a table to make sure that its memory does not get collected. When your library no longer wants to use the MNumericArray, you should call MNumericArray_disown.

An MNumericArray passed with shared passing is shared between the library and the Wolfram Language. It will be kept alive until the library and the Wolfram Language have finished all usages.

MNumericArray Return

When you return an MNumericArray from a library function, you also control how its memory is managed.

Memory management possibilities for MNumericArray results from library functions.

Automatic Return

If you select automatic return, this means that the MNumericArray goes directly back from the library to the Wolfram Language, which will use it as the result of the library function.

If the MNumericArray is owned by the library, i.e. it was created in the library or passed in with manual passing, then once it has been returned to the Wolfram Language it is no longer owned by the library, which should not make any usage of it whatsoever.

If the MNumericArray is shared between the library and the Wolfram Language, then automatic return does not change anything in the ownership of the MNumericArray. However, it is rather strange to return a shared MNumericArray from a library since the Wolfram Language already has a reference to it.

Shared Return

If you select shared return, this means that the MNumericArray will be shared between the library and the Wolfram Language. Technically this is implemented by adding it to the sharing table when the function returns. This makes sure the MNumericArray does not get collected. When your library no longer wants to use the MNumericArray, you should call MNumericArray_disown (or MNumericArray_disownAll).

Note that if you call MNumericArray_disown (or MNumericArray_disownAll) before the MNumericArray has been added to the sharing table, it will have no effect and a warning message will be issued.

MImage

An Image or Image3D in the Wolfram Language can be passed to or from a LibraryFunction where it appears as an argument or result of type MImage. MImage is a pointer to a data structure, and just like with MTensor, they are passed by reference. A consequence is that you have to think about how its memory is managed. The Wolfram Language chooses a default technique that is safe and simple, but if you want to pass in large amounts of data or save it for future usage then you need to think about its management. For the most part, memory management for MImage is very much like memory management for MTensor.

Image Type Specification

Arguments or results of type MImage are specified using LibraryDataType.

LibraryFunction type specification for MImage arguments or results.

An argument arg of expression type MImage is managed by checking if the arg is a Wolfram Language Image or Image3D object consistent with the dimensionality is allowed by the argument specification. If an image type is specified and the actual type does not match the specified type, the equivalent of Image[arg,type] is used to coerce the image type. Then a reference to the MImage data structure contained in the image expression is passed to the function.

A result of type MImage will always be returned to the Wolfram Language as a Image or Image3D object.

MImage Data Structure

An MImage is a reflection of the internal structure of Image and Image3D in the Wolfram Language. The MImage data structure holds information about the image, and much of this can be accessed through callback functions.

The most important part of the MImage data structure is an array of data of type corresponding to the image type.

Image types and the corresponding C types.

The arrangement of this array can depend on dimensionality, channels, and interleaving. Callback functions are provided that give access to the array to single pixels in the array that do not require you to know how the channels are interleaved.

Functions for getting and setting single pixel values.

There are MImage_gettype and MImage_settype unctions for each of the five image types.

The MImage callback functions are all in a field imageLibraryFunctions of WolframLibraryData, and they are declared in WolframImageLibrary.h, so a typical use looks as follows.

#include "WolframLibrary.h"
#include "WolframImageLibrary.h"
....
int err;
raw_t_bit v;
mint channel, pos[3];
MImage img;

...
err = (*libData->imageLibraryFunctions->MImage_getBit)(img, pos, channel, &v);
...

Many operations will require access to the whole array, but you need to know how the channels are interleaved to make the correspondence between pixels, channels, and array index.

Functions for getting the image data array.

MImage Input Argument Memory Management

When you pass an MImage to a library function, you have a number of options that determine how this is done.

Memory management possibilities for Image or Image3D arguments to library functions.

Automatic Passing

If you select automatic passing, this means that the MImage is copied before the function is called and it will be cleaned when the function returns.

An MImage passed with automatic passing is owned for both read and write access by the library function only as long as the function call is active.

Constant Passing

If you select constant passing, this means that a reference to the MImage is passed in and it is assumed that your function will not modify the MImage in any way. This effectively gives you very fast read-only access to the MImage data. If your code breaks this assumption and does modify the data, a grave error could result in your Wolfram Language session.

An MImage passed with constant passing is available to the library function for read access only as long as the function call is active.

Manual Passing

If you select manual passing, this means that the MImage is copied before the function is called and it is not cleaned when the function returns.

An MImage passed with manual passing is completely owned by the library; this continues until the MImage is freed or passed back to the Wolfram Language.

Shared Passing

If you select shared passing, this means that the MImage is not copied before the function is called; it is just passed directly to the library function.

When you use shared pass in arguments, the MImage is shared between the Wolfram Language and your library. The Wolfram Language keeps the MImage in a table to make sure that its memory does not get collected. When your library no longer wants to use the MImage, you should call MImage_disown.

An MImage passed with shared passing is shared between the library and the Wolfram Language. It will be kept alive until the library and the Wolfram Language have finished all usages.

Similarly to the case with MTensor, the data is only shared if the Wolfram Language argument given is a Image and no coercion of either the explicit or implicit values is necessary to make an MImage.

MImage Return

When you return an MImage from a library function, you also control how its memory is managed.

Memory management possibilities for MImage results from library functions.

Automatic Return

If you select automatic return, this means that the MImage goes directly back from the library to the Wolfram Language, which will use it as the result of the library function.

If the MImage is owned by the library, i.e. it was created in the library or passed in with manual passing, then once it has been returned to the Wolfram Language it is no longer owned by the library, which should not make any usage of it whatsoever.

If the MImage is shared between the library and the Wolfram Language, then automatic return does not change anything in the ownership of the MImage. However, it is rather strange to return a shared MImage from a library since the Wolfram Language already has a reference to it.

Shared Return

If you select shared return, this means that the MImage will be shared between the library and the Wolfram Language. Technically this is implemented by adding it to the sharing table when the function returns. This makes sure the MImage does not get collected. When your library no longer wants to use the MImage, you should call MImage_disown (or MImage_disownAll).

Note that if you call MImage_disown (or MImage_disownAll) before the MImage has been added to the sharing table, it will have no effect and a warning message will be issued.

Managed Library Expressions

There may be cases where you may want to use a Wolfram Language expression as a handle for an instance of an object or some data where the memory is managed by the library. By using managed library expressions to access these instances, it is possible to have the instance automatically freed when the expression is no longer referenced in a Wolfram Language session.

Wolfram Language functions for managed library expressions.

C/C++ callback functions for managed library expressions.

Typically it is best to register the library expression manager in library initialization and unregister the manager in library uninitialization so that the manager is active as long as the library is loaded.

/* Initialize Library */
EXTERN_C DLLEXPORT int WolframLibrary_initialize( WolframLibraryData libData) 
{
    return (*libData->registerLibraryExpressionManager)("manager", manage_instance);
}

/* Uninitialize Library */
EXTERN_C DLLEXPORT void WolframLibrary_uninitialize( WolframLibraryData libData) 
{
    int err = (*libData->unregisterLibraryExpressionManager)("manager");
}

The manger function mfun is of type void (*mfun)(WolframLibraryData libData, mbool mode, mint id). If mname has been registered with the manager function mfun, then evaluating CreateManagedLibraryExpression[mname, f] will generate a positive integer ID mid that is unique for the manager with name mname. First f[mid] is evaluated and then the function mfun is called with mode 0 and id equal to mid, and finally the evaluation f[mid] is returned as the result. If at some point during the Wolfram Language session there are no longer any references to the result f[mid], the function mfun is called with mode 1 and id equal to mid. Typically the mode 0 will indicate data creation and mode 1 data freeing.

It is sometimes possible to have hidden references to Wolfram Language expressions, so when it is certain that the library data is no longer needed, releaseManagedLibraryExpression can be used to release the expression. In releasing the expression, releaseManagedLibraryExpression(mname, mid) calls the manager function fun associated with mname with mode 1 and id equal to mid.

An example of using this to set up different instances of simple linear congruential generators is implemented in demo_managed.cxx and shown in the LibraryLink examples under demo_managed.

Library Callback Functions

Sometimes a function needs to repeatedly evaluate another function to do its task. Rootfinding and optimization functions are typical examples.

Callbacks for very general expressions can always be done using the Wolfram Symbolic Transfer Protocol (WSTP) argument interface, but this comes at a cost of some communication overhead. For the repeated evaluation of relatively simple functions it is important to have minimal overhead. This can be achieved by having a limited set of argument types.

The arguments allowed for CompiledFunction are actually a subset of those allowed for LibraryFunction, so connecting to a CompiledFunction for a callback is a natural choice for speed while still keeping reasonable generality. Furthermore, if the CompiledFunction has been compiled with CompilationTarget->"C", the callback can be made at a machine code level.

Wolfram Language functions for managed library expressions.

C/C++ callback functions for library callback functions.

Typically, it is best to register the library callback manager in library initialization and unregister the manager in library uninitialization so that the manager is active as long as the library is loaded.

/* Initialize Library */
EXTERN_C DLLEXPORT int WolframLibrary_initialize( WolframLibraryData libData) 
{
    return (*libData->registerLibraryCallbackFunction)("manager", manage_callback);
}

/* Uninitialize Library */
EXTERN_C DLLEXPORT void WolframLibrary_uninitialize( WolframLibraryData libData) 
{
    int err = (*libData->unregisterLibraryCallbackFunction)("manager");
}

The manger function mfun is of type mbool (*mfun)(WolframLibraryData libData, mint id, MTensor argtypes). If mname has been registered with the manager function mfun, then evaluating ConnectLibraryCallbackFunction[mname, cf] will generate a unique positive integer ID mid; then the function mfun is called with id equal to mid and argtypes a rank 2 MTensor, with each row having the type and rank of the corresponding argument. If there are n arguments, argtypes has length n+1 and the last row is the type and rank of the result. The types are encoded using the macros MType_Integer etc. as defined in WolframLibrary.h. Rank 0 corresponds to scalars. ConnectLibraryCallbackFunction[mname, cf] returns True only mfun returns True. The CompiledFunction cf will be saved until releaseLibraryCallbackFunction(mid) is called, and then, if there are no other references to the CompiledFunction, it will be freed up.

Once connected, the function can be called using callLibraryCallbackFunction with the ID given for the function.

An example that demonstrates how the callback functionality can be used is implemented in demo_callback.c and shown in the LibraryLink examples under demo_callback.

LinkObject Arguments and Result

If you want to send arguments or get results from the library function that are not covered by the MTensor basic numerical types, you can use a LinkObject for arguments and the result. This will let you send the structure of any Wolfram Language expression to your library.

For LinkObject arguments and the result, the library is called in a different way than for basic numerical types. When the function is called, the Wolfram Language writes a list with all the arguments onto a WSTP connection; this is done with LinkWrite, the way the Wolfram Language writes expressions over WSTP. Then it calls the library function. The library function must read the arguments from the link, do its work, and then write its result back onto the link. The Wolfram Language then reads the result from the link with LinkRead, the way that the Wolfram Language reads expressions from WSTP.

An example of a function that works for LinkObject arguments and result is shown below.

DLLEXPORT mint reverseString( WolframLibraryData libData, WSLINK mlp)
{
    mint res = 0;
    int i1, i2, sum;
    int len;
    const char *inStr = NULL;
    char* outStr = NULL;
    
    if ( !WSCheckFunction( mlp, "List", &len)) 
        goto retPt;
    if ( len != 1) 
        goto retPt;

    if(! WSGetString(mlp, &inStr))
        goto retPt;

    if ( ! WSNewPacket(mlp) ) 
        goto retPt;

    outStr = reverseStringImpl(inStr);
    res = WSPutString( mlp,outStr);
retPt: 
    if ( inStr != NULL)
        WSReleaseString(mlp, inStr);
    if ( outStr != NULL)
        free(inStr);
    return res;
}

This loads a sample function that uses LinkObject arguments and result.

This calls the function.

The LinkObject allocated for a given library function is reused in subsequent calls, so it is important that library functions completely read arguments from and write results to the LinkObject every time they are called. A failure to do this may cause the link to become out of sync, which will cause failures in future invocations of that library function.

If an abort is raised during a call to a library function or if a library function returns an error, that function's LinkObject will be immediately disposed of and reallocated after the function returns. In this case, the link can be safely left in any state. A pending abort can be detected using the AbortQ callback.

To use WSTP for communicating with the Wolfram Language, the mathlink.h header must be included before the WolframLibrary.h header.

Locating Libraries

The first argument to LibraryFunctionLoad is the library to load. This can be given as an absolute file name such as /Library/myLibrary.dylib. However, it is more convenient to give a relative specification to the library, finding it from a path. Also, since different platforms use different extensions for dynamic libraries, it can also be a problem to give the extension. The conventions for extensions for dynamic libraries are summarized in the following.

Extensions for libraries on different systems.

Functions in the Wolfram Language that work with dynamic libraries automatically solve this problem. If the input name does not have a file extension, then one suitable for your platform is added. This lets you work with libraries in ways that are independent of the machine on which you work. In addition, the Wolfram Language provides a path, $LibraryPath, to use to find libraries.

A sample setting for $LibraryPath is shown below. Note that it includes a number of Wolfram Language applications that contain libraries.

FindLibrary is the function that finds libraries. It is called by other commands, such as LibraryFunctionLoad. FindLibrary first fixes the extension of the library, then looks for the library as an absolute name, and finally looks for the library on $LibraryPath. It returns the file if it is found.

The following searches on $LibraryPath and finds the library suitable for your platform; in this example it is Windows.

You can also use a name with an extension, but note that this will only work on a platform for which the extension is appropriate.

You can add elements to $LibraryPath. However, a number of folders are automatically added; this includes any LibraryResources folder found in a Wolfram Language application in $UserBaseDirectory/Applications or $BaseDirectory/Applications. This is similar to the way that J/Link can load Java classes, and DatabaseLink can load database resources. It gives a convenient way to bundle libraries with your Wolfram Language work.

Installing Your Own Libraries

If you want to add dynamic libraries to use in your Wolfram Language session, you have a number of options. These are summarized here.

Absolute Pathname

You can assign an absolute pathname to a function such as LibraryFunctionLoad. This is easy to set up, but will cause problems if you move your work to a different computer.

Setting $LibraryPath

You can add the location of your libraries to $LibraryPath. This gives more abstraction over the use of an absolute pathname, since you can set the location once in a variable and use it many times.

In addition, you can use Block to temporarily change the setting of $LibraryPath. An example is shown below.

$BaseDirectory and $UserBaseDirectory

$LibraryPath always includes two locations, $UserBaseDirectory/SystemFiles/LibraryResources/$SystemID and $BaseDirectory/SystemFiles/LibraryResources/$SystemID. If you place your library here, it will be found by library loading functions.

This shows the location in $BaseDirectory.

Applications

If you deliver your code as a Wolfram Language application, you can also include libraries that will be placed on $LibraryPath. You should include a LibraryResources directory with a folder that matches $SystemID. A sample application is shown below.

MyApplication
  MyApplication.m
  Kernel
    init.m
  FrontEnd
  Documentation
    English
  LibraryResources
    Windows
      libraries for use on Windows
    Windows-x86-64
      libraries for use on 64 bit Windows
    Linux
      libraries for use on Linux
    Linux-x86-64
      libraries for use on 64 bit Linux
    MacOSX-x86
      libraries for use on MacOSX
    MacOSX-x86-64
      libraries for use on 64 bit MacOSX

Library Dependencies

If your library depends on other libraries, you will need to make sure that the dependent libraries are available. You could do this by changing the environment in which the Wolfram Language runs to install these extra libraries in some system location or by changing a path environment variable such as LD_LIBRARY_PATH (for Linux) or PATH (for Windows).

An alternative is to use LibraryLoad to load the dependent libraries before loading your own library. LibraryLoad does not return a function (unlike LibraryFunctionLoad), it merely exists to load dependent libraries.

Under Mac OS X, dynamic libraries (which have a file extension of dylib) have a more complicated mechanism where they can be set up to search in particular places for dependent libraries. You should consult the description of commands such as otool and install_name_tool.

If you have trouble loading a library, you can use $LibraryError to find out more about why it did not load. In some cases this returns information about which dependent libraries cannot be loaded.

Library Version Information

You can get information about the version of a library with the LibraryLink Package. This supplies some extra tools for working with shared libraries.

To use the package you have to load it.

This finds a library.

Here, LibraryVersionInformation returns a list of rules that shows information about the library.

You can also get the version information as a string with LibraryVersionString, as shown in the following.

Problems Loading a Library

If you have problems loading a library, you might want to use $LibraryError to find out more about why it did not load. This is found in the LibraryLink` context, but you do not have to load the package to use it.

Here, a library used for calendar calculations is loaded.

The library has dependencies on other libraries. Normally, the Wolfram Language will load the other libraries first. However, if you just try to load the calendar library directly (and you have not already used a calendar function) this load instruction will fail, as in the following.

You can use $LibraryError to find out more about the error, as shown below.

Windows does not give the names of dependent libraries that cannot be found. On other platforms, this information is available.