This section covers functions that create compiled code, features of compiled code objects such as serialization, cross-compilation, and packaging for the Wolfram Compiler.

Compiling Code

The Wolfram Compiler provides a number of functions that actually compile code. These are summarized in this section. This is not going to replicate the function pages for these functions, which are quite extensive.

FunctionCompile

FunctionCompile compiles Wolfram Language programs into optimized compiled code that can be immediately executed. It works by creating executable machine code that can be executed directly, and it does not require any external tools to be installed. The compiled code created can be reloaded and used without having to rerun the compilation.

FunctionCompile takes a Wolfram Language function and returns a compiled code function, as shown in this example:

It can take a list of functions and return a list of compiled code functions:

It can also take an association of functions and return an association of compiled code functions:

Declarations can be used with FunctionCompile:

The result of FunctionCompile is a CompiledCodeFunction object:

Compiled code can be saved in notebook documents or external files. When it is read back in, the compiled code is recreated:

The recreated compiled code works in the same way as the original compiled code:

More information on CompiledCodeFunction objects is found in a later section.

FunctionCompile has a number of options:

Some of the main options are discussed in the following.

The CompilerEnvironment option is used to set a specific environment to use for compilation. It is described in the section on using compiler declarations. The ProgressReporting option is used to control the system that shows progress during compilation. It is described in the section on using compiler tooling. The TargetSystem option is used to control the systems for which output is generated. It is described in the section on cross compilation.

CompilerRuntimeErrorAction

The CompilerRuntimeErrorAction option is used to set the behavior of generated code when there is a runtime error.

The default value is Automatic, and this means that when execution of a compiled function encounters an error, the computation is rerun in the Wolfram Language evaluator. An example is shown below that adds two "UnsignedInteger8" integers; the result is also an "UnsignedInteger8" integer:

This addition works fine:

The result here cannot be represented in the result type, and there is a runtime error. The computation is rerun in the evaluator, returning the result as an expression:

With the default option setting of Automatic, there is some analysis of the code, and if it contains compiler-specific code, it does not get rerun in the evaluator. For example, the following contains uses of raw pointer function that can only be executed in compiled code:

When there is a runtime error, there is a message and a failure is returned:

If CompilerRuntimeErrorAction is set to None, then runtime errors do not cause the computation to be rerun in the evaluator:

There is a message and a failure is returned:

FunctionCompileExportLibrary

FunctionCompileExportLibrary compiles Wolfram Language programs, generating shared libraries that are suitable for loading into a Wolfram Language session. It does require the installation of external tools that can create shared libraries. On many platforms, these are already present or can be easily installed.

FunctionCompileExportLibrary can take the name of a target library and a Wolfram Language function, as shown in this example:

LibraryFunctionLoad can load code in the library; it returns a compiled code object:

The compiled code works in a similar way to that created by FunctionCompile:

The name of the library can have a platform-specific extension and does not have to be relative:

FunctionCompileExportLibrary can take a list of functions and return a list of compiled code functions:

Loading the library returns a list of compiled code functions:

It can also take an association of functions and return an association of compiled code functions:

Loading the library returns an association of compiled code functions:

Declarations can be used with FunctionCompileExportLibrary:

FunctionCompileExportLibrary has a number of options:

The CompilerEnvironment option is used to set a specific environment to use for compilation. It is described in the section on using compiler declarations. The ProgressReporting option is used to control the system that shows progress during compilation. It is described in the section on using compiler tooling.

It should be noted that FunctionCompileExportLibrary creates a library that contains functions that can be loaded into the Wolfram Language interpreter. If you wish to make a library that contains raw C functions that can be used, for example, in compiled code, you should be using the raw library functions setting for Compiled Components.

FunctionCompileExport

FunctionCompileExport exports compiled code into a variety of different formats, saving the result in a file. Formats such as raw machine code, assembler, LLVM intermediate representation and Wolfram intermediate representation are supported.

In the following, the native machine is saved into a file in binary form:

Since it is in binary form, it is quite hard to see the actual contents of the file:

If a different format is chosen, this can help to see the output. This shows assembler output:

FunctionCompileExport works for other formats, such as "Binary", which generates an object file (which could be used with a linker, "LLVM", that generates textual LLVM code, and "LLVMBinary", which generates a binary form of LLVM code. There are examples of these on the FunctionCompileExport reference page.

Declarations can also be given, and examples are on the reference page.

FunctionCompileExport has a number of options:

The CompilerEnvironment option is used to set a specific environment to use for compilation. It is described in the section on using compiler declarations. The ProgressReporting option is used to control the system that shows progress during compilation. It is described in the section on using compiler tooling.

The option TargetSystem can be used to choose a different architecture for output. The following shows an assembler generated for a compilation on a Windows-x86-64 platform:

FunctionCompileExportString

FunctionCompileExportString exports compiled code into a variety of different formats, returning the output in a String. Formats such as raw machine code, assembler, LLVM intermediate representation and Wolfram intermediate representation are supported.

The default form of FunctionCompileExportString returns formatted LLVM intermediate representation:

This shows the Wolfram intermediate representation:

Other formats include "LLVMRaw", which returns unformatted LLVM code, "Assembler", which returns assembler, and "WolframFinalIR", which returns the final Wolfram intermediate code. There are examples of these on the FunctionCompileExportString reference page.

Declarations can also be given, and examples are on the reference page.

FunctionCompileExportString has a number of options:

The CompilerEnvironment option is used to set a specific environment to use for compilation. It is described in the section on using compiler declarations. The ProgressReporting option is used to control the system that shows progress during compilation. It is described in the section on using compiler tooling.

The option TargetSystem can be used to choose a different architecture for output. The following shows assembler generated for a compilation on a Windows-x86-64 platform:

FunctionCompileExportByteArray

FunctionCompileExportByteArray exports compiled code into a variety of binary formats, returning the output in a ByteArray. Formats include binary forms of machine code and LLVM intermediate representation:

There are more examples on the FunctionCompileExportByteArray reference page.

CompiledCodeFunction

CompiledCodeFunction objects are the way that the Wolfram Compiler holds executable compiled code. They are returned by FunctionCompile, as in this example:

A key way they work is to take arguments, call the compiled code and return a result:

Some of the details of how this fits with the Wolfram Language evaluator, for example, converting into and out of expressions, was covered in the section on expression serialization.

You can see details about a compiled code function with Information:

For example, the return type can be returned:

One interesting item concerns the LLVM binary that is stored:

This is the actual LLVM code that is used in compilation. This gets used for serialization and cross-compilation features.

Serialization

Wolfram Language expressions are the single uniform data type that is processed by the Wolfram Language interpreter. Expressions are a flexible and powerful way to work with data and form the core basis of the Wolfram Language.

A key aspect of Wolfram expressions is that they can be serialized; they have a structure that is accessible to Wolfram Language programs. Everything they need to operate is contained in their expression structure. This gives many benefits, such as storage and reuse. The Wolfram Compiler has been carefully designed to support serialization for compiled code.

This can be demonstrated with a simple compilation:

The compiled code expression has internal structure; generally, it is easier to see this with Information. One element is the LLVM code; this can be seen as follows:

Storing the LLVM code is very useful because it allows the compiled code to be saved. Then when it is read back in, the compiled code is recreated:

The recreated compiled code works in the same way as the original compiled code:

When the compiled code is read back in, the original Wolfram Language code does not have to be compiled; instead, the LLVM code is used. The Wolfram Compiler still has to be initialized, but generating the actual executable is faster.

One issue of the LLVM code is that it is not quite the same on all platforms. So if compiled code is generated on one platform, it can only be read back into that platform. There is a solution to this, which is to use the TargetSystem option. This can generate compiled code for all common Wolfram Language platforms. An example is shown below:

Now the compiled code contains LLVM for several different platforms, and when it can be generated on one platform and read back into a different one:

This feature of compiling code that targets a different platform from the platform on which the compiler runs is known as cross-compilation. It is covered in a later section.

Function Passing

It is possible to use a CompiledCodeFunction as an argument to another compiled function.

This is the function that accepts compiled code as an argument. It also takes an argument for the function:

This generates a function to pass an argument:

Calling the outer function with the function argument gives the expected result:

If the type signature of the function argument does not match what is expected, there will be an error:

Cross-Compilation

The Wolfram Compiler supports cross-compilation, which means it can create executable code for a platform other than the one on which it is running.

For an example, this creates some compiled code:

The compiled code holds LLVM code for only one platform:

The LLVM code matches the platform under which the Wolfram Language is running:

The LLVM code is used when the compiled code expression is loaded back into a Wolfram Language session.

However, you can request that LLVM code for additional platforms also be created:

Now the compiled code contains LLVM code for MacOSX-ARM64 and for Windows-x86-64:

This means that if the CompiledCodeFunction expression was loaded into a 64-bit Windows machine running the Wolfram Language, it would work without having to recompile the code.

This is a list of all the target systems that the Wolfram Compiler can support:

If the TargetSystem option is set to Automatic, code is generated for common platforms:

When the TargetSystem option is set to All, code is generated for all platforms on $TargetSystems:

If any declarations are passed into FunctionCompile, these can be used in the compilation:

CompilerEnvironment

When FunctionCompile is compiling for multiple platforms, it looks for a target system in the compiler environment. This shows the TargetSystem for the default compiler environment on this platform:

You can create a compiler environment that contains several target systems:

This returns the target systems in the new compiler environment:

Now you can use this compiler environment in FunctionCompile. It is quicker to run because it does not need to create all the environments when it runs:

You can append declarations to a compiler environment that contains multiple targets. They are appended to each target:

Now you can carry out a compilation that uses the compiler environment with the declarations:

It would also be possible to set the default environment, $CompilerEnvironment, to a custom environment that contained multiple declarations and multiple targets. Compilation would be quicker because it would not have to create the environments as they were needed.

Compiled Components

Compiled components are a way the Wolfram Compiler provides to collect compiler declarations into a named group. In addition to declarations, components can contain various deployment options to be specified, built and loaded.

Typically, components are placed into files contained in a paclet, but this is not necessary. You can work with components outside of files, and often this is a good way to learn about working with compiled components and to start to develop new ones.

A key usage of a component is to provide declarations. This is shown in the following for a component called "Demo":

The declarations can also be given with a setting of "Declarations":

Now you can use the component in ways you can use declarations. For example, in the following:

This calls the code:

You can also use the component with compiler environments, as shown in the following:

This calls the code:

The compiled component expression can be worked with:

There are various properties that can be obtained from the component expression. They are listed here:

For example, these are the declarations that are available in the component:

Library Functions

A compiled component can contain settings for "LibraryFunctions", which are used to build a library. The functions are made available when the component is loaded.

Typically, a component always contains some declarations, as in shown in this example component called "DemoLibrary", which has a function declaration:

Now a "LibraryFunctions" setting is given. This includes code to use in functions exported from the library with a name. Functions exported from a library must have names that are strings. In this example, there is one exported function, called "Root":

This shows the "Declarations":

This shows the "LibraryFunctions":

The library can be built with BuildCompiledComponent. It returns a Success object if building works:

The component can then be loaded:

The library functions in the library for the component can be called:

A key aspect of BuildCompiledComponent is that it can take a destination folder. This creates a temporary folder:

Now the component is built, with the library going into this folder:

The library was built and can be found by FindLibrary:

Placing the library for a component is useful when the component is embedded in a paclet.

Installed Functions

A compiled component can contain settings for "InstalledFunctions", which are used to build a library. The functions are made available when the component is loaded.

Typically, a component always contains some declarations, as in shown in this example component called "DemoInstalled", which has a function declaration:

Now an "InstalledFunctions" setting is given. This includes code to use in functions exported from the library with a name. The installed functions must have names that are symbols. In this example, there is one installed function, called InstalledRoot:

This shows the "Declarations":

This shows the "InstalledFunctions":

The library can be built with BuildCompiledComponent. It returns a Success object if building works:

The component can then be loaded:

The installed functions in the library for the component can be called directly, as in the following:

Raw Library Functions

The raw library functions of a compiled component are functions that are exported from a library for the component, which can be called by other compiled code.

Typically, a component always contains some declarations, as in shown in this example component called "DemoRawLibrary", which has a function declaration:

Now a "RawLibraryFunctions" setting is given. Each setting contains the name of the function to be exported from the library. This must be a string, and this example is called "RawRoot". It also contains a type specification of the name of a function to be compiled; in this example, it is called RawRoot:

This shows the "Declarations":

This shows the "RawLibraryFunctions":

The library can be built with BuildCompiledComponent. It returns a Success object if building works:

The component can then be loaded:

The raw library function can be used in a number of ways. It can be loaded from the component by referring to the CompiledComponentRawInterface:

This works as expected:

You can see how this works by inspecting the Wolfram intermediate code with FunctionCompileExportString. This shows that a definition for RawRoot has been made to call the library function from the component:

You could make your own LibraryFunctionDeclaration to call the function in the library:

When you use CompiledComponentRawInterface, the Wolfram Compiler is simply making the library functions declarations for you. So it is a bit easier.

The benefit of the CompiledComponentRawInterface is that you can put complicated code into a component and compile it into a library. Then you can call it in other compiled code, and it will just link to the library. Instead, you could just use the declarations in the component, which is done in the following. Note that the code for the implementation of RawRoot now appears in the output, instead of as a library function call:

A final way to use the raw library interface is from other tools that can consume C interface libraries. One example of this is to use ForeignFunctionLoad. The following creates a ForeignFunction object:

When it is called, it returns the expected result:

Typically, you would probably use the Wolfram Compiler to call the code in a raw library, but showing how it can be used in other ways can help to explain how it works.

Removing a Component

When you are developing a compiled component, sometimes you might want to clear all the settings that you have given. This can be done with DeleteObject.

For an example, here is a component called "DemoComponentX":

These are the declarations:

These are the library functions:

The deletes the component:

Now the component is removed, and you can start with new settings:

Paclets

A typical deployment of compiled components would be to include the code in a paclet. Paclets are units of Wolfram functionality that are packaged in a way to allow them to be used in the Wolfram environment. There is a good tutorial that gives information on the paclet system. What is contained here is really just specific to uses of the paclet system with the Wolfram Compiler and specifically, compiler components.

One thing to remember is that a lot of the Wolfram Language distribution is made up of paclets.

A key part of a paclet is the PacletInfo.wl file, which contains metadata about the paclet. The following is a basic paclet called CompilerDemoUtilities:

Paclet[
    "Name"  "CompilerDemoUtilities",
    "Version"  "1.0",
    "Extensions"  {
        {"Kernel", "Root"  "Kernel", "Context"  "CompilerDemoUtilities`"},
        {"Compiler", "Components"  "CompilerDemo"},
        {"LibraryLink"}
    }
]

The "Kernel" extension shows that there should be a file "Kernel/CompilerDemoUtilities.wl". This will be loaded with the paclet. Sample contents are shown below:

BeginPackage["CompilerDemoUtilities`"]

ComputeSquare

Begin["`Private"]

DeclareCompiledComponent["CompilerDemo",
    "Declarations"->{
        FunctionDeclaration[ComputeSquare,Typed[{"Real64"}->"Real64"]@Function[arg,arg^2]]}]
End[]
EndPackage[]

This contains a DeclareCompiledComponent for the component "CompilerDemo". In a real usage, there would probably be significantly more content.

The PacletInfo.wl also has a "Compiler" extension, which declares that the paclet contains a compiled component called "CompilerDemo".

Finally, there is a "LibraryLink" extension, which tells the paclet system that functions that look for libraries should look inside the LibraryResources/SystemID folder of this paclet.

Once all the declarations have been made, if there is to be a library for the component, it can be built with BuildCompiledComponent and placed into the folder of libraries of the paclet. Then the paclet will be ready to be distributed.

You can find the location of a paclet that has been installed from the PacletObject expression. The following shows the location of the "Compile" paclet (which contains a lot of the compiler implementation):

If you were going to install a library for a component into this paclet, the following would be a good location:

You could use the second argument of BuildCompiledComponent, which accepts a location for the library it builds.