NetExternalObject

NetExternalObject[…]

represents a net model in an external framework format.

Details

  • NetExternalObject is typically used to run inference using neural network models not defined in the Wolfram Language.
  • Supported file formats are:
  • "ONNX"ONNX models
    "MXNet"MXNet models
    "GGUF"GGUF models
  • For a model with a single input port, NetExternalObject[…][data] gives the result of applying the model to data.
  • For a model with multiple input ports, NetExternalObject[…][<|port1data1,…|>] provides datai to porti.
  • When data is given as NumericArray, the output will be a NumericArray. »
  • Information[NetExternalObject[…]] generates an information panel about the model. »
  • Information of a NetExternalObject may include the following properties:
  • "Backend"the evaluation back end used by the model
    "ContextWindowSize"current maximum size of the model context windows
    "InputPortNames"list of names of input ports
    "InputPorts"association of input port shapes
    "OutputPortNames"list of names of output ports
    "OutputPorts"association of output port shapes
    "Properties"available properties
    "RawData"unprocessed native contents of the model
    "TrainingContextWindowSize"original size of the model context windows
  • NetExternalObject[…][data,opts] specifies that options should be used in applying the net to data. Possible options include:
  • Method Automaticspecify details of the model evaluation
    TargetDevice "CPU"the device on which to perform evaluation
  • Possible settings for TargetDevice change depending on the system and the hardware and include:
  • "CPU"use the CPU
    "GPU"use available dedicated hardware
    "CoreML"Apple CoreML framework
    "CUDA"Nvidia CUDA API
    "DirectML"Microsoft Direct Machine Learning API
  • For a NetExternalObject with LLaMA.cpp backend ("GGUF" format) that generates text, option Method can be set to a list of rules having any of the following keys:
  • "MaxTokens"Automaticmaximum number of generated tokens
    RandomSeedingAutomaticwhat seeding of pseudorandom generators should be done internally
    "PrependPrompt"Falsewhether to prepend the input to ext to the output text
    "ResetState"Truewhether to reset the language model's state before processing the input
    "StopTokens"{}a list of strings that should stop the generation if produced

Examples

open allclose all

Basic Examples  (2)

Import an ONNX model as an external network:

Evaluate the model on some input data:

Import a quantized image classifier:

Define a minimal pipeline to pre- and post-process the data:

Run the external model:

Scope  (5)

Formats  (3)

Import an ONNX model as a NetExternalObject:

Import a net in GGUF format (model credit: Olusegun Odewole, https://huggingface.co/segestic/Tinystories-gpt-0.1-3m ):

Import an MXNet model as a NetExternalObject:

Properties  (2)

Extract a property of the external net using Information:

Get a list of properties:

Get a list of all the supported properties:

Options  (4)

Method  (2)

Import a text-generating model:

Run the model until a termination token is reached:

Prepend the prompt to the generated text:

Specify a maximum amount of tokens to generate:

Text generation is a random process:

Fix the seed of the pseudorandom generator to get reproducible results:

TargetDevice  (2)

Import an ONNX file and run it on the CPU:

Run the model using CoreML:

Some models will run faster when evaluated on specialized hardware:

Timing on CPU:

Timing on CoreML:

Applications  (3)

Run a real-time face detector:

Define a transformation from an image to a list of arrays:

Run the detector on the encoded image:

Extract the bounding box and probability information from the result:

Visualize the detections, filtering the ones with low probability or large overlap:

This network can analyze more that 40 images per second on a consumer laptop:

Import a text-generating model:

Run the model until a termination token is reached:

Import a "GGUF" file as an embedding model:

Generate an embedding vector for each token in a string:

Properties & Relations  (2)

Input given as a NumericArray will cause the output to also be a NumericArray:

External net models typically have a hard-coded batch dimension:

Ordinary networks do not have an explicit batch dimension:

They work both on single examples and batches of any size:

Possible Issues  (2)

External models cannot be modified:

External models cannot be retrained:

Wolfram Research (2022), NetExternalObject, Wolfram Language function, https://reference.wolfram.com/language/ref/NetExternalObject.html (updated 2025).

Text

Wolfram Research (2022), NetExternalObject, Wolfram Language function, https://reference.wolfram.com/language/ref/NetExternalObject.html (updated 2025).

CMS

Wolfram Language. 2022. "NetExternalObject." Wolfram Language & System Documentation Center. Wolfram Research. Last Modified 2025. https://reference.wolfram.com/language/ref/NetExternalObject.html.

APA

Wolfram Language. (2022). NetExternalObject. Wolfram Language & System Documentation Center. Retrieved from https://reference.wolfram.com/language/ref/NetExternalObject.html

BibTeX

@misc{reference.wolfram_2024_netexternalobject, author="Wolfram Research", title="{NetExternalObject}", year="2025", howpublished="\url{https://reference.wolfram.com/language/ref/NetExternalObject.html}", note=[Accessed: 17-January-2025 ]}

BibLaTeX

@online{reference.wolfram_2024_netexternalobject, organization={Wolfram Research}, title={NetExternalObject}, year={2025}, url={https://reference.wolfram.com/language/ref/NetExternalObject.html}, note=[Accessed: 17-January-2025 ]}