wolframclient.deserializers.wxf package

Submodules

wolframclient.deserializers.wxf.wxfconsumer module

class wolframclient.deserializers.wxf.wxfconsumer.WXFConsumer

Bases: object

Map WXF types to Python object generating functions.

This class exposes a comprehensive list of methods consuming WXF types. Subclasses can override these members to implement custom parsing logic.

Example implementing a consumer that maps any function with head DirectedInfinity to float(‘inf’):

class ExampleConsumer(WXFConsumer):
    Infinity = wl.DirectedInfinity
    def build_function(self, head, arg_list, **kwargs):
        if head == self.Infinity:
            return float('inf')
        else:
            super().build_function(head, args_list, **kwargs)

Test the new consumer:

>>> wxf = export({'-inf': wl.DirectedInfinity(-1), '+inf': wl.DirectedInfinity(1)}, target_format='wxf')
>>> binary_deserialize(wxf, consumer=ExampleConsumer())
{'-inf': inf, '+inf': inf}

Compare with default result:

>>> binary_deserialize(wxf)
{'-inf': DirectedInfinity[-1], '+inf': DirectedInfinity[1]}

Once initialized, the entry point of a consumer is the method next_expression(). It takes a token generator and returns a Python object. This method is particularly useful when building nested expressions, e.g: build_function(), consume_association(), etc, in order to fetch sub-expressions.

BIGREAL_RE = re.compile('([^`]+)(`[0-9.]+){0,1}(\\*\\^[0-9]+){0,1}')

BUILTIN_SYMBOL = {'False': False, 'Indeterminate': nan, 'Null': None, 'True': True}

See documentation of encode_none() for more information about the mapping of None and Null.

build_function(head, arg_list, **kwargs)

Create a Python object from head and args.

This function can be conveniently overloaded to create specific Python objects from various heads. e.g: DateObject, Complex, etc.

consume_association(current_token, tokens, dict_class=<class 'dict'>, **kwargs)

Consume a WXFToken of type association.

By default, return a dict made from the rules. The named option dict_class can be set to any type in which case an instance of dict_class is returned.

consume_bigint(current_token, tokens, **kwargs)

Consume a WXFToken of type big integer as a int.

consume_bigreal(current_token, tokens, **kwargs)

Parse a WXF big real as a WXF serializable big real.

There is not such thing as a big real, in Wolfram Language notation, in Python. This wrapper ensures round tripping of big reals without the need of ToExpression. Introducing ToExpression would imply to marshall the big real data to avoid malicious code from being introduced in place of an actual real.

consume_binary_string(current_token, tokens, **kwargs)

Consume a WXFToken of type binary string as a string of bytes.

consume_function(current_token, tokens, **kwargs)

Consume a WXFToken of type function.

Return a list if the head is symbol List, otherwise returns the result of build_function() applied to the head and arguments.

Usually custom parsing rules target Functions, but not List. To do so, it is recommended to override build_function().

consume_integer16(current_token, tokens, **kwargs)

Consume a WXFToken of type integer as a int.

consume_integer32(current_token, tokens, **kwargs)

Consume a WXFToken of type integer as a int.

consume_integer64(current_token, tokens, **kwargs)

Consume a WXFToken of type integer as a int.

consume_integer8(current_token, tokens, **kwargs)

Consume a WXFToken of type integer as a int.

consume_numeric_array(current_token, tokens, **kwargs)

Consume a WXFToken of type raw array.

This method return list, and made the assumption that system is little endian.

consume_packed_array(current_token, tokens, **kwargs)

Consume a WXFToken of type packed array.

This method return list, and made the assumption that system is little endian.

consume_real64(current_token, tokens, **kwargs)

Consume a WXFToken of type real as a float.

consume_rule(current_token, tokens, **kwargs)

Consume a WXFToken of type rule as a tuple

consume_rule_delayed(current_token, tokens, **kwargs)

Consume a WXFToken of type rule as a tuple

consume_string(current_token, tokens, **kwargs)

Consume a WXFToken of type string as a string of unicode utf8 encoded.

consume_symbol(current_token, tokens, **kwargs)

Consume a WXFToken of type symbol as a WLSymbol

next_expression(tokens, **kwargs)

Deserialize the next expression starting at the next token yield by tokens.

class wolframclient.deserializers.wxf.wxfconsumer.WXFConsumerNumpy

Bases: wolframclient.deserializers.wxf.wxfconsumer.WXFConsumer

A WXF consumer that maps WXF array types to NumPy arrays.

WXF_TYPE_TO_DTYPE = {b'\x00': 'int8', b'\x01': 'int16', b'\x02': 'int32', b'\x03': 'int64', b'\x10': 'uint8', b'\x11': 'uint16', b'\x12': 'uint32', b'\x13': 'uint64', b'"': 'float32', b'#': 'float64', b'3': 'complex64', b'4': 'complex128'}

consume_numeric_array(current_token, tokens, **kwargs)

Consume a WXFToken of type raw array.

This method return list, and made the assumption that system is little endian.

consume_packed_array(current_token, tokens, **kwargs)

Consume a WXFToken of type packed array.

This method return list, and made the assumption that system is little endian.

wolframclient.deserializers.wxf.wxfparser module

class wolframclient.deserializers.wxf.wxfparser.WXFParser(wxf_input)

Bases: object

Parse a WXF input.

This class is initialized with a WXF input, and exposes a generator of WXFToken. The input wxf_input can be a string of bytes with the serialized expression, a string of unicodes in which case it is considered as a filename, a object implementing a read method.

The generator outputs WXF tokens one by one:

with open('/tmp/data.wxf', 'rb') as fp:
    parser = WXFParser(fp)
    gen = parser.tokens()
    print(next(gen))

This low level class is providing intermediary objects to ease the parsing of WXF. Most of the time one should directly use high level interface such as binary_deserialize().

The token generator is generally consumed by an instance of WXFConsumer.

next_token()

parse_array(token)

parse_header()

token_for_association(token)

token_for_binary_string(token)

token_for_function(token)

token_for_integer16(token)

token_for_integer32(token)

token_for_integer64(token)

token_for_integer8(token)

token_for_numeric_array(token)

token_for_packed_array(token)

token_for_real64(token)

token_for_rule(token)

token_for_string(token)

tokens()

Generate instances WXFToken from a WXF input.

class wolframclient.deserializers.wxf.wxfparser.WXFToken(wxf_type)

Bases: object

Represent a WXF element, often referred as WXF tokens.

array_type

data

dimensions

element_count

length

wxf_type

wolframclient.deserializers.wxf.wxfparser.parse_varint(reader)

Parse a readable binary buffer for a positive varint encoded integer.

Module contents

wolframclient.deserializers.wxf.binary_deserialize(wxf_input, consumer=None, **kwargs)

Deserialize binary data and return a Python object.

Serialize a Python object to WXF:

>>> wxf = export({'key' : [1,2,3]}, target_format='wxf')

Retrieve the input object:

>>> binary_deserialize(wxf)
{'key': [1, 2, 3]}

A stream of WXFToken is generated from the WXF input by a instance of WXFParser.

The consumer must be an instance of WXFConsumer. If none is provided, WXFConsumerNumpy is used. To disable NumPy array support, use WXFConsumer.

Named parameters are passed to the consumer. They can be any valid parameter of next_expression(), namely:

• dict_class: map WXF Association to dict_class in place of a regular dict