wolframclient.serializers.wxfencoder package

Submodules

wolframclient.serializers.wxfencoder.constants module

wolframclient.serializers.wxfencoder.constants.ARRAY_TYPES_ELEM_SIZE = {b'\x00': 1, b'\x01': 2, b'\x02': 4, b'\x03': 8, b'\x10': 1, b'\x11': 2, b'\x12': 4, b'\x13': 8, b'"': 4, b'#': 8, b'3': 8, b'4': 16}

A set of all valid value type tokens for PackedArray. There is no restriction for NumericArray value types.

wolframclient.serializers.wxfencoder.serializer module

class wolframclient.serializers.wxfencoder.serializer.WXFExprSerializer(stream, expr_provider=None, compress=False, enforce=True)

Bases: object

Main serialization class that convert internal object into bytes.

Pulls instances of WXFExpr from an WXFExprProvider, serializes them into wxf bytes and write the data to a stream.

This class also ensures the output data is a valid WXF encoded expression, and raises an exception otherwise.

For an in-depth description of the format see tutorial/WXFFormatDescription from Wolfram product documentation or visit http://reference.wolfram.com/language/tutorial/WXFFormatDescription.html.

context

provide_wxfexpr(pyExpr)

serialize(pyExpr)

Serialize the python expression given as parameter.

class wolframclient.serializers.wxfencoder.serializer.SerializationContext

Bases: wolframclient.serializers.wxfencoder.serializer._Context

Keeps track of various parameter associated to an expression being serialized. The finalized expression has a tree structure; it is serialized depth first. During the serialization process of an expression involving many non-atomic elements (e.g List), we end up having incomplete parts at various level.

For each level of the expression tree we have to remember the expected length, the number of elements already inserted, and whether or not the node is an association. The first two parameters prevent inconsistencies in the number of elements and the declared length, the last one avoid incorrect use of WXFExprRule(Delayed) tokens.

add_part()

is_rule_valid()

is_valid_final_state()

step_into_new_assoc(length)

step_into_new_expr(length, is_assoc=False)

Indicate the beginning of a new expr of a given length.

Note that the length is the number of WXF elements which includes the head for functions. Association and rules don’t have head in WXF so their length value matches the one of the expression in the Wolfram Language.

step_into_new_function(length)

step_into_new_rule()

exception wolframclient.serializers.wxfencoder.serializer.WXFSerializerException

Bases: Exception

wolframclient.serializers.wxfencoder.streaming module

class wolframclient.serializers.wxfencoder.streaming.ExactSizeReader(reader)

Bases: object

Read exactly the amount of bytes requested and fails otherwise.

read(size=-1)

Read from an underlying readable object.

If size is negative the data is read until EOF. If it is 0 then b’’ is returned. Otherwise the exact amount of bytes is read from the source. More than one call may be necessary.

class wolframclient.serializers.wxfencoder.streaming.ZipCompressedReader(reader)

Bases: object

A buffer implementation reading zip compressed data from a source buffer and returning uncompressed data.

This class is instantiated from a reader, any object implementing a read() method.

CHUNK_SIZE = 8192

read(size=-1)

Read from a compressed stream of bytes and return the inflated byte sequence.

Parameter size specifies the amount of bytes to return at most. If size is set to -1 then the reader is read until EOF is reached.

class wolframclient.serializers.wxfencoder.streaming.ZipCompressedWriter(writer)

Bases: object

flush()

Must be called when closing or destroying the object.

write(data)

Write the compression of data to the underlying buffer writer.

wolframclient.serializers.wxfencoder.utils module

wolframclient.serializers.wxfencoder.utils.array_to_list(data, dimensions, wl_type)

wolframclient.serializers.wxfencoder.utils.array_to_wxf(wxf_token, data, dimensions, array_type_token)

wolframclient.serializers.wxfencoder.utils.float_to_bytes(value)

wolframclient.serializers.wxfencoder.utils.integer_size(value)

wolframclient.serializers.wxfencoder.utils.integer_to_bytes(value, int_size)

wolframclient.serializers.wxfencoder.utils.numeric_array_to_wxf(data, dimensions, wl_type)

wolframclient.serializers.wxfencoder.utils.packed_array_to_wxf(data, dimensions, wl_type)

wolframclient.serializers.wxfencoder.utils.valid_dimension_or_fail(dimension)

wolframclient.serializers.wxfencoder.utils.varint_bytes(int_value)

Serialize int_value into varint bytes and return them as a byetarray.

wolframclient.serializers.wxfencoder.utils.write_varint(int_value, stream)

Serialize int_value into varint bytes and write them to stream, return the stream.

wolframclient.serializers.wxfencoder.wxfencoder module

class wolframclient.serializers.wxfencoder.wxfencoder.DefaultWXFEncoder

Bases: wolframclient.serializers.wxfencoder.wxfencoder.WXFEncoder

The most straight forward serialization of python expressions to their Wolfram Language counterpart.

This class is meant to represent basically JSON like objects, and is intended to be used in all providers. As such it should only deal with obvious convertion.

e.g: int to Wolfram Language Integer, but iterators are not supported but user defined implementation can be added easily.

encode(pythonExpr)

Encode most common Python types to their Wolfram Language counterpart.

exception wolframclient.serializers.wxfencoder.wxfencoder.NotEncodedException

Bases: Exception

Exception used during encoding to signal that a given Python object has been ignored by a WXFEncoder.

class wolframclient.serializers.wxfencoder.wxfencoder.WXFEncoder

Bases: object

Encode a given Python object into a stream of WXFExpr.

This class is meant to be subclassed in order to add support for new classes. The encoder does not have to do anything since more than one can be attached to a given WXFExprProvider. The library provides a default encoder that should cover basic needs, more or less json types, and that should be useful in any case.

To implement a new encoder one needs to sub-class WXFEncoder and implements method encode(). Encode is a generator function that takes a given Python object and instances of WXFExpr. If it returns before yielding anything a NotEncodedException is raised to signal that the encoder is not supporting the given object, and that the encoding must be delegating to the next encoder (if any).

Sometimes it is useful to start a new serialization using the provider, re-entrant call, especially when dealing with non-atomic WXFExpr, such as Function or Association. To do so one must call self.serialize on the target object and yield the results (yield from in PY3).

NOT_PROVIDED = <object object>

encode(o)

The method to implement in sub-classes.

serialize(o)

Re-entrant method used to serialize part of a Python object.

Example: when serializing a custom class foo[{'k1'->1,'k2'->2}], the user defined encoder for class foo could encode it as a function with head ‘foo’ and a dict. The generator would be something similar to:

yield WXFFunction(3)
yield WXFSymbol('foo')
yield from self.serialize({'k1'->1,'k2'->2})

Using a re-entrant call (line 3) allows the dictionnary to be encoded as a new expr; assuming WXFDefaultEncoder is also registered as a provider, the dict will get encoded as an association.

It also enables transformation mechanism, say apply list to all iterable object and pass the result to the provider.

wolframclient.serializers.wxfencoder.wxfexpr module

class wolframclient.serializers.wxfencoder.wxfexpr.WXFExprFunction(length)

Bases: wolframclient.serializers.wxfencoder.wxfexpr.WXFExpr

Functions have a length representing the number of parts (including zero). Each function has a head which is itself a expr, usually a WXFExprSymbol which is not accounted in the length.

length

class wolframclient.serializers.wxfencoder.wxfexpr.WXFExprInteger(value)

Bases: wolframclient.serializers.wxfencoder.wxfexpr.WXFExpr

Integers have various length, from one byte up to eight and are signed values. Values above 2^63-1 are represented with WXFExprBigInteger. Internally WXF uses the two’s complement representation of integer values. The endianness is system independent and is always little-endian.

int_size

to_bytes()

value

class wolframclient.serializers.wxfencoder.wxfexpr.WXFExprString(value)

Bases: wolframclient.serializers.wxfencoder.wxfexpr._WXFExprStringLike

A string of unicode character. The string is always utf8 encoded.

Notabene: Python 3 does not allow utf8 encoding of the surrogate range from 0xD800 to 0xDFFF. Python 2 and the Wolfram Language on the other hand handle those characters as any other unicode code points.

class wolframclient.serializers.wxfencoder.wxfexpr.WXFExprSymbol(value)

Bases: wolframclient.serializers.wxfencoder.wxfexpr._WXFExprStringLike

A symbol represented by a string name. The name is always utf8 encoded.

class wolframclient.serializers.wxfencoder.wxfexpr.WXFExprBinaryString(data)

Bases: wolframclient.serializers.wxfencoder.wxfexpr.WXFExpr

A string of arbitrary bytes. Contrary to WXFExprString no encoding is required.

data

wolframclient.serializers.wxfencoder.wxfexprprovider module

class wolframclient.serializers.wxfencoder.wxfexprprovider.WXFExprProvider(encoder=None, default=None)

Bases: object

Expression provider pull instances of WXFExpr from instances of WXFEncoder.

WXFExprProvider can be initialized with an encoder. If none is provided the default class DefaultWXFEncoder is used to instantiate one. It is possible to add extra encoder using add_encoder. The order in which encoders are called is the one in which they were added. Note that for performance reasons, it is recommended to have the default encoder be first, as such one should not initialize a provider with an encoder except if the default one is not suited for ones needs.

An optional default function can be passed to the provider at initialization. It is used in last resort if no encoder handled the object. It is applied to the object and is expected to transform it to something serializable e.g: a string using default=repr.

One must carefully design the default function to avoid stack overflow.

add_encoder(*encoders)

Add a new encoder to be called last if all others failed to encode an object.

provide_wxfexpr(o)

Main function, a generator of wxf expr.

wolframclient.serializers.wxfencoder.wxfnumpyencoder module

class wolframclient.serializers.wxfencoder.wxfnumpyencoder.NumPyWXFEncoder(packed_array_support=True, numeric_array_support=False)

Bases: wolframclient.serializers.wxfencoder.wxfencoder.WXFEncoder

NumPy array encoder. Encode numpy array as instances of packed array and / or raw array. By default only packed arrays are generated. Unsigned integer data are cast to a type that can fit the maximum value.

It’s possible to add support for raw arrays only for unsigned data, in which case both packed_array_support and numeric_array_support must be true.

>>> NumPyWXFEncoder(packed_array_support=True, numeric_array_support=True)

Finally it’s possible to only output raw arrays with:

>>> NumPyWXFEncoder(packed_array_support=False, numeric_array_support=True)

encode(python_expr)

The method to implement in sub-classes.

Module contents