
-------------------------------------------------------------------------------
What is an encoder?

Encoders are functions which accept a python object associated with
the field being serialized and convert it to and return some raw form
(usually a bytes object).

Encoders are usually fairly simple, especially if built-ins already exist
to do most of the encoding for you. For more esoteric data types, an encoder
may get a bit lengthy and complex.


Here is the most simple encoder in the library:

def encode_numeric(self, node, parent=None, attr_index=None):
    '''
    Encodes a python int into a bytes representation.
    Encoding is done using struct.pack

    Returns a bytes object encoded represention of the "node" argument.
    '''
    return pack(self.enc, node)

It takes an int and runs the struct modules pack function on it using the
encoding stored in the FieldType instance as the unpack format code.


Here is an encoder that is more complex:

def encode_24bit_numeric(self, node, parent=None, attr_index=None):
    '''
    Encodes a python int to a signed or unsigned 24-bit bytes representation.

    Returns a bytes object encoded represention of the "node" argument.
    '''
    if self.enc[1] == 'i':
        # int can be signed
        assert node >= -0x800000 and node <= 0x7fffff, (
            '%s is too large to pack as a 24bit signed int.' % node)
        if node < 0:
            # int IS signed
            node += 0x10000000
    else:
        assert node >= 0 and node <= 0xffffff (
            '%s is too large to pack as a 24bit unsigned int.' % node)

    # pack and return the int
    if self.endian == '<':
        return pack('<I', node)[0:3]
    return pack('>I', node)[1:4]

This decoder must manually set the sign bit if the int being encoded
is signed, as the struct.pack function cant handle 24-bit ints.
It also requires manually checking the range of the int since an
int too large to fit in 3 bytes could be given, but the encoder
might happily pack it and the most significant byte be pruned.


Because encoders are designed to only be used by a FieldTypes serializer
function, the return value can be anything as long as the encoder returns
what the serializer is expecting to recieve. Most of the time the return value
is a bytes string and the serializer simply writes it to where it should go in
the output buffer. A current exception to this is bitints, where the encoder
instead returns an unsigned int, offset, and mask and the serializer masks
and shifts it into the bytes it should be spread across.


Here is an example of a bitint encoder:

def encode_bit_int(self, node, parent=None, attr_index=None):
    '''
    Encodes arbitrarily sized signed or unsigned integers
    on the bit level in either ones or twos compliment

    Returns the encoded int, the offset it should be
    shifted to, and a mask that covers its range.
    '''

    bitcount = parent.get_size(attr_index)
    offset = parent.ATTR_OFFS[attr_index]
    mask = (1 << bitcount) - 1

    # if the number is signed
    if node < 0:
        signmask = 1 << (bitcount - 1)
        if self.enc == 'S':
            return(2*signmask + node, offset, mask)
        return(signmask - node, offset, mask)
    return(node, offset, mask)


-------------------------------------------------------------------------------
Where and why are they used?

Encoders are used only by the serializer function of the field they are in.
Encoders exist to separate the task of encoding data into a python object from
the task of actually writing the data to the output buffer. This is because a
serializer is usually many lines of code, whereas an encoder is much shorter.

By allowing FieldType instances to have an encoder, generic serializers can be
written that work for many different types of data(integers, strings,
half-floats). This makes it easy to implement FieldTypes for new data types
while requiring less lines of code since a serializer is being reused.

Encoders are not always needed however. FieldTypes which describe something
other than data dont use encoders. Any hierarchy FieldTypes(such as Struct,
Container, Array, Switch, and BitStruct) wont use encoders as they dont
represent data. Other times one may simply want their FieldType to be more
optimized and decide to build the encoder into the serializer.
These same statements apply to decoders.


-------------------------------------------------------------------------------
What positional and keyword arguments should an encoder
function expect and what are their purposes?

required positional:
    self -------- The FieldType instance whose encoder function is being run.

    node -------- The node which is being encoded into a bytes string.

keyword:
    parent ------ The Block that the object being encoded is parented to.
                  If this argument is to be considered valid, attr_index must
                  be provided and valid as well.

    attr_index -- The index that node can be found under in parent by doing:
                      parent[attr_index].
                  If this is None, assume the parent argument is not valid.