A lifted mutable byte array type wrapping MutableByteArray# RealWorld. This is a low level array used to back high level unboxed arrays and serialized data.

Return True if the array is allocated in pinned memory.

Return a copy of the array in pinned memory if unpinned, else return the original array.

Return a copy of the array in unpinned memory if pinned, else return the original array.

The Unbox type class provides operations for serialization (unboxing) and deserialization (boxing) of fixed-length, non-recursive Haskell data types to and from their byte stream representation.

Unbox uses fixed size encoding, therefore, size is independent of the value, it must be determined solely by the type. This restriction makes types with Unbox instances suitable for storing in arrays. Note that sum types may have multiple constructors of different sizes, the size of a sum type is computed as the maximum required by any constructor.

The peekAt operation reads as many bytes from the mutable byte array as the size of the data type and builds a Haskell data type from these bytes. pokeAt operation converts a Haskell data type to its binary representation which consists of size bytes and then stores these bytes into the mutable byte array. These operations do not check the bounds of the array, the user of the type class is expected to check the bounds before peeking or poking.

IMPORTANT: The serialized data's byte ordering remains the same as the host machine's byte order. Therefore, it can not be deserialized from host machines with a different byte ordering.

Instances can be derived via Generics, Template Haskell, or written manually. Note that the data type must be non-recursive. WARNING! Generic and Template Haskell deriving, both hang for recursive data types. Deriving via Generics is more convenient but Template Haskell should be preferred over Generics for the following reasons:

1. Instances derived via Template Haskell provide better and more reliable performance.
2. Generic deriving allows only 256 fields or constructor tags whereas template Haskell has no limit.

Here is an example, for deriving an instance of this type class using generics:

>>> import GHC.Generics (Generic)
>>> :{
data Object = Object
    { _int0 :: Int
    , _int1 :: Int
    } deriving Generic
:}

>>> import Streamly.Data.MutByteArray (Unbox(..))
>>> instance Unbox Object

To derive the instance via Template Haskell:

import Streamly.Data.MutByteArray (deriveUnbox)
$(deriveUnbox [d|instance Unbox Object|])

See deriveUnbox for more information on deriving using Template Haskell.

If you want to write the instance manually:

>>> :{
instance Unbox Object where
    sizeOf _ = 16
    peekAt i arr = do
       -- Check the array bounds
        x0 <- peekAt i arr
        x1 <- peekAt (i + 8) arr
        return $ Object x0 x1
    pokeAt i arr (Object x0 x1) = do
       -- Check the array bounds
        pokeAt i arr x0
        pokeAt (i + 8) arr x1
:}

Given an Unbox instance declaration splice without the methods (e.g. [d|instance Unbox a => Unbox (Maybe a)|]), generate an instance declaration including all the type class method implementations.

Usage:

$(deriveUnbox [d|instance Unbox a => Unbox (Maybe a)|])

The Serialize type class provides operations for serialization and deserialization of general Haskell data types to and from their byte stream representation.

Unlike Unbox, Serialize uses variable length encoding, therefore, it can serialize recursive and variable length data types like lists, or variable length sum types where the length of the value may vary depending on a particular constructor. For variable length data types the length is encoded along with the data.

The deserializeAt operation reads bytes from the mutable byte array and builds a Haskell data type from these bytes, the number of bytes it reads depends on the type and the encoded value it is reading. serializeAt operation converts a Haskell data type to its binary representation which must consist of as many bytes as added by the addSizeTo operation for that value and then stores these bytes into the mutable byte array. The programmer is expected to use the addSizeTo operation and allocate an array of sufficient length before calling serializeAt.

IMPORTANT: The serialized data's byte ordering remains the same as the host machine's byte order. Therefore, it can not be deserialized from host machines with a different byte ordering.

Instances can be derived via Template Haskell, or written manually.

Here is an example, for deriving an instance of this type class using template Haskell:

>>> :{
data Object = Object
    { _obj1 :: [Int]
    , _obj2 :: Int
    }
:}

import Streamly.Data.MutByteArray (deriveSerialize)
$(deriveSerialize [d|instance Serialize Object|])

See deriveSerialize and deriveSerializeWith for more information on deriving using Template Haskell.

Here is an example of a manual instance.

>>> import Streamly.Data.MutByteArray (Serialize(..))

>>> :{
instance Serialize Object where
    addSizeTo acc obj = addSizeTo (addSizeTo acc (_obj1 obj)) (_obj2 obj)
    deserializeAt i arr len = do
         -- Check the array bounds before reading
        (i1, x0) <- deserializeAt i arr len
        (i2, x1) <- deserializeAt i1 arr len
        pure (i2, Object x0 x1)
    serializeAt i arr (Object x0 x1) = do
        i1 <- serializeAt i arr x0
        i2 <- serializeAt i1 arr x1
        pure i2
:}

Configuration to control how the Serialize instance is generated. The configuration is opaque and is modified by composing config modifier functions, for example:

>>> (inlineSerializeAt (Just NoInline)) . (inlineSerializeAt (Just Inlinable))

The default configuration settings are:

• inlineAddSizeTo Nothing
• inlineSerializeAt (Just Inline)
• inlineDeserializeAt (Just Inline)

The following experimental options are also available:

• encodeConstrNames False
• encodeRecordFields False

How should we inline the addSizeTo function? The default is Nothing which means left to the compiler. Forcing inline on addSizeTo function actually worsens some benchmarks and improves none.

How should we inline the serialize function? The default 'Just Inline'. However, aggressive inlining can bloat the code and increase in compilation times when there are big functions and too many nesting levels so you can change it accordingly. A Nothing value leaves the decision to the compiler.

How should we inline the deserialize function? See guidelines in inlineSerializeAt.

Given an Serialize instance declaration splice without the methods (e.g. [d|instance Serialize a => Serialize (Maybe a)|]), generate an instance declaration including all the type class method implementations.

>>> deriveSerialize = deriveSerializeWith id

Usage:

$(deriveSerialize
      [d|instance Serialize a => Serialize (Maybe a)|])

deriveSerializeWith config-modifier instance-dec generates a template Haskell splice consisting of a declaration of a Serialize instance. instance-dec is a template Haskell declaration splice consisting of a standard Haskell instance declaration without the type class methods (e.g. [d|instance Serialize a => Serialize (Maybe a)|]).

The type class methods for the given instance are generated according to the supplied config-modifier parameter. See SerializeConfig for default configuration settings.

Usage:

$(deriveSerializeWith
      ( inlineSerializeAt (Just NoInline)
      . inlineDeserializeAt (Just NoInline)
      )
      [d|instance Serialize a => Serialize (Maybe a)|])