Safe Haskell | None |
---|---|
Language | Haskell2010 |
Efficient in-memory (in-core) storage of tabular data.
- type family VectorFor t :: * -> *
- type VectorMFor a = Mutable (VectorFor a)
- initialCapacity :: Int
- type family VectorMs m rs
- type family Vectors rs
- class RecVec rs where
- allocRec :: PrimMonad m => proxy rs -> m (Record (VectorMs m rs))
- freezeRec :: PrimMonad m => proxy rs -> Int -> Record (VectorMs m rs) -> m (Record (Vectors rs))
- growRec :: PrimMonad m => proxy rs -> Record (VectorMs m rs) -> m (Record (VectorMs m rs))
- writeRec :: PrimMonad m => proxy rs -> Int -> Record (VectorMs m rs) -> Record rs -> m ()
- indexRec :: proxy rs -> Int -> Record (Vectors rs) -> Record rs
- produceRec :: proxy rs -> Record (Vectors rs) -> Rec ((->) Int) rs
- inCoreSoA :: forall m rs. (PrimMonad m, RecVec rs) => Producer (Record rs) m () -> m (Int, Rec ((->) Int) rs)
- inCoreAoS :: (PrimMonad m, RecVec rs) => Producer (Record rs) m () -> m (FrameRec rs)
- inCoreAoS' :: (PrimMonad m, RecVec rs) => (Rec ((->) Int) rs -> Rec ((->) Int) ss) -> Producer (Record rs) m () -> m (FrameRec ss)
- toAoS :: Int -> Rec ((->) Int) rs -> FrameRec rs
- inCore :: forall m n rs. (PrimMonad m, RecVec rs, Monad n) => Producer (Record rs) m () -> m (Producer (Record rs) n ())
- toFrame :: (Foldable f, RecVec rs) => f (Record rs) -> Frame (Record rs)
- filterFrame :: RecVec rs => (Record rs -> Bool) -> FrameRec rs -> FrameRec rs
Documentation
type VectorMFor a = Mutable (VectorFor a) Source
The mutable version of VectorFor
a particular type.
Since we stream into the in-memory representation, we use an exponential growth strategy to resize arrays as more data is read in. This is the initial capacity of each column.
Immutable vector types for each column in a row.
Tooling to allocate, grow, write to, freeze, and index into records of vectors.
allocRec :: PrimMonad m => proxy rs -> m (Record (VectorMs m rs)) Source
freezeRec :: PrimMonad m => proxy rs -> Int -> Record (VectorMs m rs) -> m (Record (Vectors rs)) Source
growRec :: PrimMonad m => proxy rs -> Record (VectorMs m rs) -> m (Record (VectorMs m rs)) Source
writeRec :: PrimMonad m => proxy rs -> Int -> Record (VectorMs m rs) -> Record rs -> m () Source
indexRec :: proxy rs -> Int -> Record (Vectors rs) -> Record rs Source
produceRec :: proxy rs -> Record (Vectors rs) -> Rec ((->) Int) rs Source
inCoreSoA :: forall m rs. (PrimMonad m, RecVec rs) => Producer (Record rs) m () -> m (Int, Rec ((->) Int) rs) Source
Stream a finite sequence of rows into an efficient in-memory
representation for further manipulation. Each column of the input
table will be stored optimally based on its type, making use of the
resulting generators a matter of indexing into a densely packed
representation. Returns the number of rows and a record of column
indexing functions. See toAoS
to convert the result to a Frame
which provides an easier-to-use function that indexes into the
table in a row-major fashion.
inCoreAoS :: (PrimMonad m, RecVec rs) => Producer (Record rs) m () -> m (FrameRec rs) Source
Stream a finite sequence of rows into an efficient in-memory
representation for further manipulation. Each column of the input
table will be stored optimally based on its type, making use of the
resulting generators a matter of indexing into a densely packed
representation. Returns a Frame
that provides a function to index
into the table.
inCoreAoS' :: (PrimMonad m, RecVec rs) => (Rec ((->) Int) rs -> Rec ((->) Int) ss) -> Producer (Record rs) m () -> m (FrameRec ss) Source
toAoS :: Int -> Rec ((->) Int) rs -> FrameRec rs Source
Convert a structure-of-arrays to an array-of-structures. This can simplify usage of an in-memory representation.
inCore :: forall m n rs. (PrimMonad m, RecVec rs, Monad n) => Producer (Record rs) m () -> m (Producer (Record rs) n ()) Source
Stream a finite sequence of rows into an efficient in-memory representation for further manipulation. Each column of the input table will be stored optimally based on its type, making use of the resulting generator a matter of indexing into a densely packed representation.