Safe Haskell | None |
---|---|
Language | Haskell2010 |
Acknowledgments
The functionality for the limits and getting the environment and database, in particular the idea of specifying the read-only or read-write mode at the type level, was mostly obtained from the lmdb-simple library.
Synopsis
- data Environment mode
- openEnvironment :: Mode mode => FilePath -> Limits -> IO (Environment mode)
- isReadOnlyEnvironment :: Mode mode => Environment mode -> Bool
- class Mode a
- data ReadWrite
- data ReadOnly
- data Limits = Limits {
- mapSize :: !Int
- maxDatabases :: !Int
- maxReaders :: !Int
- defaultLimits :: Limits
- gibibyte :: Int
- tebibyte :: Int
- data Database mode
- getDatabase :: Mode mode => Environment mode -> Maybe String -> IO (Database mode)
- clearDatabase :: Mode mode => Database mode -> IO ()
- readLMDB :: (MonadIO m, Mode mode) => Database mode -> Maybe ReadOnlyTxn -> ReadOptions -> Unfold m Void (ByteString, ByteString)
- unsafeReadLMDB :: (MonadIO m, Mode mode) => Database mode -> Maybe ReadOnlyTxn -> ReadOptions -> (CStringLen -> IO k) -> (CStringLen -> IO v) -> Unfold m Void (k, v)
- data ReadOnlyTxn
- beginReadOnlyTxn :: Environment mode -> IO ReadOnlyTxn
- abortReadOnlyTxn :: ReadOnlyTxn -> IO ()
- data ReadOptions = ReadOptions {
- readDirection :: !ReadDirection
- readStart :: !(Maybe ByteString)
- defaultReadOptions :: ReadOptions
- data ReadDirection
- writeLMDB :: MonadIO m => Database ReadWrite -> WriteOptions -> Fold m (ByteString, ByteString) ()
- data WriteOptions = WriteOptions {}
- defaultWriteOptions :: WriteOptions
- data OverwriteOptions
- data LMDB_Error = LMDB_Error {}
- data MDB_ErrCode
- = MDB_KEYEXIST
- | MDB_NOTFOUND
- | MDB_PAGE_NOTFOUND
- | MDB_CORRUPTED
- | MDB_PANIC
- | MDB_VERSION_MISMATCH
- | MDB_INVALID
- | MDB_MAP_FULL
- | MDB_DBS_FULL
- | MDB_READERS_FULL
- | MDB_TLS_FULL
- | MDB_TXN_FULL
- | MDB_CURSOR_FULL
- | MDB_PAGE_FULL
- | MDB_MAP_RESIZED
- | MDB_INCOMPATIBLE
- | MDB_BAD_RSLOT
- | MDB_BAD_TXN
- | MDB_BAD_VALSIZE
- | MDB_BAD_DBI
Environment
With LMDB, one first creates a so-called “environment,” which one can think of as a file or folder on disk.
data Environment mode Source #
openEnvironment :: Mode mode => FilePath -> Limits -> IO (Environment mode) Source #
Open an LMDB environment in either ReadWrite
or ReadOnly
mode. The FilePath
argument may
be either a directory or a regular file, but it must already exist. If a regular file, an
additional file with "-lock" appended to the name is used for the reader lock table.
Note that an environment must have been opened in ReadWrite
mode at least once before it can be
opened in ReadOnly
mode.
An environment opened in ReadOnly
mode may still modify the reader lock table (except when the
filesystem is read-only, in which case no locks are used).
isReadOnlyEnvironment :: Mode mode => Environment mode -> Bool Source #
Mode
Instances
Mode ReadOnly Source # | |
Defined in Streamly.External.LMDB.Internal isReadOnlyMode :: ReadOnly -> Bool Source # | |
Mode ReadWrite Source # | |
Defined in Streamly.External.LMDB.Internal isReadOnlyMode :: ReadWrite -> Bool Source # |
Instances
Mode ReadWrite Source # | |
Defined in Streamly.External.LMDB.Internal isReadOnlyMode :: ReadWrite -> Bool Source # |
Instances
Mode ReadOnly Source # | |
Defined in Streamly.External.LMDB.Internal isReadOnlyMode :: ReadOnly -> Bool Source # |
Limits
LMDB environments have various limits on the size and number of databases and concurrent readers.
Limits | |
|
defaultLimits :: Limits Source #
The default limits are 1 MiB map size, 0 named databases, and 126 concurrent readers. These can
be adjusted freely, and in particular the mapSize
may be set very large (limited only by
available address space). However, LMDB is not optimized for a large number of named databases so
maxDatabases
should be kept to a minimum.
The default mapSize
is intentionally small, and should be changed to something appropriate for
your application. It ought to be a multiple of the OS page size, and should be chosen as large as
possible to accommodate future growth of the database(s). Once set for an environment, this limit
cannot be reduced to a value smaller than the space already consumed by the environment, however
it can later be increased.
If you are going to use any named databases then you will need to change maxDatabases
to the
number of named databases you plan to use. However, you do not need to change this field if you
are only going to use the single main (unnamed) database.
Database
After creating an environment, one creates within it one or more databases.
Note: We currently have no functionality here for closing and disposing of databases or environments because we have had no need for it yet. In any case, it is a common practice with LMDB to create one’s environments and databases once and reuse them for the remainder of the program’s execution.
getDatabase :: Mode mode => Environment mode -> Maybe String -> IO (Database mode) Source #
Gets a database with the given name. When creating a database (i.e., getting it for the first
time), one must do so in ReadWrite
mode.
If only one database is desired within the environment, the name can be Nothing
(known as the
“unnamed database”).
If one or more named databases (a database with a Just
name) are desired, the maxDatabases
of
the environment’s limits should have been adjusted accordingly. The unnamed database will in this
case contain the names of the named databases as keys, which one is allowed to read but not
write.
clearDatabase :: Mode mode => Database mode -> IO () Source #
Clears, i.e., removes all key-value pairs from, the given database.
Reading
readLMDB :: (MonadIO m, Mode mode) => Database mode -> Maybe ReadOnlyTxn -> ReadOptions -> Unfold m Void (ByteString, ByteString) Source #
Creates an unfold with which we can stream key-value pairs from the given database.
If an existing read-only transaction is not provided, a read-only transaction is automatically
created and kept open for the duration of the unfold; we suggest doing this as a first option.
However, if you find this to be a bottleneck (e.g., if you find upon profiling that a significant
time is being spent at mdb_txn_begin
, or if you find yourself having to increase maxReaders
in the environment’s limits because the transactions are not being garbage collected fast
enough), consider precreating a transaction using beginReadOnlyTxn
.
In any case, bear in mind at all times LMDB’s caveats regarding long-lived transactions.
If you don’t want the overhead of intermediate ByteString
s (on your way to your eventual data
structures), use unsafeReadLMDB
instead.
unsafeReadLMDB :: (MonadIO m, Mode mode) => Database mode -> Maybe ReadOnlyTxn -> ReadOptions -> (CStringLen -> IO k) -> (CStringLen -> IO v) -> Unfold m Void (k, v) Source #
Similar to readLMDB
, except that the keys and values are not automatically converted into
Haskell ByteString
s.
To ensure safety, make sure that the memory pointed to by the CStringLen
for each key/value
mapping function call is (a) only read (and not written to); and (b) not used after the mapping
function has returned. One way to transform the CStringLen
s to your desired data structures is
to use unsafePackCStringLen
.
Read-only transactions
data ReadOnlyTxn Source #
beginReadOnlyTxn :: Environment mode -> IO ReadOnlyTxn Source #
Begins an LMDB read-only transaction for use with readLMDB
or unsafeReadLMDB
. It is your
responsibility to (a) use the transaction only on databases in the same environment, (b) make
sure that those databases were already obtained before the transaction was begun, and (c) dispose
of the transaction with abortReadOnlyTxn
.
abortReadOnlyTxn :: ReadOnlyTxn -> IO () Source #
Disposes of a read-only transaction created with beginReadOnlyTxn
.
Read options
data ReadOptions Source #
Instances
Show ReadOptions Source # | |
Defined in Streamly.External.LMDB showsPrec :: Int -> ReadOptions -> ShowS # show :: ReadOptions -> String # showList :: [ReadOptions] -> ShowS # |
defaultReadOptions :: ReadOptions Source #
By default, we start reading from the beginning of the database (i.e., from the smallest key).
data ReadDirection Source #
Direction of key iteration.
Instances
Show ReadDirection Source # | |
Defined in Streamly.External.LMDB showsPrec :: Int -> ReadDirection -> ShowS # show :: ReadDirection -> String # showList :: [ReadDirection] -> ShowS # |
Writing
writeLMDB :: MonadIO m => Database ReadWrite -> WriteOptions -> Fold m (ByteString, ByteString) () Source #
Creates a fold with which we can stream key-value pairs into the given database.
It is the responsibility of the user to execute the fold on a bound thread.
The fold currently cannot be used with a scan. (The plan is for this shortcoming to be remedied with or after a future release of streamly that addresses the underlying issue.)
Please specify a suitable transaction size in the write options; the default of 1 (one write transaction for each key-value pair) could yield suboptimal performance. One could try, e.g., 100 KB chunks and benchmark from there.
data WriteOptions Source #
data OverwriteOptions Source #
Instances
Eq OverwriteOptions Source # | |
Defined in Streamly.External.LMDB (==) :: OverwriteOptions -> OverwriteOptions -> Bool # (/=) :: OverwriteOptions -> OverwriteOptions -> Bool # |
Error types
data LMDB_Error Source #
Instances
Show LMDB_Error Source # | |
Defined in Streamly.External.LMDB.Internal.Foreign showsPrec :: Int -> LMDB_Error -> ShowS # show :: LMDB_Error -> String # showList :: [LMDB_Error] -> ShowS # | |
Exception LMDB_Error Source # | |
Defined in Streamly.External.LMDB.Internal.Foreign toException :: LMDB_Error -> SomeException # fromException :: SomeException -> Maybe LMDB_Error # displayException :: LMDB_Error -> String # |
data MDB_ErrCode Source #
Instances
Eq MDB_ErrCode Source # | |
Defined in Streamly.External.LMDB.Internal.Foreign (==) :: MDB_ErrCode -> MDB_ErrCode -> Bool # (/=) :: MDB_ErrCode -> MDB_ErrCode -> Bool # | |
Show MDB_ErrCode Source # | |
Defined in Streamly.External.LMDB.Internal.Foreign showsPrec :: Int -> MDB_ErrCode -> ShowS # show :: MDB_ErrCode -> String # showList :: [MDB_ErrCode] -> ShowS # |