cuckoo: Haskell Implementation of Cuckoo Filters

This is a package candidate release! Here you can preview how this package release will appear once published to the main package index (which can be accomplished via the 'maintain' link below). Please note that once a package has been published to the main package index it cannot be undone! Please consult the package uploading documentation for more information.

[maintain] [Publish]

Haskell implementation of Cuckoo filters as described in

B. Fan, D.G. Anderson, M. Kaminsky, M.D. Mitzenmacher. Cuckoo Filter: Practically Better Than Bloom. In Proc. CoNEXT, 2014.

Cuckoo filters are a data structure for probabilistic set membership. They support insertion, deletion, and membership queries for set elements.

Membership queries may return false positive results. But queries don't return false negative results.

Unlike Bloom filters, Cuckoo filters maintain an upper bound on the false positive rate that is independent of the load of the filter. However, insertion of new elements in the filter can fail. For typical configurations this probability is very small for load factors smaller than 90 percent.


[Skip to Readme]

Properties

Versions 0.1.0.0, 0.2.0.0, 0.2.0.1, 0.2.0.1, 0.2.1, 0.2.2, 0.3.0, 0.3.1
Change log CHANGELOG.md
Dependencies base (>=4.11 && <4.15), cuckoo, memory (>=0.14), primitive (>=0.7), random (>=1.1), vector (>=0.12) [details]
License BSD-3-Clause
Copyright Copyright (c) 2019, Lars Kuhtz <lakuhtz@gmail.com>
Author Lars Kuhtz
Maintainer lakuhtz@gmail.com
Category Data
Home page https://github.com/larskuhtz/cuckoo
Bug tracker https://github.com/larskuhtz/cuckoo/issues
Source repo head: git clone https://github.com/larskuhtz/cuckoo.git
Uploaded by larsk at 2019-08-21T04:35:45Z

Modules

[Index] [Quick Jump]

Flags

Manual Flags

NameDescriptionDefault
mwc-random

Use mwc-random instead of the random package

Disabled
pcg-random

Use pcg-random instead of the random package

Disabled

Use -f <flag> to enable a flag, or -f -<flag> to disable that flag. More info

Downloads

Maintainer's Corner

Package maintainers

For package maintainers and hackage trustees


Readme for cuckoo-0.2.0.1

[back to package description]

Build Status Hackage

Haskell implementation of Cuckoo filters as described in

B. Fan, D.G. Anderson, M. Kaminsky, M.D. Mitzenmacher. Cuckoo Filter: Practically Better Than Bloom. In Proc. CoNEXT, 2014.

Cuckoo filters are a data structure for probabilistic set membership. They support insertion, deletion, and membership queries for set elements.

Membership queries may return false positive results. But queries don't return false negative results.

Unlike Bloom filters, Cuckoo filters maintain an upper bound on the false positive rate that is independent of the load of the filter. However, insertion of new elements in the filter can fail. For typical configurations this probability is very small for load factors smaller than 90 percent.

The implementation allows the user to specify the bucket size and the fingerprint size in addition to the capacity of the filter. The user can also provide custom functions for computing the primary hash and fingerprint.

Installation

cabal v2-install cuckoo

For running the test-suites

cabal v2-test cuckoo

For running the benchmarks

cabal v2-bench cuckoo

Example

{-# LANGUAGE DataKinds #-}
{-# LANGUAGE TypeApplications #-}
{-# LANGUAGE TypeFamilies #-}
{-# OPTIONS_GHC -fno-warn-orphans #-}

import Control.Monad (filterM)
import Data.Cuckoo
import Data.List ((\\))

-- Define CuckooFilterHash instance (this uses the default implementation)
instance CuckooFilterHash Int

main :: IO ()
main = do
    -- Create Filter for a minimum of 500000 entries
    f <- newCuckooFilter @4 @8 @Int 0 500000

    -- Insert 450000 items
    failed <- filterM (fmap not . insert f) [0..500000-1]

    -- Query inserted items
    missing <- filterM (fmap not . member f) [0..500000-1]

    -- Test for false positives
    false <- filterM (member f) [500000..1000000 - 1]

    -- Report results
    putStrLn $ "failed inserts: " <> show (length failed)
    putStrLn $ "false positives: " <> show (length false)
    putStrLn $ "false positive rate (%): " <> show @Double (fromIntegral (length false) * 100 / 500000)
    putStrLn $ "missing (must be 0): " <> show (length $ missing \\ failed)

    -- Filter properties
    putStrLn $ "capacity: " <> show (capacityInItems f)
    putStrLn $ "size in allocated bytes: " <> show (sizeInAllocatedBytes f)

    -- computing the following is a bit slow
    c <- itemCount f
    putStrLn $ "item count: " <> show c
    lf <- loadFactor f
    putStrLn $ "load factor (%): " <> show lf
    putStrLn $ "bits per item: " <> show @Double (fromIntegral (sizeInAllocatedBytes f) * 8 / fromIntegral c)

Which produces the following results:

$ ghc -o main -threaded -O -with-rtsopts=-N Main.hs
[1 of 1] Compiling Main             ( Main.hs, Main.o )
Linking main ...
$ ./main
failed inserts: 0
false positives: 14796
false postive rate (%): 2.9592
missing (must be 0): 0
capacity: 524288
size in allocated bytes: 524292
item count: 500000
load factor (%): 95.367431640625
bits per item: 8.388672

Another example can be found in the file bench/SpellChecker.hs.