conduit-throttle: Throttle Conduit Producers
This is a package candidate release! Here you can preview how this package release will appear once published to the main package index (which can be accomplished via the 'maintain' link below). Please note that once a package has been published to the main package index it cannot be undone! Please consult the package uploading documentation for more information.
This packages is based on the throttle-io-stream package and provides functionality for throttling Conduit producers according to a provided configuration.
[Skip to Readme]
Properties
| Versions | 0.2.0.0, 0.2.0.1, 0.3.0.0, 0.3.0.0, 0.3.1.0 |
|---|---|
| Change log | None available |
| Dependencies | async, base (>=4.7 && <5), conduit, conduit-combinators, conduit-extra, monad-control, resourcet, stm, stm-chans, throttle-io-stream, unliftio, unliftio-core [details] |
| License | BSD-3-Clause |
| Copyright | (c) 2017 Moritz Schulte |
| Author | Moritz Schulte |
| Maintainer | mtesseract@silverratio.net |
| Category | Data |
| Home page | https://github.com/mtesseract/conduit-throttle#readme |
| Bug tracker | https://github.com/mtesseract/conduit-throttle/issues |
| Source repo | head: git clone https://github.com/mtesseract/conduit-throttle |
| Uploaded | by mtesseract at 2017-11-01T12:53:59Z |
Modules
[Index]
Downloads
- conduit-throttle-0.3.0.0.tar.gz [browse] (Cabal source package)
- Package description (as included in the package)
Maintainer's Corner
Package maintainers
For package maintainers and hackage trustees
Readme for conduit-throttle-0.3.0.0
[back to package description]conduit-throttle 
About
This is conduit-throttle, a compact Haskell package providing
configurable throttling support for Conduit. It is built on top of the
packages throttle-io-stream and unliftio.
The API (Data.Conduit.Throttle) is designed for being imported
qualified, which is why the exported functions and values are not
prefixed by the package name or something similar.
The core function exported by this package is the following:
throttleProducer :: (MonadUnliftIO m, MonadResource m)
=> Conf a
-> Producer m a
-> Producer m a
That is, given a configuration and a Conduit Producer, create and
return a new Conduit Producer, which yields the same stream of
values like the provided one but throttled according to the provided
throttling configuration.
A throttling configuration is created using newConf and then
configured using the functions setMeasure, setMaxThroughput,
setBufferSize and others.
Primarily, the throughput is defined by the provided measure function
and the configured maximum throughput. A measure function is a
function which defines how "big" a Conduit element a is. In other
words, it is a function of type a -> Double. For instance, it could
simply calculate the length of some deserialized value in bytes or it
could simply be the constant function const 1 in case it is desired
to throttle the number of elements a Conduit producer produces. The
maximum throughput together with the measure function then defines the
maximum throughput to produce (of course it can be less, if the
original producer provides data with a smaller throughput). The
throughput is considered with respect to a base time interval, which
can be changed using setInterval, by default it is 1000 (ms).
The underlying package throttle-io-stream uses exponentially
weighted moving averages for smoothing the actual throughput changes.
Example
import qualified Data.Conduit.Throttle as ConduitThrottle
simpleProducerThrottling :: IO ()
simpleProducerThrottling = do
let conf = ConduitThrottle.newConf
& ConduitThrottle.setInterval 1000
& ConduitThrottle.setMaxThroughput 1
runResourceT . runConduit $
ConduitThrottle.throttleProducer conf (sourceList [1..5]) .| printC