pooled-io: Run jobs on a limited number of threads and support data dependencies

[ bsd3, library, parallelism ] [ Propose Tags ]

The motivation for this package was to run computations on multiple cores that need to write intermediate results to disk. The functions restrict the number of simultaneously running jobs to a user given number or to the number of capabilities the Haskell program was started with, i.e. the number after the RTS option -N.

There are some flavors of this functionality:

Additionally there is the module Control.Concurrent.PooledIO.Sequence that helps to serialize I/O actions from multiple threads. It is certainly most useful in connection with Control.Concurrent.PooledIO.Independent.

Related packages:

  • lazyio: interleave IO actions in a single thread

  • async: start threads and wait for their results, forward exceptions, but do not throttle concurrency with respect to number of available cores

  • parallel-tasks:

  • parallel-io:

  • threadPool:

  • threads-pool:

  • cio:

  • Control-Engine:

  • taskpool, async-pool:

Flags

Automatic Flags
NameDescriptionDefault
buildexamples

Build example executables

Disabled

Use -f <flag> to enable a flag, or -f -<flag> to disable that flag. More info

Downloads

Maintainer's Corner

For package maintainers and hackage trustees

Candidates

Versions [RSS] 0.0, 0.0.0.1, 0.0.1, 0.0.1.1, 0.0.1.2, 0.0.2, 0.0.2.1, 0.0.2.2
Dependencies base (>=4 && <5), concurrent-split (>=0.0 && <0.1), containers (>=0.4 && <0.7), deepseq (>=1.3 && <1.5), pooled-io, transformers (>=0.2.2 && <0.6), unsafe (>=0.0 && <0.1), utility-ht (>=0.0.9 && <0.1) [details]
License BSD-3-Clause
Author Henning Thielemann <haskell@henning-thielemann.de>
Maintainer Henning Thielemann <haskell@henning-thielemann.de>
Category Parallelism
Home page http://hub.darcs.net/thielema/pooled-io/
Source repo this: darcs get http://hub.darcs.net/thielema/pooled-io/ --tag 0.0.2.2
head: darcs get http://hub.darcs.net/thielema/pooled-io/
Uploaded by HenningThielemann at 2018-09-01T10:36:22Z
Distributions LTSHaskell:0.0.2.2, NixOS:0.0.2.2, Stackage:0.0.2.2
Executables pooled-io-demo
Downloads 5774 total (17 in the last 30 days)
Rating (no votes yet) [estimated by Bayesian average]
Your Rating
  • λ
  • λ
  • λ
Status Docs available [build log]
Last success reported on 2018-09-01 [all 1 reports]