pooled-io: Run jobs on a limited number of threads and support data dependencies

[ bsd3, library, parallelism ] [ Propose Tags ]

The motivation for this package was to run computations on multiple cores that need to write intermediate results to disk. The functions restrict the number of simultaneously running jobs to a user given number or to the number of capabilities the Haskell program was started with, i.e. the number after the RTS option -N.

There are some flavors of this functionality:

Additionally there is the module Control.Concurrent.PooledIO.Sequence that helps to serialize I/O actions from multiple threads. It is certainly most useful in connection with Control.Concurrent.PooledIO.Independent.

Related packages:

  • lazyio: interleave IO actions in a single thread

  • async: start threads and wait for their results, forward exceptions, but do not throttle concurrency with respect to number of available cores

  • parallel-tasks:

  • parallel-io:

  • threadPool:

  • threads-pool:

  • cio:

  • Control-Engine:

  • taskpool, async-pool:

Flags

Automatic Flags
NameDescriptionDefault
buildexamples

Build example executables

Disabled

Use -f <flag> to enable a flag, or -f -<flag> to disable that flag. More info

Downloads

Note: This package has metadata revisions in the cabal description newer than included in the tarball. To unpack the package including the revisions, use 'cabal get'.

Maintainer's Corner

Package maintainers

For package maintainers and hackage trustees

Candidates

Versions [RSS] 0.0, 0.0.0.1, 0.0.1, 0.0.1.1, 0.0.1.2, 0.0.2, 0.0.2.1, 0.0.2.2, 0.0.2.3
Dependencies base (>=4 && <5), concurrent-split (>=0.0 && <0.1), containers (>=0.4 && <0.8), deepseq (>=1.3 && <1.6), pooled-io, transformers (>=0.2.2 && <0.7), unsafe (>=0.0 && <0.1), utility-ht (>=0.0.9 && <0.1) [details]
License BSD-3-Clause
Author Henning Thielemann <haskell@henning-thielemann.de>
Maintainer Henning Thielemann <haskell@henning-thielemann.de>
Revised Revision 1 made by HenningThielemann at 2023-10-07T15:30:05Z
Category Parallelism
Home page https://hub.darcs.net/thielema/pooled-io/
Source repo this: darcs get https://hub.darcs.net/thielema/pooled-io/ --tag 0.0.2.3
head: darcs get https://hub.darcs.net/thielema/pooled-io/
Uploaded by HenningThielemann at 2023-02-15T09:48:00Z
Distributions LTSHaskell:0.0.2.3, NixOS:0.0.2.3, Stackage:0.0.2.3
Reverse Dependencies 3 direct, 0 indirect [details]
Executables pooled-io-demo
Downloads 6283 total (42 in the last 30 days)
Rating (no votes yet) [estimated by Bayesian average]
Your Rating
  • λ
  • λ
  • λ
Status Docs available [build log]
Last success reported on 2023-02-15 [all 1 reports]