cctools-workqueue: High-level interface to CCTools' WorkQueue library

[ distributed-computing, gpl, library ] [ Propose Tags ]

This package provides a high-level Haskell interface to the WorkQueue library:

Work Queue is a framework for building large master-worker applications that span many computers including clusters, clouds, and grids. Work Queue applications are written in C, Perl, or Python using a simple API that allows users to define tasks, submit them to the queue, and wait for completion. Tasks are executed by a standard worker process that can run on any available machine. Each worker calls home to the master process, arranges for data transfer, and executes the tasks. Together, the master and worker handle a wide variety of failures, allowing for dynamically scalable and robust applications.

Example usage:

By default the master listens on 9123 on the localhost host. Start a worker:

$ work_queue_worker -d all localhost 9123

Now we can execute the following master program:

module Main where

import Control.Distributed.CCTools.WorkQueue

import Control.Applicative ((<$>))
import qualified Data.ByteString.Char8 as BS (pack, putStrLn)
import Foreign.C.String (newCStringLen)
import Control.Monad (forM_)

mktask :: Show a => a -> IO Task
mktask v = do
  let script = BS.pack . unlines $ [
              "t=$(echo $RANDOM % 10 | bc)"
            , "sleep $t"
            , "echo " ++ show v
  t <- task $ cmd "bash"
  specifyBuffer t script (remote "") False
  specifyTag    t $ show v
  return t

printStats :: WorkQueue -> IO ()
printStats q = do
  s <- getStats q
  print $ map ($ s) [tasksRunning, tasksWaiting, tasksComplete]

processResult :: WorkQueue -> Task -> IO ()
processResult q r = do
  putStrLn $ "Got: " ++ show (tag r)
  BS.putStrLn . output $ r
  delete r

main = do
  setDebugFlags [All]
  q  <- workqueue defaultQParams
  ts <- mapM mktask [1..10]
  forM_ ts (submit q)

  eventLoop q (seconds 1) printStats processResult

  putStrLn "Done!"


  • Control
    • Distributed
      • CCTools
        • Control.Distributed.CCTools.WorkQueue
          • Control.Distributed.CCTools.WorkQueue.Debug
          • Control.Distributed.CCTools.WorkQueue.Task
          • Control.Distributed.CCTools.WorkQueue.Types
          • Control.Distributed.CCTools.WorkQueue.WorkQueue


Automatic Flags

Turn on warnings


Use -f <flag> to enable a flag, or -f -<flag> to disable that flag. More info


Maintainer's Corner

Package maintainers

For package maintainers and hackage trustees


  • No Candidates
Versions [RSS],
Dependencies base (>=4.5 && <4.7), bindings-cctools (>= && <, bytestring, lens, monad-loops, unix [details]
License GPL-2.0-only
Author Badi' Abdul-Wahid
Category Distributed Computing
Home page
Source repo head: git clone
Uploaded by BadiAbdulWahid at 2013-01-02T00:14:51Z
Reverse Dependencies 1 direct, 0 indirect [details]
Downloads 2012 total (7 in the last 30 days)
Rating (no votes yet) [estimated by Bayesian average]
Your Rating
  • λ
  • λ
  • λ
Status Docs not available [build log]
All reported builds failed as of 2016-11-05 [all 11 reports]