cloben: Clone and benchmark Haskell cabal projects

[ benchmarking, bsd3, program ] [ Propose Tags ]
This version is deprecated.

cloben is a Haskell shell script transforms cabal bench/stack bench results into a CSV file readable by gipeda for visualization. Prior to that, it optionally clones a specific commit of a given git repository into a temporary folder in which it then performs the benchmarking.

Example run of some default fib benchmarks:

$ cloben https://github.com/sgraf812/benchmark-test 9982583
build/warnings;1.0
benchmarks/fib/10;487.7
benchmarks/fib/35;81910000.0
benchmarks/fib/37;215200000.0

[Skip to Readme]

Downloads

Maintainer's Corner

Package maintainers

For package maintainers and hackage trustees

Candidates

Versions [RSS] 0.1.0.0, 0.1.0.1, 0.1.0.2, 0.1.0.3, 0.1.1.0 (info)
Dependencies base (>=4.7 && <5), foldl (<2.0), process (>=1.1.0.0 && <1.5), system-filepath (>=0.3.1 && <0.5), temporary (>=1.1 && <1.3), text (<1.3), turtle (>=1.2 && <1.3) [details]
License BSD-3-Clause
Copyright 2015 Sebastian Graf
Author Sebastian Graf
Maintainer sgraf1337@gmail.com
Category Benchmarking
Home page http://github.com/sgraf812/cloben#readme
Source repo head: git clone https://github.com/sgraf812/cloben.git
Uploaded by sgraf812 at 2016-07-01T17:50:57Z
Distributions
Reverse Dependencies 1 direct, 0 indirect [details]
Executables cloben
Downloads 3418 total (18 in the last 30 days)
Rating (no votes yet) [estimated by Bayesian average]
Your Rating
  • λ
  • λ
  • λ
Status Docs not available [build log]
All reported builds failed as of 2016-11-22 [all 3 reports]

Readme for cloben-0.1.0.2

[back to package description]

cloben Build Status Hackage

cloben is a Haskell shell script transforms cabal bench/stack bench results into a CSV file readable by gipeda for visualization. Prior to that, it optionally clones a specific commit of a given git repository into a temporary folder in which it then performs the benchmarking.

It parses build warnings and timing data output in the standard criterion format.

Usage

There are two modes of operation:

cloben will benchmark the cabal project in the current working directory and output the CSV data on stdout. It will try to use stack as an optimization, but the fall back mechanism (cabal sandboxing) takes 10 minutes on my laptop for even the simplest dependency tree, so stay calm :).

cloben <repo> <commit> will also attempt to recursively clone a remote git repo at a specific commit into a temporary directory and cd into it prior to benchmarking.

For usage with gipeda, stdout should be redirected into a csv file.

See also cloben --help.

How to build

Simplest way? Don't! Use stacks excellent support for runghc:

$ stack cloben.hs

Or on unixoid systems:

$ chmod +x cloben.hs
$ ./cloben.hs

Of course, cloben can be built both in a cabal and in a stack environment.

$ stack build && stack exec cloben
...
$ cabal install -j && cabal run
...

Or even just through hackage:

$ stack install cloben && cloben
...
$ cabal install cloben && cloben
...

Example

What running cloben on the Pipes library yielded:

$ ./cloben.hs https://github.com/Gabriel439/Haskell-Pipes-Library 930c834aacfa7bf8ec65d072e0d0a982aa7a2bc1 > logs/930c834aacfa7bf8ec65d072e0d0a982aa7a2bc1.csv
$ cat logs/930c834aacfa7bf8ec65d072e0d0a982aa7a2bc1.csv
build/warnings;81.0
benchmark/prelude-benchmarks/Folds/all;152900.0
benchmark/prelude-benchmarks/Folds/any;163700.0
benchmark/prelude-benchmarks/Folds/find;170800.0
benchmark/prelude-benchmarks/Folds/findIndex;164500.0
benchmark/prelude-benchmarks/Folds/fold;68150.0
benchmark/prelude-benchmarks/Folds/foldM;67690.0
benchmark/prelude-benchmarks/Folds/head;10.84
benchmark/prelude-benchmarks/Folds/index;117000.0
benchmark/prelude-benchmarks/Folds/last;119600.0
benchmark/prelude-benchmarks/Folds/length;53960.0
benchmark/prelude-benchmarks/Folds/null;11.2
benchmark/prelude-benchmarks/Folds/toList;138100.0
benchmark/prelude-benchmarks/Pipes/chain;1351000.0
benchmark/prelude-benchmarks/Pipes/drop;110400.0
benchmark/prelude-benchmarks/Pipes/dropWhile;159000.0
benchmark/prelude-benchmarks/Pipes/filter;585100.0
benchmark/prelude-benchmarks/Pipes/findIndices;397900.0
benchmark/prelude-benchmarks/Pipes/map;324600.0
benchmark/prelude-benchmarks/Pipes/mapM;1276000.0
benchmark/prelude-benchmarks/Pipes/take;346500.0
benchmark/prelude-benchmarks/Pipes/takeWhile;332400.0
benchmark/prelude-benchmarks/Pipes/scan;370900.0
benchmark/prelude-benchmarks/Pipes/scanM;1177000.0
benchmark/prelude-benchmarks/Pipes/concat;159600.0
benchmark/prelude-benchmarks/Zips/zip;1218000.0
benchmark/prelude-benchmarks/Zips/zipWith;1318000.0
benchmark/prelude-benchmarks/enumFromTo.vs.each/enumFromTo;205700.0
benchmark/prelude-benchmarks/enumFromTo.vs.each/each;209000.0
benchmark/lift-benchmarks/ReaderT/runReaderP_B;4791000.0
benchmark/lift-benchmarks/ReaderT/runReaderP_A;266600.0
benchmark/lift-benchmarks/StateT/runStateP_B;4912000.0
benchmark/lift-benchmarks/StateT/runStateP_A;344800.0
benchmark/lift-benchmarks/StateT/evalStateP_B;5534000.0
benchmark/lift-benchmarks/StateT/evalStateP_A;349400.0
benchmark/lift-benchmarks/StateT/execStateP_B;5451000.0
benchmark/lift-benchmarks/StateT/execStateP_A;324300.0

Which cabal, stack and ghc binaries are used?

Short answer: That found on the path at the time of executing this script.

If cloben is compiled to an executable which is then called from your shell, it will use cabal, stack and ghc binaries found on your path.

If however you prefer to run this as a script through stack (see above), note that stack modifies the environment, so it will use a 'stack-local' version of cabal when stack install cabal-install happened.

Finally, note that the fallback mechanism (try stack bench, fall back to cabal sandbox init && cabal bench) might fail if it can't find ghc on the path.