Hackage :: [Package]

porcupine-core: Express portable, composable and reusable data tasks and pipelines

This is a package candidate release! Here you can preview how this package release will appear once published to the main package index (which can be accomplished via the 'maintain' link below). Please note that once a package has been published to the main package index it cannot be undone! Please consult the package uploading documentation for more information.

[maintain] [Publish]

porcupine is centered around the PTask datatype, which represents a computation that will request access to some resources (both in read and write modes) and require some options (parameters, with docstrings and default values). PTasks are composable both sequentially and in parallel, into a pipeline of tasks. The resources and parameters are organized in a tree which will be automatically exposed to the outside world. This makes the pipeline self-documented, and makes it so any option or file required at some point by any task can be visualized and set or remapped (via a combination of a YAML or JSON config file and command-line arguments) before the pipeline will run. That means that the PTasks are completely agnostic of their data inputs, and that new data sources can be added without having to change any of the tasks' logic or even their types. This is done via the LocationAccessor typeclass. `porcupine-core` provides only access to local files (via resourcet), other location accessors will be in separate packages. See for instance the https://hackage.haskell.org/package/porcupine-http package to access HTTP urls. PTasks also provide caching thanks to the funflow package. See the README at https://github.com/tweag/porcupine#README.md and the examples in `porcupine-core` package.

Properties

Versions 0.1.0.0, 0.1.0.1, 0.1.0.1
Change log None available
Dependencies aeson, aeson-pretty, attoparsec, base (>=4.10 && <5), binary, binary-orphans, bytestring, cassava, clock, conduit, containers, contravariant, data-default, deepseq, directory, docrecords (>=0.1 && <0.2), filepath, foldl, formatting, funflow (>=1.5.0), hashable, hvega, katip (>=0.8 && <0.9), lens, log-domain, monad-bayes, monad-control, mtl, optparse-applicative, path, porcupine-core, profunctors, reader-soup (>=0.1 && <0.2), resourcet, safe-exceptions, store, streaming, streaming-bytestring, streaming-conduit, streaming-utils, template-haskell, temporary, text, transformers, transformers-base, unix, unliftio-core, unordered-containers, url, vector, vinyl, yaml, zlib [details]
License MIT
Copyright 2018 EURL Tweag, NovaDiscovery
Author
Maintainer Yves Parès <yves.pares@tweag.io>
Category Data, Arrows, Combinators, Control, Pipes, Streaming, Numerical, JSON
Home page https://github.com/tweag/porcupine#readme
Bug tracker https://github.com/tweag/porcupine/issues
Source repo head: git clone https://github.com/tweag/porcupine
Uploaded by YvesPares at 2019-10-09T13:23:07Z

Modules

Flags

Manual Flags

NameDescriptionDefault
usemonadbayesDisabled

Use -f <flag> to enable a flag, or -f -<flag> to disable that flag. More info

Downloads

Maintainer's Corner

Package maintainers

For package maintainers and hackage trustees