streamly-process: Use OS processes as stream transformation functions

[ apache, library, streaming, streamly, system ] [ Propose Tags ]

Use operating system (OS) commands in Haskell programs as if they were native Haskell functions, by treating their inputs and outputs as Haskell streams. This allows you to write high-level Haskell scripts that can perform tasks similar to shell scripts, but with C-like performance, and with strong safety guarantees, refactorability, and modularity.


[Skip to Readme]

Flags

Manual Flags

NameDescriptionDefault
fusion-plugin

Use fusion plugin for benchmarks and executables

Disabled
use-native

Do not depend on the process package

Disabled

Use -f <flag> to enable a flag, or -f -<flag> to disable that flag. More info

Downloads

Note: This package has metadata revisions in the cabal description newer than included in the tarball. To unpack the package including the revisions, use 'cabal get'.

Maintainer's Corner

Package maintainers

For package maintainers and hackage trustees

Candidates

  • No Candidates
Versions [RSS] 0.1.0, 0.2.0, 0.2.0.1, 0.3.0, 0.3.1
Change log CHANGELOG.md
Dependencies base (>=4.8 && <5), exceptions (>=0.8 && <0.11), process (>=1.0 && <1.7), streamly (>=0.9 && <0.11), streamly-core (>=0.1 && <0.3) [details]
License Apache-2.0
Copyright Composewell Technologies
Author Composewell Technologies
Maintainer streamly@composewell.com
Revised Revision 1 made by harendra at 2024-01-20T19:48:25Z
Category Streamly, Streaming, System
Home page https://streamly.composewell.com
Bug tracker https://github.com/composewell/streamly-process/issues
Source repo head: git clone https://github.com/composewell/streamly-process
Uploaded by harendra at 2023-12-21T18:21:02Z
Distributions NixOS:0.3.1, Stackage:0.3.1
Reverse Dependencies 1 direct, 1 indirect [details]
Downloads 480 total (23 in the last 30 days)
Rating 2.0 (votes: 1) [estimated by Bayesian average]
Your Rating
  • λ
  • λ
  • λ
Status Docs available [build log]
Last success reported on 2023-12-21 [all 1 reports]

Readme for streamly-process-0.3.1

[back to package description]

OS processes as streams

Use operating system (OS) commands in Haskell programs as if they were native Haskell functions, by treating their inputs and outputs as Haskell streams. This allows you to write high-level Haskell scripts that can perform tasks similar to shell scripts, but with C-like performance, and with strong safety guarantees, refactorability, and modularity.

Use the following imports in the examples below:

>>> :set -XFlexibleContexts
>>> :set -XScopedTypeVariables
>>> :set -XQuasiQuotes
>>> import Data.Function ((&))
>>> import Streamly.Unicode.String (str)
>>> import qualified Streamly.Data.Array as Array
>>> import qualified Streamly.Console.Stdio as Stdio
>>> import qualified Streamly.Data.Fold as Fold
>>> import qualified Streamly.Data.Stream.Prelude as Stream
>>> import qualified Streamly.System.Command as Command
>>> import qualified Streamly.Internal.FileSystem.Dir as Dir

Commands as streaming functions

The shell command echo "hello world" | tr [a-z] [A-Z] can be written as follows using this package:

>>> :{
   Command.toBytes [str|echo "hello world"|] -- Stream IO Word8
 & Command.pipeBytes [str|tr [a-z] [A-Z]|]   -- Stream IO Word8
 & Stream.fold Stdio.write                   -- IO ()
:}
HELLO WORLD

Shell commands as streaming functions

If you want to execute the same command using the shell pipe syntax:

>>> :{
   Command.toBytes [str|sh "-c" "echo 'hello world' | tr [a-z] [A-Z]"|] -- Stream IO Word8
 & Stream.fold Stdio.write                                              -- IO ()
:}
HELLO WORLD

Running Commands Concurrently

You can run commands concurrently by using streamly's concurrent combinators.

Running @grep@ concurrently on many files:

>>> :{
grep file =
   Command.toBytes [str|grep -H "pattern" #{file}|]             -- Stream IO Word8
 & Stream.handle (\(_ :: Command.ProcessFailure) -> Stream.nil) -- Stream IO Word8
 & Stream.foldMany (Fold.takeEndBy (== 10) Array.write)         -- Stream IO (Array Word8)
:}

>>> :{
pgrep =
   Dir.readFiles "."             -- Stream IO FilePath
 & Stream.parConcatMap id grep   -- Stream IO (Array Word8)
 & Stream.fold Stdio.writeChunks -- IO ()
:}

Streamly

Please visit Streamly homepage for more details about streamly.

Licensing

Available under Apache-2.0 license.