Dflow
=====
Real time event processing using data flow declarations.
The following has been tested using ghc 7.0.4 under Fedora 15 64-bit.
Installation
------------
$ cabal configure
$ cabal build
$ cabal install
Tests
-----
$ cabal configure --enable-test
$ cabal build
$ cabal test
The tests work by building random networks of event processors and
checking them against an equivalent list implementation. Occasionally
they seem to hit a pathological case that chews up memory and CPU
time. If memory usage goes over 1GB then kill the test and try again.
Test Coverage
-------------
Edit the Dflow.cabal file and add "-fhpc" to the GHC options for the
test (see comment in the file). Then do "cabal clean" and follow the
instructions for "Tests" above. Then do:
$ hpc markup ArbTest.tix --destdir=coverage
The coverage report is in coverage/hpc_index.html.
Concepts
--------
The main data types for the application programmer are:
Event: A value that occurs at a certain time. For instance an
"Event Char" might represent a key press.
RTSP: The Real Time Stream Processor. A value of type "RTSP x y"
takes in events of type "x" and emits events of type "y". RTSPs can
be strung together into pipelines using "." (or ">>>" if you prefer
your data to flow left-to-right). RTSPs are also Monoids, so you can
fork your data through two parallel RTSPs and then merge the results.
RTA: A monad for building stateful RTSPs. Convert an RTA into an
RTSP using "execRTA" or "accumulateRTA" depending what you want to
do with pending output events when a new input event arrives.
You can test an RTSP in "fast time" (that is, without waiting for
real-time delays) by using "simulateRTSP". However the argument list
of input events must be finite. Then you can execute the RTSP in real
time using "execRTSP" and be confident that the real time behaviour
will match the fast-time behaviour.
To Do
-----
Events are currently sorted into time order using (in effect) an
O(n) insertion sort. Event streams should use something more
sophisticated internally, such as a heap.
What API, if any, should be exposed for event streams? On one hand
developers should be able to create new kinds of RTSP primitives
using event streams, but on the other hand anything you can do with
event streams can be done using RTA anyway.
Figure out some way for RTAs and RTSPs to execute non-blocking IO with
timeouts, probably using continuations in some way. Hence develop true
distribution by having RTSPs on different machines interact in a
declarative manner.
Make state persistent, together with a replay mechanism for lost
events.
