Fun with finite distributions!

This all pretty much comes from Eric Kidd's series of blog posts at http://www.randomhacks.net/probability-monads/.

I have adapted it a bit by making it fit into my own random generation/sampling scheme.

The idea and purpose of this module should be clear after going through an example. First, let's import the library and vector.

import Math.Probable
import qualified Data.Vector as V

We are going to talk about Books, and particularly about whether a given book is interesting or not.

data Book = Interesting 
          | Boring
    deriving (Eq, Show)

Let's say we have very particular tastes, and that we think that only 20% of all books are interesting (that's not so small actually. oh well).

bookPrior :: Finite d => d Book
bookPrior = weighted [ (Interesting, 0.2) 
                     , (Boring, 0.8) 
                     ]

weighted belongs to the Finite class, which represents types that can somehow represent a distribution over a finite set. That makes our distribution polymorphic in how we will use it. Awesome!

So how does it look?

λ> exact bookPrior -- in ghci
[Event Interesting 20.0%,Event Boring 80.0%]

exact takes Fin a and gives you the inner list that Fin uses to represent the distribution.

Now, what if we pick two books? First, how do we even do that? Well, any instance of Finite must be a Monad, so you have your good old do notation. The ones provided by this package also provide Functor and Applicative instances, but let's use do.

twoBooks :: Finite d => d (Book, Book)
twoBooks = do
    book1 <- bookPrior
    book2 <- bookPrior
    return (book1, book2)

Nothing impressive. We pick a book with the prior we defined above, then another, pair them together and hand the pair back. What this will actually do is behave just like in the list monad, but in addition to this it will combine the probabilities of the various events we could be dealing with in the appropriate way.

So, how about we verify what I just said:

λ> exact twoBooks
[ Event (Interesting,Interesting) 4.0%
, Event (Interesting,Boring) 16.0%
, Event (Boring,Interesting) 16.0%
, Event (Boring,Boring) 64.0%
]

Nice! Let's take a look at a more complicated scenario now.

What if we wanted to take a look at the same distribution, with just a difference: we want at least one of the books to be an Interesting one.

oneInteresting :: Fin (Book, Book)
oneInteresting = bayes $ do -- notice the call to bayes
    (b1, b2) <- twoBooks
    condition (b1 == Interesting || b2 == Interesting)
    return (b1, b2)

We get two books from the previous distribution, and use condition to restrict the current distribution to the values of b1 and b2 that verify our condition. This lifts us in the FinBayes type, where our probabilistic computations can "fail" in some sense. If you want to discard values and restrict the ones on which you'll run further computations, use condition.

However, how do we view the distribution now, without having all those Maybes in the middle? That's what bayes is for. It runs the computations for the distribution and discards all the ones where any condition wasn't satisfied. In particular, it means it hands you back a normal Fin distribution.

If we run this one:

λ> exact oneInteresting
[ Event (Interesting,Interesting) 11.1%
, Event (Interesting,Boring) 44.4%
, Event (Boring,Interesting) 44.4%
]

Note that these finite distribution types support random sampling too:

• If one of your distributions has a type like "Finite d => d X", you can actually consider it as a RandT value, from which you can sample.
• If you have a Fin distribution, you can use liftF (lift Fin) to randomly sample an element from it, by more or less following the distribution's probabilities.

-- example of the former
sampleBooks :: RandT IO (V.Vector Book)
sampleBooks = vectorOf 10 bookPrior

λ> mwc sampleBooks
fromList [Interesting,Boring,Boring,Boring,Boring
         ,Boring,Boring,Interesting,Boring,Boring]

λ> mwc $ listOf 4 (liftF oneInteresting) -- example of the latter
[ (Boring,Interesting)
, (Boring,Interesting)
, (Boring,Interesting)
, (Interesting,Boring)
]