extrapolate: generalize counter-examples of test properties

[ bsd3, library, testing ] [ Propose Tags ] [ Report a vulnerability ]

Extrapolate is a tool able to provide generalized counter-examples of test properties where irrelevant sub-expressions are replaces with variables.

For the incorrect property \xs -> nub xs == (xs::[Int]):

  • [0,0] is a counter-example;

  • x:x:_ is a generalized counter-example.


[Skip to Readme]

Downloads

Maintainer's Corner

Package maintainers

For package maintainers and hackage trustees

Candidates

  • No Candidates
Versions [RSS] 0.0.1, 0.1.0, 0.2.0, 0.2.1, 0.2.2, 0.2.3, 0.2.4, 0.3.0, 0.3.1, 0.3.2, 0.3.3, 0.4.0, 0.4.1, 0.4.2, 0.4.4, 0.4.6
Dependencies base (>=4 && <5), express (>=0.1.6), leancheck (>=0.9.4), speculate (>=0.4.6), template-haskell [details]
Tested with ghc ==9.0, ghc ==8.10, ghc ==8.8, ghc ==8.6, ghc ==8.4, ghc ==8.2, ghc ==8.0, ghc ==7.10, ghc ==7.8
License BSD-3-Clause
Author Rudy Matela
Maintainer Rudy Matela <rudy@matela.com.br>
Category Testing
Home page https://github.com/rudymatela/extrapolate#readme
Source repo head: git clone https://github.com/rudymatela/extrapolate
this: git clone https://github.com/rudymatela/extrapolate(tag v0.4.6)
Uploaded by rudymatela at 2021-06-15T22:04:50Z
Distributions LTSHaskell:0.4.6, NixOS:0.4.6, Stackage:0.4.6
Reverse Dependencies 1 direct, 0 indirect [details]
Downloads 9199 total (44 in the last 30 days)
Rating 2.0 (votes: 1) [estimated by Bayesian average]
Your Rating
  • λ
  • λ
  • λ
Status Docs available [build log]
Last success reported on 2021-06-15 [all 1 reports]

Readme for extrapolate-0.4.6

[back to package description]

Extrapolate

Extrapolate Build Status Extrapolate on Hackage Extrapolate on Stackage LTS Extrapolate on Stackage Nightly

Extrapolate logo

Extrapolate is a property-based testing library for Haskell capable of reporting generalized counter-examples.

Installing Extrapolate

To install the latest version of Extrapolate from Hackage using cabal, just:

$ cabal update
$ cabal install extrapolate

To test if it installed correctly, follow through the next section.

Starting from Cabal v3, you need to pass --lib to cabal install:

$ cabal install extrapolate --lib

Using Extrapolate

To use Extrapolate, you first import the Test.Extrapolate module, then pass any properties you with to test to the function check:

$ ghci
> import Test.Extrapolate
> check $ \x y -> x + y == y + (x :: Int)
+++ OK, passed 360 tests.

> import Data.List (nub)
> check $ \xs -> nub xs == (xs :: [Int])
*** Failed! Falsifiable (after 3 tests):
[0,0]

Generalization:
x:x:_

The operator + is commutative. The function nub is not an identity.

Configuring the number of tests

To increase the number of tests, use the for combinator:

$ ghci
> import Test.Extrapolate
> check `for` 1000 $ \x y -> x + y == y + (x :: Int)
+++ OK, passed 1000 tests.

Customizing the background functions (allowed in side-conditions)

To customize the background functions, use the withBackground combinator:

$ ghci
> import Test.Extrapolate
> import Data.List (nub)
> let hasDups xs  =  nub xs /= (xs :: [Int])
> check `withBackground` [constant "hasDups" hasDups] $ \xs -> nub xs == (xs :: [Int])
*** Failed! Falsifiable (after 3 tests):
[0,0]

Generalization:
x:x:_

Conditional Generalization:
xs  when  hasDups xs

Perhaps the example above is silly (hasDups is the negation of the property itself!), but it illustrates the use of withBackground.

The combinators for and withBackground can be used in conjunction:

> check `for` 100 `withBackground` [...] $ property

Don't forget the dollar sign $.

Another Example

Consider the following (faulty) sort function and a property about it:

sort :: Ord a => [a] -> [a]
sort []      =  []
sort (x:xs)  =  sort (filter (< x) xs)
             ++ [x]
             ++ sort (filter (> x) xs)

prop_sortCount :: Ord a => a -> [a] -> Bool
prop_sortCount x xs  =  count x (sort xs) == count x xs
  where
  count x = length . filter (== x)

After testing the property, Extrapolate returns a fully defined counter-example along with a generalization:

> import Test.Extrapolate
> check (prop_sortCount :: Int -> [Int] -> Bool)
*** Failed! Falsifiable (after 4 tests):
0 [0,0]

Generalization:
x (x:x:_)

This hopefully makes it easier to find the source of the bug. In this case, the faulty sort function discards repeated elements.

Further reading

For more examples, see the eg folder. For type signatures, other options and uses, see Extrapolate's API documentation.

There are two other tools for Haskell capable of producing generalized counter-examples: SmartCheck and Lazy SmallCheck 2012.

Extrapolate was accepted for presentation at IFL 2017, see the IFL paper about Extrapolate. Extrapolate is also subject to a chapter in a PhD Thesis (2017).