servant-quickcheck-0.0.4: QuickCheck entire APIs

Safe HaskellNone
LanguageHaskell2010

Servant.QuickCheck

Contents

Description

Servant.QuickCheck provides utilities related to using QuickCheck over an API. Rather than specifying properties that individual handlers must satisfy, you can state properties that ought to hold true of the entire API.

While the API must be described with servant types, the server being tested itself need not be implemented with servant-server (or indeed, written in Haskell).

The documentation of the Useful predicates sections is meant to serve as a set of helpful pointers for learning more about best practices concerning REST APIs.

Synopsis

Property testing

serverSatisfies :: HasGenRequest a => Proxy a -> BaseUrl -> Args -> Predicates -> Expectation Source #

Check that a server satisfies the set of properties specified.

Note that, rather than having separate tests for each property you'd like to test, you should generally prefer to combine all properties into a single test. This enables a more parsimonious generation of requests and responses with the same testing depth.

Example usage:

goodAPISpec = describe "my server" $ do

  it "follows best practices" $ do
    withServantServer api server $ \burl ->
      serverSatisfies api burl stdArgs (not500
                                    <%> onlyJsonObjects
                                    <%> notAllowedContainsAllowHeader
                                    <%> mempty)

Since 0.0.0.0

Predicates

Useful predicates

The predicates below are often useful. Some check RFC compliance; some are best practice, and some are useful to check that APIs follow in-house best-practices. Included in the documentation for each is a list of references to any relevant RFCs and other links, as well as what type of predicate it is (RFC Compliance, Best Practice, Optional).

RFCs distinguish between the force of requirements (e.g. MUST vs. SHOULD). RFC Compliance includes any absolute requirements present in RFCs. The Best Practices includes, in addition to RFC recommendations, recommendations found elsewhere or generally accepted.

not500 :: ResponsePredicate Source #

Best Practice

500 Internal Server Error should be avoided - it may represent some issue with the application code, and it moreover gives the client little indication of how to proceed or what went wrong.

This function checks that the response code is not 500.

Since 0.0.0.0

notLongerThan :: Integer -> RequestPredicate Source #

Optional

This function checks that the response from the server does not take longer than the specified number of nanoseconds.

Since 0.0.2.1

onlyJsonObjects :: ResponsePredicate Source #

Best Practice

Returning anything other than an object when returning JSON is considered bad practice, as:

  1. it is hard to modify the returned value while maintaining backwards compatibility
  2. many older tools do not support top-level arrays
  3. whether top-level numbers, booleans, or strings are valid JSON depends on what RFC you're going by
  4. there are security issues with top-level arrays

This function checks that any application/json responses only return JSON objects (and not arrays, strings, numbers, or booleans) at the top level.

References:

Since 0.0.0.0

notAllowedContainsAllowHeader :: RequestPredicate Source #

RFC Compliance

When an HTTP request has a method that is not allowed, a 405 response should be returned. Additionally, it is good practice to return an Allow header with the list of allowed methods.

This function checks that every 405 Method Not Allowed response contains an Allow header with a list of standard HTTP methods.

Note that servant itself does not currently set the Allow headers.

References:

Since 0.0.0.0

unauthorizedContainsWWWAuthenticate :: ResponsePredicate Source #

RFC Compliance

Any 401 Unauthorized response must include a WWW-Authenticate header.

This function checks that, if a response has status code 401, it contains a WWW-Authenticate header.

References:

Since 0.0.0.0

getsHaveCacheControlHeader :: RequestPredicate Source #

Best Practice

Whether or not a representation should be cached, it is good practice to have a Cache-Control header for GET requests. If the representation should not be cached, used Cache-Control: no-cache.

This function checks that GET responses have Cache-Control header. It does NOT currently check that the header is valid.

References:

Since 0.0.0.0

headsHaveCacheControlHeader :: RequestPredicate Source #

Best Practice

Like getsHaveCacheControlHeader, but for HEAD requests.

Since 0.0.0.0

createContainsValidLocation :: RequestPredicate Source #

Optional

When creating a new resource, it is good practice to provide a Location header with a link to the created resource.

This function checks that every 201 Created response contains a Location header, and that the link in it responds with a 2XX response code to GET requests.

This is considered optional because other means of linking to the resource (e.g. via the response body) are also acceptable; linking to the resource in some way is considered best practice.

References:

Since 0.0.0.0

Html Predicates

htmlIncludesDoctype :: ResponsePredicate Source #

RFC Compliance
An HTML
document will start with exactly this string: html

This function checks that HTML documents (those with `Content-Type: text/html...`) include a DOCTYPE declaration at the top. We do not enforce capital case for the string DOCTYPE.

References:

Predicate utilities and types

(<%>) :: JoinPreds a => a -> Predicates -> Predicates infixr 6 Source #

Adds a new predicate (either ResponsePredicate or RequestPredicate) to the existing predicates.

not500 <%> onlyJsonObjects <%> empty

Since 0.0.0.0

data Predicates Source #

A set of predicates. Construct one with mempty and <%>.

Instances

Generic Predicates Source # 

Associated Types

type Rep Predicates :: * -> * #

Monoid Predicates Source # 
type Rep Predicates Source # 
type Rep Predicates = D1 (MetaData "Predicates" "Servant.QuickCheck.Internal.Predicates" "servant-quickcheck-0.0.4-7QE5TVMz9ssGLzXTaojPUl" False) (C1 (MetaCons "Predicates" PrefixI True) ((:*:) (S1 (MetaSel (Just Symbol "requestPredicates") NoSourceUnpackedness NoSourceStrictness DecidedLazy) (Rec0 RequestPredicate)) (S1 (MetaSel (Just Symbol "responsePredicates") NoSourceUnpackedness NoSourceStrictness DecidedLazy) (Rec0 ResponsePredicate))))

newtype ResponsePredicate Source #

A predicate that depends only on the response.

Since 0.0.0.0

newtype RequestPredicate Source #

A predicate that depends on both the request and the response.

Since 0.0.0.0

Equality testing

serversEqual :: HasGenRequest a => Proxy a -> BaseUrl -> BaseUrl -> Args -> ResponseEquality ByteString -> Expectation Source #

Check that the two servers running under the provided BaseUrls behave identically by randomly generating arguments (captures, query params, request bodies, headers, etc.) expected by the server. If, given the same request, the response is not the same (according to the definition of == for the return datatype), the Expectation fails, printing the counterexample.

The Int argument specifies maximum number of test cases to generate and run.

Evidently, if the behaviour of the server is expected to be non-deterministic, this function may produce spurious failures

Note that only valid requests are generated and tested. As an example of why this matters, let's say your API specifies that a particular endpoint can only generate JSON. serversEqual will then not generate any requests with an Accept header _other_ than application/json. It may therefore fail to notice that one application, when the request has Accept: text/html, returns a 406 Not Acceptable HTTP response, and another returns a 200 Success, but with application/json as the content-type.

The fact that only valid requests are tested also means that no endpoints not listed in the API type are tested.

Since 0.0.0.0

Response equality

Often the normal equality of responses is not what we want. For example, if responses contain a Date header with the time of the response, responses will fail to be equal even though they morally are. This datatype represents other means of checking equality *** Useful ResponseEqualitys

bodyEquality :: Eq b => ResponseEquality b Source #

ByteString Eq instance over the response body.

Since 0.0.0.0

jsonEquality :: JsonEq b => ResponseEquality b Source #

Equality as Value. This means that if two bodies are equal as JSON (e.g., insignificant whitespace difference) they are considered equal.

Since 0.0.3.0

allEquality :: Eq b => ResponseEquality b Source #

Use Eq instance for Response

Since 0.0.0.0

Response equality type

Test setup helpers

Helpers to setup and teardown servant servers during tests.

withServantServer :: HasServer a '[] => Proxy a -> IO (Server a) -> (BaseUrl -> IO r) -> IO r Source #

Start a servant application on an open port, run the provided function, then stop the application.

Since 0.0.0.0

withServantServerAndContext :: HasServer a ctx => Proxy a -> Context ctx -> IO (Server a) -> (BaseUrl -> IO r) -> IO r Source #

Like withServantServer, but allows passing in a Context to the application.

Since 0.0.0.0

defaultArgs :: Args Source #

QuickCheck Args with 1000 rather than 100 test cases.

Since 0.0.0.0

Re-exports

Types and constructors from other packages that are generally needed for using servant-quickcheck.

data BaseUrl :: * #

Simple data type to represent the target of HTTP requests for servant's automatically-generated clients.

Constructors

BaseUrl 

Fields

Instances

Eq BaseUrl 

Methods

(==) :: BaseUrl -> BaseUrl -> Bool #

(/=) :: BaseUrl -> BaseUrl -> Bool #

Ord BaseUrl 
Show BaseUrl 
Generic BaseUrl 

Associated Types

type Rep BaseUrl :: * -> * #

Methods

from :: BaseUrl -> Rep BaseUrl x #

to :: Rep BaseUrl x -> BaseUrl #

type Rep BaseUrl 
type Rep BaseUrl = D1 (MetaData "BaseUrl" "Servant.Client.Core.Internal.BaseUrl" "servant-client-core-0.12-HcpTqRoS8IX6WJaOe47uKX" False) (C1 (MetaCons "BaseUrl" PrefixI True) ((:*:) ((:*:) (S1 (MetaSel (Just Symbol "baseUrlScheme") NoSourceUnpackedness NoSourceStrictness DecidedLazy) (Rec0 Scheme)) (S1 (MetaSel (Just Symbol "baseUrlHost") NoSourceUnpackedness NoSourceStrictness DecidedLazy) (Rec0 String))) ((:*:) (S1 (MetaSel (Just Symbol "baseUrlPort") NoSourceUnpackedness NoSourceStrictness DecidedLazy) (Rec0 Int)) (S1 (MetaSel (Just Symbol "baseUrlPath") NoSourceUnpackedness NoSourceStrictness DecidedLazy) (Rec0 String)))))

data Scheme :: * #

URI scheme to use

Constructors

Http

http://

Https

https://

Instances

Eq Scheme 

Methods

(==) :: Scheme -> Scheme -> Bool #

(/=) :: Scheme -> Scheme -> Bool #

Ord Scheme 
Show Scheme 
Generic Scheme 

Associated Types

type Rep Scheme :: * -> * #

Methods

from :: Scheme -> Rep Scheme x #

to :: Rep Scheme x -> Scheme #

type Rep Scheme 
type Rep Scheme = D1 (MetaData "Scheme" "Servant.Client.Core.Internal.BaseUrl" "servant-client-core-0.12-HcpTqRoS8IX6WJaOe47uKX" False) ((:+:) (C1 (MetaCons "Http" PrefixI False) U1) (C1 (MetaCons "Https" PrefixI False) U1))

data Args :: * #

Args specifies arguments to the QuickCheck driver

Constructors

Args 

Fields

  • replay :: Maybe (QCGen, Int)

    Should we replay a previous test? Note: saving a seed from one version of QuickCheck and replaying it in another is not supported. If you want to store a test case permanently you should save the test case itself.

  • maxSuccess :: Int

    Maximum number of successful tests before succeeding. Testing stops at the first failure. If all tests are passing and you want to run more tests, increase this number.

  • maxDiscardRatio :: Int

    Maximum number of discarded tests per successful test before giving up

  • maxSize :: Int

    Size to use for the biggest test cases

  • chatty :: Bool

    Whether to print anything

  • maxShrinks :: Int

    Maximum number of shrinks to before giving up. Setting this to zero turns shrinking off.

data Proxy k t :: forall k. k -> * #

A concrete, poly-kinded proxy type

Constructors

Proxy 

Instances

Monad (Proxy *) 

Methods

(>>=) :: Proxy * a -> (a -> Proxy * b) -> Proxy * b #

(>>) :: Proxy * a -> Proxy * b -> Proxy * b #

return :: a -> Proxy * a #

fail :: String -> Proxy * a #

Functor (Proxy *) 

Methods

fmap :: (a -> b) -> Proxy * a -> Proxy * b #

(<$) :: a -> Proxy * b -> Proxy * a #

Applicative (Proxy *) 

Methods

pure :: a -> Proxy * a #

(<*>) :: Proxy * (a -> b) -> Proxy * a -> Proxy * b #

(*>) :: Proxy * a -> Proxy * b -> Proxy * b #

(<*) :: Proxy * a -> Proxy * b -> Proxy * a #

Foldable (Proxy *) 

Methods

fold :: Monoid m => Proxy * m -> m #

foldMap :: Monoid m => (a -> m) -> Proxy * a -> m #

foldr :: (a -> b -> b) -> b -> Proxy * a -> b #

foldr' :: (a -> b -> b) -> b -> Proxy * a -> b #

foldl :: (b -> a -> b) -> b -> Proxy * a -> b #

foldl' :: (b -> a -> b) -> b -> Proxy * a -> b #

foldr1 :: (a -> a -> a) -> Proxy * a -> a #

foldl1 :: (a -> a -> a) -> Proxy * a -> a #

toList :: Proxy * a -> [a] #

null :: Proxy * a -> Bool #

length :: Proxy * a -> Int #

elem :: Eq a => a -> Proxy * a -> Bool #

maximum :: Ord a => Proxy * a -> a #

minimum :: Ord a => Proxy * a -> a #

sum :: Num a => Proxy * a -> a #

product :: Num a => Proxy * a -> a #

Traversable (Proxy *) 

Methods

traverse :: Applicative f => (a -> f b) -> Proxy * a -> f (Proxy * b) #

sequenceA :: Applicative f => Proxy * (f a) -> f (Proxy * a) #

mapM :: Monad m => (a -> m b) -> Proxy * a -> m (Proxy * b) #

sequence :: Monad m => Proxy * (m a) -> m (Proxy * a) #

Generic1 (Proxy *) 

Associated Types

type Rep1 (Proxy * :: * -> *) :: * -> * #

Methods

from1 :: Proxy * a -> Rep1 (Proxy *) a #

to1 :: Rep1 (Proxy *) a -> Proxy * a #

FromJSON1 (Proxy *) 

Methods

liftParseJSON :: (Value -> Parser a) -> (Value -> Parser [a]) -> Value -> Parser (Proxy * a) #

liftParseJSONList :: (Value -> Parser a) -> (Value -> Parser [a]) -> Value -> Parser [Proxy * a] #

Alternative (Proxy *) 

Methods

empty :: Proxy * a #

(<|>) :: Proxy * a -> Proxy * a -> Proxy * a #

some :: Proxy * a -> Proxy * [a] #

many :: Proxy * a -> Proxy * [a] #

MonadPlus (Proxy *) 

Methods

mzero :: Proxy * a #

mplus :: Proxy * a -> Proxy * a -> Proxy * a #

Eq1 (Proxy *)

Since: 4.9.0.0

Methods

liftEq :: (a -> b -> Bool) -> Proxy * a -> Proxy * b -> Bool #

Ord1 (Proxy *)

Since: 4.9.0.0

Methods

liftCompare :: (a -> b -> Ordering) -> Proxy * a -> Proxy * b -> Ordering #

Read1 (Proxy *)

Since: 4.9.0.0

Methods

liftReadsPrec :: (Int -> ReadS a) -> ReadS [a] -> Int -> ReadS (Proxy * a) #

liftReadList :: (Int -> ReadS a) -> ReadS [a] -> ReadS [Proxy * a] #

Show1 (Proxy *)

Since: 4.9.0.0

Methods

liftShowsPrec :: (Int -> a -> ShowS) -> ([a] -> ShowS) -> Int -> Proxy * a -> ShowS #

liftShowList :: (Int -> a -> ShowS) -> ([a] -> ShowS) -> [Proxy * a] -> ShowS #

Hashable1 (Proxy *) 

Methods

liftHashWithSalt :: (Int -> a -> Int) -> Int -> Proxy * a -> Int #

Bounded (Proxy k s) 

Methods

minBound :: Proxy k s #

maxBound :: Proxy k s #

Enum (Proxy k s) 

Methods

succ :: Proxy k s -> Proxy k s #

pred :: Proxy k s -> Proxy k s #

toEnum :: Int -> Proxy k s #

fromEnum :: Proxy k s -> Int #

enumFrom :: Proxy k s -> [Proxy k s] #

enumFromThen :: Proxy k s -> Proxy k s -> [Proxy k s] #

enumFromTo :: Proxy k s -> Proxy k s -> [Proxy k s] #

enumFromThenTo :: Proxy k s -> Proxy k s -> Proxy k s -> [Proxy k s] #

Eq (Proxy k s) 

Methods

(==) :: Proxy k s -> Proxy k s -> Bool #

(/=) :: Proxy k s -> Proxy k s -> Bool #

Data t => Data (Proxy * t) 

Methods

gfoldl :: (forall d b. Data d => c (d -> b) -> d -> c b) -> (forall g. g -> c g) -> Proxy * t -> c (Proxy * t) #

gunfold :: (forall b r. Data b => c (b -> r) -> c r) -> (forall r. r -> c r) -> Constr -> c (Proxy * t) #

toConstr :: Proxy * t -> Constr #

dataTypeOf :: Proxy * t -> DataType #

dataCast1 :: Typeable (* -> *) t => (forall d. Data d => c (t d)) -> Maybe (c (Proxy * t)) #

dataCast2 :: Typeable (* -> * -> *) t => (forall d e. (Data d, Data e) => c (t d e)) -> Maybe (c (Proxy * t)) #

gmapT :: (forall b. Data b => b -> b) -> Proxy * t -> Proxy * t #

gmapQl :: (r -> r' -> r) -> r -> (forall d. Data d => d -> r') -> Proxy * t -> r #

gmapQr :: (r' -> r -> r) -> r -> (forall d. Data d => d -> r') -> Proxy * t -> r #

gmapQ :: (forall d. Data d => d -> u) -> Proxy * t -> [u] #

gmapQi :: Int -> (forall d. Data d => d -> u) -> Proxy * t -> u #

gmapM :: Monad m => (forall d. Data d => d -> m d) -> Proxy * t -> m (Proxy * t) #

gmapMp :: MonadPlus m => (forall d. Data d => d -> m d) -> Proxy * t -> m (Proxy * t) #

gmapMo :: MonadPlus m => (forall d. Data d => d -> m d) -> Proxy * t -> m (Proxy * t) #

Ord (Proxy k s) 

Methods

compare :: Proxy k s -> Proxy k s -> Ordering #

(<) :: Proxy k s -> Proxy k s -> Bool #

(<=) :: Proxy k s -> Proxy k s -> Bool #

(>) :: Proxy k s -> Proxy k s -> Bool #

(>=) :: Proxy k s -> Proxy k s -> Bool #

max :: Proxy k s -> Proxy k s -> Proxy k s #

min :: Proxy k s -> Proxy k s -> Proxy k s #

Read (Proxy k s) 
Show (Proxy k s) 

Methods

showsPrec :: Int -> Proxy k s -> ShowS #

show :: Proxy k s -> String #

showList :: [Proxy k s] -> ShowS #

Ix (Proxy k s) 

Methods

range :: (Proxy k s, Proxy k s) -> [Proxy k s] #

index :: (Proxy k s, Proxy k s) -> Proxy k s -> Int #

unsafeIndex :: (Proxy k s, Proxy k s) -> Proxy k s -> Int

inRange :: (Proxy k s, Proxy k s) -> Proxy k s -> Bool #

rangeSize :: (Proxy k s, Proxy k s) -> Int #

unsafeRangeSize :: (Proxy k s, Proxy k s) -> Int

Generic (Proxy k t) 

Associated Types

type Rep (Proxy k t) :: * -> * #

Methods

from :: Proxy k t -> Rep (Proxy k t) x #

to :: Rep (Proxy k t) x -> Proxy k t #

Semigroup (Proxy k s) 

Methods

(<>) :: Proxy k s -> Proxy k s -> Proxy k s #

sconcat :: NonEmpty (Proxy k s) -> Proxy k s #

stimes :: Integral b => b -> Proxy k s -> Proxy k s #

Monoid (Proxy k s) 

Methods

mempty :: Proxy k s #

mappend :: Proxy k s -> Proxy k s -> Proxy k s #

mconcat :: [Proxy k s] -> Proxy k s #

Hashable (Proxy * a) 

Methods

hashWithSalt :: Int -> Proxy * a -> Int #

hash :: Proxy * a -> Int #

FromJSON (Proxy k a) 

Methods

parseJSON :: Value -> Parser (Proxy k a) #

parseJSONList :: Value -> Parser [Proxy k a] #

type Rep1 (Proxy *) 
type Rep1 (Proxy *) = D1 (MetaData "Proxy" "Data.Proxy" "base" False) (C1 (MetaCons "Proxy" PrefixI False) U1)
type Rep (Proxy k t) 
type Rep (Proxy k t) = D1 (MetaData "Proxy" "Data.Proxy" "base" False) (C1 (MetaCons "Proxy" PrefixI False) U1)
type Code (Proxy * t0) 
type Code (Proxy * t0) = (:) [*] ([] *) ([] [*])
type DatatypeInfoOf (Proxy * t0) 
type DatatypeInfoOf (Proxy * t0) = ADT "Data.Proxy" "Proxy" ((:) ConstructorInfo (Constructor "Proxy") ([] ConstructorInfo))