http-client-0.4.2.2: An HTTP client engine, intended as a base layer for more user-friendly packages.

Safe HaskellNone

Network.HTTP.Client

Contents

Description

This is the main entry point for using http-client. Used by itself, this module provides low-level access for streaming request and response bodies, and only non-secure HTTP connections. Helper packages such as http-conduit provided higher level streaming approaches, while other helper packages like http-client-tls provide secure connections.

There are three core components to be understood here: requests, responses, and managers. A Manager keeps track of open connections to various hosts, and when requested, will provide either an existing open connection or create a new connection on demand. A Manager also automatically reaps connections which have been unused for a certain period of time. A Manager allows for more efficient HTTP usage by allowing for keep-alive connections. Secure HTTP connections can be allowed by modifying the settings used for creating a manager. The simplest way to create a Manager is with:

 newManager defaultManagerSettings

or using the bracket pattern with

 withManager defaultManagerSettings

While generally speaking it is a good idea to share a single Manager throughout your application, there are cases where it makes more sense to create and destroy Managers more frequently. As an example, if you have an application which will make a large number of requests to different hosts, and will never make more than one connection to a single host, then sharing a Manager will result in idle connections being kept open longer than necessary. In such a situation, it makes sense to use withManager around each new request, to avoid running out of file descriptors. (Note that the managerIdleConnectionCount setting mitigates the risk of leaking too many file descriptors.)

The next core component is a Request, which represents a single HTTP request to be sent to a specific server. Requests allow for many settings to control exact how they function, but usually the simplest approach for creating a Request is to use parseUrl.

Finally, a Response is the result of sending a single Request to a server, over a connection which was acquired from a Manager. Note that you must close the response when you're done with it to ensure that the connection is recycled to the Manager to either be used by another request, or to be reaped. Usage of withResponse will ensure that this happens automatically.

Helper packages may provide replacements for various recommendations listed above. For example, if using http-client-tls, instead of using defaultManagerSettings, you would want to use tlsManagerSettings. Be sure to read the relevant helper library documentation for more information.

A note on exceptions: for the most part, all actions that perform I/O should be assumed to throw an HttpException in the event of some problem, and all pure functions will be total. For example, withResponse, httpLbs, and BodyReader can all throw exceptions. Functions like responseStatus and applyBasicAuth are guaranteed to be total (or there's a bug in the library).

One thing to be cautioned about: the type of parseUrl allows it to work in different monads. If used in the IO monad, it will throw an exception in the case of an invalid URI. In addition, if you leverage the IsString instance of the Request value via OverloadedStrings, an invalid URI will result in a partial value. Caveat emptor!

Non-2xx responses: the default behavior of all functions in http-client is to automatically perform up to 10 redirects (response codes 301, 302, 303, and 307), and to throw a StatusCodeException on all responses whose status are not in the 2xx range. These behaviors can be overridden by the redirectCount and checkStatus settings on a request, respectively.

Synopsis

Performing requests

withResponse :: Request -> Manager -> (Response BodyReader -> IO a) -> IO aSource

Perform a Request using a connection acquired from the given Manager, and then provide the Response to the given function. This function is fully exception safe, guaranteeing that the response will be closed when the inner function exits. It is defined as:

 withResponse req man f = bracket (responseOpen req man) responseClose f

It is recommended that you use this function in place of explicit calls to responseOpen and responseClose.

You will need to use functions such as brRead to consume the response body.

Since 0.1.0

httpLbs :: Request -> Manager -> IO (Response ByteString)Source

A convenience wrapper around withResponse which reads in the entire response body and immediately closes the connection. Note that this function performs fully strict I/O, and only uses a lazy ByteString in its response for memory efficiency. If you are anticipating a large response body, you are encouraged to use withResponse and brRead instead.

Since 0.1.0

httpNoBody :: Request -> Manager -> IO (Response ())Source

A convenient wrapper around withResponse which ignores the response body. This is useful, for example, when performing a HEAD request.

Since 0.3.2

responseOpen :: Request -> Manager -> IO (Response BodyReader)Source

The most low-level function for initiating an HTTP request.

The first argument to this function gives a full specification on the request: the host to connect to, whether to use SSL, headers, etc. Please see Request for full details. The second argument specifies which Manager should be used.

This function then returns a Response with a BodyReader. The Response contains the status code and headers that were sent back to us, and the BodyReader contains the body of the request. Note that this BodyReader allows you to have fully interleaved IO actions during your HTTP download, making it possible to download very large responses in constant memory.

An important note: the response body returned by this function represents a live HTTP connection. As such, if you do not use the response body, an open socket will be retained indefinitely. You must be certain to call responseClose on this response to free up resources.

This function automatically performs any necessary redirects, as specified by the redirectCount setting.

When implementing a (reverse) proxy using this function or relating functions, it's wise to remove Transfer-Encoding:, Content-Length:, Content-Encoding: and Accept-Encoding: from request and response headers to be relayed.

Since 0.1.0

responseClose :: Response a -> IO ()Source

Close any open resources associated with the given Response. In general, this will either close an active Connection or return it to the Manager to be reused.

Since 0.1.0

Tracking redirect history

withResponseHistory :: Request -> Manager -> (HistoriedResponse BodyReader -> IO a) -> IO aSource

A variant of withResponse which keeps a history of all redirects performed in the interim, together with the first 1024 bytes of their response bodies.

Since 0.4.1

responseOpenHistory :: Request -> Manager -> IO (HistoriedResponse BodyReader)Source

A variant of responseOpen which keeps a history of all redirects performed in the interim, together with the first 1024 bytes of their response bodies.

Since 0.4.1

data HistoriedResponse body Source

A datatype holding information on redirected requests and the final response.

Since 0.4.1

Instances

Functor HistoriedResponse 
Typeable1 HistoriedResponse 
Foldable HistoriedResponse 
Traversable HistoriedResponse 
Show body => Show (HistoriedResponse body) 
Generic (HistoriedResponse body) 

hrRedirects :: HistoriedResponse body -> [(Request, Response ByteString)]Source

Requests which resulted in a redirect, together with their responses. The response contains the first 1024 bytes of the body.

Since 0.4.1

hrFinalRequest :: HistoriedResponse body -> RequestSource

The final request performed.

Since 0.4.1

hrFinalResponse :: HistoriedResponse body -> Response bodySource

The response from the final request.

Since 0.4.1

Connection manager

data Manager Source

Keeps track of open connections for keep-alive.

If possible, you should share a single Manager between multiple threads and requests.

Since 0.1.0

Instances

Typeable Manager 

newManager :: ManagerSettings -> IO ManagerSource

Create a Manager. You may manually call closeManager to shut it down, or allow the Manager to be shut down automatically based on garbage collection.

Creating a new Manager is a relatively expensive operation, you are advised to share a single Manager between requests instead.

The first argument to this function is often defaultManagerSettings, though add-on libraries may provide a recommended replacement.

Since 0.1.0

closeManager :: Manager -> IO ()Source

Close all connections in a Manager.

Note that this doesn't affect currently in-flight connections, meaning you can safely use it without hurting any queries you may have concurrently running.

Since 0.1.0

withManager :: ManagerSettings -> (Manager -> IO a) -> IO aSource

Create, use and close a Manager.

Since 0.2.1

Connection manager settings

data ManagerSettings Source

Settings for a Manager. Please use the defaultManagerSettings function and then modify individual settings. For more information, see http://www.yesodweb.com/book/settings-types.

Since 0.1.0

Instances

Typeable ManagerSettings 

defaultManagerSettings :: ManagerSettingsSource

Default value for ManagerSettings.

Note that this value does not have support for SSL/TLS. If you need to make any https connections, please use the network-client-tls package, which provides a tlsManagerSettings value.

Since 0.1.0

managerConnCount :: ManagerSettings -> IntSource

Number of connections to a single host to keep alive. Default: 10.

Since 0.1.0

managerRawConnection :: ManagerSettings -> IO (Maybe HostAddress -> String -> Int -> IO Connection)Source

Create an insecure connection.

Since 0.1.0 FIXME in the future, combine managerTlsConnection and managerTlsProxyConnection

managerTlsConnection :: ManagerSettings -> IO (Maybe HostAddress -> String -> Int -> IO Connection)Source

Create a TLS connection. Default behavior: throw an exception that TLS is not supported.

Since 0.1.0

managerResponseTimeout :: ManagerSettings -> Maybe IntSource

Default timeout (in microseconds) to be applied to requests which do not provide a timeout value.

Default is 30 seconds

Since 0.1.0

managerRetryableException :: ManagerSettings -> SomeException -> BoolSource

Exceptions for which we should retry our request if we were reusing an already open connection. In the case of IOExceptions, for example, we assume that the connection was closed on the server and therefore open a new one.

Since 0.1.0

managerWrapIOException :: ManagerSettings -> forall a. IO a -> IO aSource

Action wrapped around all attempted Requests, usually used to wrap up exceptions in library-specific types.

Default: wrap all IOExceptions in the InternalIOException constructor.

Since 0.1.0

managerIdleConnectionCount :: ManagerSettings -> IntSource

Total number of idle connection to keep open at a given time.

This limit helps deal with the case where you are making a large number of connections to different hosts. Without this limit, you could run out of file descriptors.

Default: 512

Since 0.3.7

Helpers

rawConnectionModifySocket :: (Socket -> IO ()) -> IO (Maybe HostAddress -> String -> Int -> IO Connection)Source

A value for the managerRawConnection setting, but also allows you to modify the underlying Socket to set additional settings. For a motivating use case, see: https://github.com/snoyberg/http-client/issues/71.

Since 0.3.8

Request

parseUrl :: MonadThrow m => String -> m RequestSource

Convert a URL into a Request.

This defaults some of the values in Request, such as setting method to GET and requestHeaders to [].

Since this function uses MonadThrow, the return monad can be anything that is an instance of MonadThrow, such as IO or Maybe.

Since 0.1.0

applyBasicAuth :: ByteString -> ByteString -> Request -> RequestSource

Add a Basic Auth header (with the specified user name and password) to the given Request. Ignore error handling:

  applyBasicAuth "user" "pass" $ fromJust $ parseUrl url

Since 0.1.0

urlEncodedBody :: [(ByteString, ByteString)] -> Request -> RequestSource

Add url-encoded parameters to the Request.

This sets a new requestBody, adds a content-type request header and changes the method to POST.

Since 0.1.0

getUri :: Request -> URISource

Extract a URI from the request.

Since 0.1.0

setQueryString :: [(ByteString, Maybe ByteString)] -> Request -> RequestSource

Set the query string to the given key/value pairs.

Since 0.3.6

Request type and fields

data Request Source

All information on how to connect to a host and what should be sent in the HTTP request.

If you simply wish to download from a URL, see parseUrl.

The constructor for this data type is not exposed. Instead, you should use either the def method to retrieve a default instance, or parseUrl to construct from a URL, and then use the records below to make modifications. This approach allows http-client to add configuration options without breaking backwards compatibility.

For example, to construct a POST request, you could do something like:

 initReq <- parseUrl "http://www.example.com/path"
 let req = initReq
             { method = "POST"
             }

For more information, please see http://www.yesodweb.com/book/settings-types.

Since 0.1.0

Instances

Show Request 
Typeable Request 
IsString Request 
Default Request 

method :: Request -> MethodSource

HTTP request method, eg GET, POST.

Since 0.1.0

secure :: Request -> BoolSource

Whether to use HTTPS (ie, SSL).

Since 0.1.0

host :: Request -> ByteStringSource

Requested host name, used for both the IP address to connect to and the host request header.

Since 0.1.0

port :: Request -> IntSource

The port to connect to. Also used for generating the host request header.

Since 0.1.0

path :: Request -> ByteStringSource

Everything from the host to the query string.

Since 0.1.0

queryString :: Request -> ByteStringSource

Query string appended to the path.

Since 0.1.0

requestHeaders :: Request -> RequestHeadersSource

Custom HTTP request headers

The Content-Length and Transfer-Encoding headers are set automatically by this module, and shall not be added to requestHeaders.

If not provided by the user, Host will automatically be set based on the host and port fields.

Moreover, the Accept-Encoding header is set implicitly to gzip for convenience by default. This behaviour can be overridden if needed, by setting the header explicitly to a different value. In order to omit the Accept-Header altogether, set it to the empty string "". If you need an empty Accept-Header (i.e. requesting the identity encoding), set it to a non-empty white-space string, e.g. " ". See RFC 2616 section 14.3 for details about the semantics of the Accept-Header field. If you request a content-encoding not supported by this module, you will have to decode it yourself (see also the decompress field).

Note: Multiple header fields with the same field-name will result in multiple header fields being sent and therefore it's the responsibility of the client code to ensure that the rules from RFC 2616 section 4.2 are honoured.

Since 0.1.0

requestBody :: Request -> RequestBodySource

Request body to be sent to the server.

Since 0.1.0

proxy :: Request -> Maybe ProxySource

Optional HTTP proxy.

Since 0.1.0

applyBasicProxyAuth :: ByteString -> ByteString -> Request -> RequestSource

Add a Proxy-Authorization header (with the specified username and password) to the given Request. Ignore error handling:

 applyBasicProxyAuth "user" "pass" <$> parseUrl "http://example.org"

Since 0.3.4

decompress :: Request -> ByteString -> BoolSource

Predicate to specify whether gzipped data should be decompressed on the fly (see alwaysDecompress and browserDecompress). Argument is the mime type. Default: browserDecompress.

Since 0.1.0

redirectCount :: Request -> IntSource

How many redirects to follow when getting a resource. 0 means follow no redirects. Default value: 10.

Since 0.1.0

checkStatus :: Request -> Status -> ResponseHeaders -> CookieJar -> Maybe SomeExceptionSource

Check the status code. Note that this will run after all redirects are performed. Default: return a StatusCodeException on non-2XX responses.

Since 0.1.0

responseTimeout :: Request -> Maybe IntSource

Number of microseconds to wait for a response. If Nothing, will wait indefinitely. Default: use managerResponseTimeout (which by default is 30 seconds).

Since 0.1.0

cookieJar :: Request -> Maybe CookieJarSource

A user-defined cookie jar. If Nothing, no cookie handling will take place, "Cookie" headers in requestHeaders will be sent raw, and responseCookieJar will be empty.

Since 0.1.0

Request body

data RequestBody Source

When using one of the RequestBodyStream / RequestBodyStreamChunked constructors, you must ensure that the GivesPopper can be called multiple times. Usually this is not a problem.

The RequestBodyStreamChunked will send a chunked request body. Note that not all servers support this. Only use RequestBodyStreamChunked if you know the server you're sending to supports chunked request bodies.

Since 0.1.0

Instances

Typeable RequestBody 
Monoid RequestBody 

type Popper = IO ByteStringSource

A function which generates successive chunks of a request body, provider a single empty bytestring when no more data is available.

Since 0.1.0

type NeedsPopper a = Popper -> IO aSource

A function which must be provided with a Popper.

Since 0.1.0

type GivesPopper a = NeedsPopper a -> IO aSource

A function which will provide a Popper to a NeedsPopper. This seemingly convoluted structure allows for creation of request bodies which allocate scarce resources in an exception safe manner.

Since 0.1.0

Response

data Response body Source

A simple representation of the HTTP response.

Since 0.1.0

Instances

Functor Response 
Typeable1 Response 
Foldable Response 
Traversable Response 
Eq body => Eq (Response body) 
Show body => Show (Response body) 

responseStatus :: Response body -> StatusSource

Status code of the response.

Since 0.1.0

responseVersion :: Response body -> HttpVersionSource

HTTP version used by the server.

Since 0.1.0

responseHeaders :: Response body -> ResponseHeadersSource

Response headers sent by the server.

Since 0.1.0

responseBody :: Response body -> bodySource

Response body sent by the server.

Since 0.1.0

responseCookieJar :: Response body -> CookieJarSource

Cookies set on the client after interacting with the server. If cookies have been disabled by setting cookieJar to Nothing, then this will always be empty.

Since 0.1.0

Response body

type BodyReader = IO ByteStringSource

An IO action that represents an incoming response body coming from the server. Data provided by this action has already been gunzipped and de-chunked, and respects any content-length headers present.

The action gets a single chunk of data from the response body, or an empty bytestring if no more data is available.

Since 0.4.0

brRead :: BodyReader -> IO ByteStringSource

Get a single chunk of data from the response body, or an empty bytestring if no more data is available.

Since 0.1.0

brConsume :: BodyReader -> IO [ByteString]Source

Strictly consume all remaining chunks of data from the stream.

Since 0.1.0

Misc

data HttpException Source

Constructors

StatusCodeException Status ResponseHeaders CookieJar 
InvalidUrlException String String 
TooManyRedirects [Response ByteString]

List of encountered responses containing redirects in reverse chronological order; including last redirect, which triggered the exception and was not followed.

UnparseableRedirect (Response ByteString)

Response containing unparseable redirect.

TooManyRetries 
HttpParserException String 
HandshakeFailed 
OverlongHeaders 
ResponseTimeout 
FailedConnectionException String Int

host/port

FailedConnectionException2 String Int Bool SomeException

hostportsecure

ExpectedBlankAfter100Continue 
InvalidStatusLine ByteString 
InvalidHeader ByteString 
InternalIOException IOException 
ProxyConnectException ByteString Int (Either ByteString HttpException)

host/port

NoResponseDataReceived 
TlsException SomeException 
TlsNotSupported 
ResponseBodyTooShort Word64 Word64

Expected size/actual size.

Since 1.9.4

InvalidChunkHeaders

Since 1.9.4

IncompleteHeaders 
InvalidDestinationHost ByteString 
HttpZlibException ZlibException

Since 0.3

data CookieJar Source

Instances

Eq CookieJar 
Read CookieJar 
Show CookieJar 
Typeable CookieJar 
Default CookieJar 
Monoid CookieJar

Since 1.9

data Proxy Source

Define a HTTP proxy, consisting of a hostname and port number.

Constructors

Proxy 

Fields

proxyHost :: ByteString

The host name of the HTTP proxy.

proxyPort :: Int

The port number of the HTTP proxy.

Instances

Eq Proxy 
Ord Proxy 
Read Proxy 
Show Proxy 
Typeable Proxy 

Cookies

updateCookieJarSource

Arguments

:: Response a

Response received from server

-> Request

Request which generated the response

-> UTCTime

Value that should be used as "now"

-> CookieJar

Current cookie jar

-> (CookieJar, Response a)

(Updated cookie jar with cookies from the Response, The response stripped of any "Set-Cookie" header)

This applies receiveSetCookie to a given Response

receiveSetCookieSource

Arguments

:: SetCookie

The SetCookie the cookie jar is receiving

-> Request

The request that originated the response that yielded the SetCookie

-> UTCTime

Value that should be used as "now"

-> Bool

Whether or not this request is coming from an "http" source (not javascript or anything like that)

-> CookieJar

Input cookie jar to modify

-> CookieJar

Updated cookie jar

This corresponds to the algorithm described in Section 5.3 "Storage Model" This function consists of calling generateCookie followed by insertCheckedCookie. Use this function if you plan to do both in a row. generateCookie and insertCheckedCookie are only provided for more fine-grained control.

generateCookieSource

Arguments

:: SetCookie

The SetCookie we are encountering

-> Request

The request that originated the response that yielded the SetCookie

-> UTCTime

Value that should be used as "now"

-> Bool

Whether or not this request is coming from an "http" source (not javascript or anything like that)

-> Maybe Cookie

The optional output cookie

Turn a SetCookie into a Cookie, if it is valid

insertCheckedCookieSource

Arguments

:: Cookie

The SetCookie the cookie jar is receiving

-> CookieJar

Input cookie jar to modify

-> Bool

Whether or not this request is coming from an "http" source (not javascript or anything like that)

-> CookieJar

Updated (or not) cookie jar

Insert a cookie created by generateCookie into the cookie jar (or not if it shouldn't be allowed in)

insertCookiesIntoRequestSource

Arguments

:: Request

The request to insert into

-> CookieJar

Current cookie jar

-> UTCTime

Value that should be used as "now"

-> (Request, CookieJar)

(Ouptut request, Updated cookie jar (last-access-time is updated))

This applies the computeCookieString to a given Request

computeCookieStringSource

Arguments

:: Request

Input request

-> CookieJar

Current cookie jar

-> UTCTime

Value that should be used as "now"

-> Bool

Whether or not this request is coming from an "http" source (not javascript or anything like that)

-> (ByteString, CookieJar)

(Contents of a "Cookie" header, Updated cookie jar (last-access-time is updated))

This corresponds to the algorithm described in Section 5.4 "The Cookie Header"

evictExpiredCookiesSource

Arguments

:: CookieJar

Input cookie jar

-> UTCTime

Value that should be used as "now"

-> CookieJar

Filtered cookie jar

This corresponds to the eviction algorithm described in Section 5.3 "Storage Model"

pathMatches :: ByteString -> ByteString -> BoolSource

This corresponds to the subcomponent algorithm entitled "Path-Match" detailed in section 5.1.4

domainMatches :: ByteString -> ByteString -> BoolSource

This corresponds to the subcomponent algorithm entitled "Domain Matching" detailed in section 5.1.3

defaultPath :: Request -> ByteStringSource

This corresponds to the subcomponent algorithm entitled "Paths" detailed in section 5.1.4