http2-client: A native HTTP2 client library.

This is a package candidate release! Here you can preview how this package release will appear once published to the main package index (which can be accomplished via the 'maintain' link below). Please note that once a package has been published to the main package index it cannot be undone! Please consult the package uploading documentation for more information.

[maintain] [Publish]

Please read the README.md at the homepage.


[Skip to Readme]

Properties

Versions 0.1.0.0, 0.1.0.1, 0.2.0.0, 0.2.0.1, 0.2.0.2, 0.3.0.0, 0.3.0.1, 0.3.0.2, 0.4.0.0, 0.5.0.0, 0.6.0.0, 0.7.0.0, 0.8.0.0, 0.8.0.1, 0.8.0.2, 0.9.0.0, 0.10.0.0, 0.10.0.0
Change log ChangeLog.md
Dependencies async (>=2.1 && <3), base (>=4.7 && <5), bytestring (>=0.10 && <1), containers (>=0.5 && <1), deepseq (>=1.4 && <2), http2 (>=1.6 && <2.1), lifted-async (>=0.10 && <0.11), lifted-base (>=0.2 && <0.3), mtl (>=2.2 && <3), network (>=2.6 && <4), stm (>=2.4 && <3), time (>=1.8 && <2), tls (>=1.4 && <2), transformers-base (>=0.4 && <0.5) [details]
License BSD-3-Clause
Copyright 2017 Lucas DiCioccio
Author Lucas DiCioccio
Maintainer lucas@dicioccio.fr
Category Web
Home page https://github.com/lucasdicioccio/http2-client
Source repo head: git clone https://github.com/lucasdicioccio/http2-client
Uploaded by LucasDiCioccio at 2020-11-06T12:28:31Z

Modules

[Index] [Quick Jump]

Downloads

Maintainer's Corner

Package maintainers

For package maintainers and hackage trustees


Readme for http2-client-0.10.0.0

[back to package description]

http2-client

An native-Haskell HTTP2 client library based on http2 and tls packages.

Hackage: https://hackage.haskell.org/package/http2-client .

General design

HTTP2 is a heavy protocol. HTTP2 features pipelining, query and responses interleaving, server-pushes, pings, stateful compression and flow-control, priorization etc. This library aims at exposing these features so that library users can integrate http2-client in a variety of applications. In short, we'd like to expose as many HTTP2 features as possible. Hence, the http2-client programming interface can feel low-level for users with expectations to get an API as simple as in HTTP1.x.

Exposing most HTTP2 primitives as a drawback: the library allows a client to behave abnormally with-respect to the HTTP2 spec. That said, we try to prevent notoriously-difficult errors such as concurrency bugs by coercing users with the programming API following Haskell's philosophy to factor-out errors at compile time. For instance, a client can send DATA frames on a stream after closing it with a RST (easy to spot). However, a multi-threaded client will not be able to interleave DATA frames with HEADERS and their CONTINUATIONs and the locking required to achieve this invariant is hidden (hard to implement).

Following this philosophy, we prefer to offer a somewhat low-level API in Network.HTTP2.Client and higher-level APIs (with a different performance trade-off) in Network.HTTP2.Client.Helpers. For instance, Network.HTTP2.Client.Helpers.waitStream will consume a whole stream in memory before returning whereas Network.HTTP2.Client users will have to take chunks one at a time. We look forward to the linear arrows extension for improving the library design.

Versioning and GHC support

We try to follow https://pvp.haskell.org/ as a.b.c.d with the caveat that if a=0 then we are still slightly unhappy with some APIs and we'll break things arbitrarily.

We aim at supporting GHC-8.x, contributions to support GHC-7.x are welcome.

Installation

This package is a standard Stack project, please also refer to Stack's documentation if you have trouble installing or using this package. Please also have a look at the Hackage Matrix CI: https://matrix.hackage.haskell.org/package/http2-client .

Usage

First, make sure you are somewhat familiar with HTTP and HTTP2 standards by reading RFCs or Wikipedia pages. If you use the library, feel free to shoot me an e-mail (cf. commits) or a tweet @lucasdicioccio .

Help and examples

Please see some literate Haskell documents in the examples/ directory. For a more involved usage, we currently provide a command-line example client: http2-client-exe which I use as a test client and you could use to test various flow-control parameters. This binary lives in a separate package at https://github.com/lucasdicioccio/http2-client-exe .

The Haddocks, at https://hackage.haskell.org/package/http2-client, should have plenty implementation details, so please have a look. Otherwise, you can ask help by creating an Issue on the bug-tracker.

Opening a stream

First, you open a (TLS-protected) connection to a server and configure the initial SETTINGS to advertise. Then you can open and consume streams. Opening streams takes a stream-definition and expresses two sequential parts. First, sending the HTTP headers, which reserves an increasing stream-ID with the server. Second, you consume a stream by sending DATA chunk or receiving DATA chunks. One thing that can prevent concurrency is if you have too many opened streams for the server. The http2-client library tracks server's max concurrency preference and will prevent you from opening too many streams.

Sending chunked data

Sent data must be chunked according to server's preferences. A function named sendData performs the chunking but this chunking could have some suboptimal overhead if you want to repeatedly call sendData with a buffer size that is not a multiple of the server's preferred chunk size.

Flow control

HTTP2 mandates a flow-control system that cannot be disabled. DATA chunks consume credit from the flow-control system. The standard defines a flow-control context per stream plus one global per-connection.

** Received DATA flow control ** In order to keep receiving data you need to periodically transfer credit to the server. One transfers credit to server by calling _updateWindow, which transfers locally-accumulated credit (you accumulate credit with addCredit). The current implementation already follows a "zero-sum" credit where received DATA is immediately consumed and re-credited. That is, if you only keep calling _updateWindow at some frequency the stream will progress. You can also _addCredit to permit receiving more DATA on a stream/connection (e.g., if you want to implement something like TCP slow-start).

** Sent DATA flow control ** A server following the HTTP2 specification strictly will kick you for sending too much data. The http2-client library allows you to be more aggressive than the server allows and you have to care for your streams. We provide an incoming flow-control context that will allow you to call _withdrawCredit to wait until some credit is available. At the time of this writing, the sendData function does not call _withdrawCredit and we provide no equivalent. Note that the chunking and flow-control mechanisms have interesting interactions in HTTP2 in a multi-threaded context. Pay attention to always take credit in the per-stream flow-control context before taking it from the global per-connection flow-control context. Otherwise, you risk starving the global per-connection flow-control with no guarantee that you'll be allowed to send a DATA frame.

Settings changes

The HTTP2 RFC acknowledges the inherent race conditions that may occur when changing SETTINGS. The http2-client library should be rather permissive and accept rather than reject frames caught violating inconsistent settings once client settings are made stricter. Conversely, the http2-client library tries to enforce server-SETTINGS strictly before ACKnowledging the setting changes. This configuration can lead to problems if the server send more-permissive SETTINGS (e.g., allowing a large default window size -> which recredits all streams) but if the server applies this change locally only after receiving the client ACK. One way to be double-sure the http2-client library is always strict would be to apply settings changes in two steps: settings that move in the "stricter direction" (e.g., fewer concurrency, smaller initial window) should be applied before ACK-ing the SETTING frame. Meanwhile settings that move in the "looser direction" (e.g., more concurrency) should be applied after ACK-ing the SETTINGS frame.

The current design apply SETTINGS:

Fortunately, changing settings mid-stream is probably a rare behavior and the default SETTINGS are large enough to avoid creating fatal errors before sending/receiving the initial SETTINGS frames.

Things that are hardcoded

A number of HTTP2 features are currently hardcoded:

Contributing

Contributions are welcome. As I start integrating external contribution I plan to follow the following procedure:

Please pay attention to the following:

General mindset to have during code-reviews:

Bugtracker

Most of the following points have their own issues on the issue tracker at GitHub: https://github.com/lucasdicioccio/http2-client/issues .

Things that will likely change the API

I think the fundamentals are right but the following needs tweaking:

Support of the HTTP2 standard

The current implementation follows the HTTP2 standard except for the following: