cabal-version: >= 1.6 name: http-conduit-downloader version: 1.0.31 author: Vladimir Shabanov maintainer: Vladimir Shabanov homepage: https://github.com/bazqux/http-conduit-downloader license: BSD3 category: Web license-file: LICENSE build-type: Simple synopsis: HTTP downloader tailored for web-crawler needs. description: HTTP/HTTPS downloader built on top of @http-conduit@ and used in crawler. . * Handles all possible http-conduit exceptions and returns human readable error messages. . * Handles some web server bugs (returning 'deflate' data instead of 'gzip', invalid 'gzip' encoding). . * Uses OpenSSL instead of 'tls' package (since 'tls' doesn't handle all sites). . * Ignores invalid SSL sertificates. . * Receives data in 32k chunks internally to reduce memory fragmentation on many parallel downloads. . * Download timeout. . * Total download size limit. . * Returns HTTP headers for subsequent redownloads and handles 'Not modified' results. . * Can be used with external DNS resolver (e.g. concurrent-dns-cache). source-repository head type: git location: https://github.com/bazqux/http-conduit-downloader library build-depends: base == 4.*, http-conduit >= 2.3.2, http-client >= 0.5.0, connection, zlib, conduit, resourcet, http-types, data-default, bytestring, text, mtl, time >= 1.5.0, HsOpenSSL >= 0.11.2, network-uri >= 2.6, network >= 2.6 exposed-modules: Network.HTTP.Conduit.Downloader ghc-options: -O2 -Wall