pipes-s3-0.3.0.3: A simple interface for streaming data to and from Amazon S3

Safe HaskellNone
LanguageHaskell2010

Pipes.Aws.S3.Upload

Contents

Description

pipes utilities for uploading data to AWS S3 objects.

Synopsis

Documentation

These internally use the S3 multi-part upload interface to achieve streaming upload behavior.

In the case of failure one of two exceptions will be thrown,

The FailedUploadError exception carries the UploadId of the failed upload as well as the inner exception. Note that while the library makes an attempt to clean up the parts of the partial upload, there may still be remnants due to limitations in the aws library.

toS3 :: forall m a. (MonadIO m, MonadCatch m) => ChunkSize -> Bucket -> Object -> Producer ByteString m a -> m a Source #

Upload content to an S3 object.

May throw a EmptyS3UploadError if the producer fails to provide any content.

toS3' Source #

Upload content to an S3 object, explicitly specifying an Configuration, which provides credentials and logging configuration.

May throw a EmptyS3UploadError if the producer fails to provide any content.

toS3WithManager Source #

Download an object from S3 explicitly specifying an http-client Manager and aws Configuration (which provides credentials and logging configuration).

This can be more efficient when submitting many small requests as it allows re-use of the Manager across requests. Note that the Manager provided must support TLS; such a manager can be created with

import qualified Aws.Core as Aws
import qualified Network.HTTP.Client as HTTP.Client
import qualified Network.HTTP.Client.TLS as HTTP.Client.TLS

putObject :: MonadSafe m => Bucket -> Object -> Producer BS.ByteString m () -> m ()
putObject bucket object prod = do
    cfg <- liftIO baseConfiguration
    mgr <- liftIO $ newManager tlsManagerSettings
    toS3WithManager mgr cfg defServiceConfig defaultChunkSize bucket object prod

May throw a EmptyS3UploadError if the producer fails to provide any content.

Chunk size

type ChunkSize = Int Source #

To maintain healthy streaming uploads are performed in a chunked manner. This is the size of the upload chunk size in bytes. Due to S3 interface restrictions this must be at least five megabytes.

defaultChunkSize :: ChunkSize Source #

A reasonable chunk size of 10 megabytes.

Error handling

newtype UploadId Source #

An identifier representing an active upload.

Constructors

UploadId Text