Copyright	(c) 2013-2023 Brendan Hay
License	Mozilla Public License, v. 2.0.
Maintainer	Brendan Hay
Stability	auto-generated
Portability	non-portable (GHC extensions)
Safe Haskell	Safe-Inferred
Language	Haskell2010

Amazonka.Glue.Types.KinesisStreamingSourceOptions

Description

Synopsis

Documentation

data KinesisStreamingSourceOptions Source #

Additional options for the Amazon Kinesis streaming data source.

See: newKinesisStreamingSourceOptions smart constructor.

Constructors

KinesisStreamingSourceOptions'

Fields

addIdleTimeBetweenReads :: Maybe Bool
Adds a time delay between two consecutive getRecords operations. The default value is "False". This option is only configurable for Glue version 2.0 and above.
avoidEmptyBatches :: Maybe Bool
Avoids creating an empty microbatch job by checking for unread data in the Kinesis data stream before the batch is started. The default value is "False".
classification :: Maybe Text
An optional classification.
delimiter :: Maybe Text
Specifies the delimiter character.
describeShardInterval :: Maybe Natural
The minimum time interval between two ListShards API calls for your script to consider resharding. The default value is 1s.
endpointUrl :: Maybe Text
The URL of the Kinesis endpoint.
idleTimeBetweenReadsInMs :: Maybe Natural
The minimum time delay between two consecutive getRecords operations, specified in ms. The default value is 1000. This option is only configurable for Glue version 2.0 and above.
maxFetchRecordsPerShard :: Maybe Natural
The maximum number of records to fetch per shard in the Kinesis data stream. The default value is 100000.
maxFetchTimeInMs :: Maybe Natural
The maximum time spent in the job executor to fetch a record from the Kinesis data stream per shard, specified in milliseconds (ms). The default value is 1000.
maxRecordPerRead :: Maybe Natural
The maximum number of records to fetch from the Kinesis data stream in each getRecords operation. The default value is 10000.
maxRetryIntervalMs :: Maybe Natural
The maximum cool-off time period (specified in ms) between two retries of a Kinesis Data Streams API call. The default value is 10000.
numRetries :: Maybe Natural
The maximum number of retries for Kinesis Data Streams API requests. The default value is 3.
retryIntervalMs :: Maybe Natural
The cool-off time period (specified in ms) before retrying the Kinesis Data Streams API call. The default value is 1000.
roleArn :: Maybe Text
The Amazon Resource Name (ARN) of the role to assume using AWS Security Token Service (AWS STS). This role must have permissions for describe or read record operations for the Kinesis data stream. You must use this parameter when accessing a data stream in a different account. Used in conjunction with "awsSTSSessionName".
roleSessionName :: Maybe Text
An identifier for the session assuming the role using AWS STS. You must use this parameter when accessing a data stream in a different account. Used in conjunction with "awsSTSRoleARN".
startingPosition :: Maybe StartingPosition
The starting position in the Kinesis data stream to read data from. The possible values are "latest", "trim_horizon", or "earliest". The default value is "latest".
streamArn :: Maybe Text
The Amazon Resource Name (ARN) of the Kinesis data stream.
streamName :: Maybe Text
The name of the Kinesis data stream.

Instances

Instances details

FromJSON KinesisStreamingSourceOptions Source #
Instance details Defined in Amazonka.Glue.Types.KinesisStreamingSourceOptions Methods parseJSON :: Value -> Parser KinesisStreamingSourceOptions # parseJSONList :: Value -> Parser [KinesisStreamingSourceOptions] #
ToJSON KinesisStreamingSourceOptions Source #
Instance details Defined in Amazonka.Glue.Types.KinesisStreamingSourceOptions Methods toJSON :: KinesisStreamingSourceOptions -> Value # toEncoding :: KinesisStreamingSourceOptions -> Encoding # toJSONList :: [KinesisStreamingSourceOptions] -> Value # toEncodingList :: [KinesisStreamingSourceOptions] -> Encoding #
Generic KinesisStreamingSourceOptions Source #
Instance details Defined in Amazonka.Glue.Types.KinesisStreamingSourceOptions Associated Types type Rep KinesisStreamingSourceOptions :: Type -> Type # Methods from :: KinesisStreamingSourceOptions -> Rep KinesisStreamingSourceOptions x # to :: Rep KinesisStreamingSourceOptions x -> KinesisStreamingSourceOptions #
Read KinesisStreamingSourceOptions Source #
Instance details Defined in Amazonka.Glue.Types.KinesisStreamingSourceOptions Methods readsPrec :: Int -> ReadS KinesisStreamingSourceOptions # readList :: ReadS [KinesisStreamingSourceOptions] # readPrec :: ReadPrec KinesisStreamingSourceOptions # readListPrec :: ReadPrec [KinesisStreamingSourceOptions] #
Show KinesisStreamingSourceOptions Source #
Instance details Defined in Amazonka.Glue.Types.KinesisStreamingSourceOptions Methods showsPrec :: Int -> KinesisStreamingSourceOptions -> ShowS # show :: KinesisStreamingSourceOptions -> String # showList :: [KinesisStreamingSourceOptions] -> ShowS #
NFData KinesisStreamingSourceOptions Source #
Instance details Defined in Amazonka.Glue.Types.KinesisStreamingSourceOptions Methods rnf :: KinesisStreamingSourceOptions -> () #
Eq KinesisStreamingSourceOptions Source #
Instance details Defined in Amazonka.Glue.Types.KinesisStreamingSourceOptions Methods (==) :: KinesisStreamingSourceOptions -> KinesisStreamingSourceOptions -> Bool # (/=) :: KinesisStreamingSourceOptions -> KinesisStreamingSourceOptions -> Bool #
Hashable KinesisStreamingSourceOptions Source #
Instance details Defined in Amazonka.Glue.Types.KinesisStreamingSourceOptions Methods hashWithSalt :: Int -> KinesisStreamingSourceOptions -> Int # hash :: KinesisStreamingSourceOptions -> Int #
type Rep KinesisStreamingSourceOptions Source #
Instance details Defined in Amazonka.Glue.Types.KinesisStreamingSourceOptions type Rep KinesisStreamingSourceOptions = D1 ('MetaData "KinesisStreamingSourceOptions" "Amazonka.Glue.Types.KinesisStreamingSourceOptions" "amazonka-glue-2.0-7miPWwBHdfn8N8SvbpLgE0" 'False) (C1 ('MetaCons "KinesisStreamingSourceOptions'" 'PrefixI 'True) ((((S1 ('MetaSel ('Just "addIdleTimeBetweenReads") 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedStrict) (Rec0 (Maybe Bool)) :: S1 ('MetaSel ('Just "avoidEmptyBatches") 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedStrict) (Rec0 (Maybe Bool))) :: (S1 ('MetaSel ('Just "classification") 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedStrict) (Rec0 (Maybe Text)) :: S1 ('MetaSel ('Just "delimiter") 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedStrict) (Rec0 (Maybe Text)))) :: ((S1 ('MetaSel ('Just "describeShardInterval") 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedStrict) (Rec0 (Maybe Natural)) :: S1 ('MetaSel ('Just "endpointUrl") 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedStrict) (Rec0 (Maybe Text))) :: (S1 ('MetaSel ('Just "idleTimeBetweenReadsInMs") 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedStrict) (Rec0 (Maybe Natural)) :: (S1 ('MetaSel ('Just "maxFetchRecordsPerShard") 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedStrict) (Rec0 (Maybe Natural)) :: S1 ('MetaSel ('Just "maxFetchTimeInMs") 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedStrict) (Rec0 (Maybe Natural)))))) :: (((S1 ('MetaSel ('Just "maxRecordPerRead") 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedStrict) (Rec0 (Maybe Natural)) :: S1 ('MetaSel ('Just "maxRetryIntervalMs") 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedStrict) (Rec0 (Maybe Natural))) :: (S1 ('MetaSel ('Just "numRetries") 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedStrict) (Rec0 (Maybe Natural)) :: S1 ('MetaSel ('Just "retryIntervalMs") 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedStrict) (Rec0 (Maybe Natural)))) :: ((S1 ('MetaSel ('Just "roleArn") 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedStrict) (Rec0 (Maybe Text)) :: S1 ('MetaSel ('Just "roleSessionName") 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedStrict) (Rec0 (Maybe Text))) :: (S1 ('MetaSel ('Just "startingPosition") 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedStrict) (Rec0 (Maybe StartingPosition)) :: (S1 ('MetaSel ('Just "streamArn") 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedStrict) (Rec0 (Maybe Text)) :*: S1 ('MetaSel ('Just "streamName") 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedStrict) (Rec0 (Maybe Text))))))))

newKinesisStreamingSourceOptions :: KinesisStreamingSourceOptions Source #

Create a value of KinesisStreamingSourceOptions with all optional fields omitted.

Use generic-lens or optics to modify other optional fields.

The following record fields are available, with the corresponding lenses provided for backwards compatibility:

$sel:addIdleTimeBetweenReads:KinesisStreamingSourceOptions', kinesisStreamingSourceOptions_addIdleTimeBetweenReads - Adds a time delay between two consecutive getRecords operations. The default value is "False". This option is only configurable for Glue version 2.0 and above.

$sel:avoidEmptyBatches:KinesisStreamingSourceOptions', kinesisStreamingSourceOptions_avoidEmptyBatches - Avoids creating an empty microbatch job by checking for unread data in the Kinesis data stream before the batch is started. The default value is "False".

$sel:classification:KinesisStreamingSourceOptions', kinesisStreamingSourceOptions_classification - An optional classification.

$sel:delimiter:KinesisStreamingSourceOptions', kinesisStreamingSourceOptions_delimiter - Specifies the delimiter character.

$sel:describeShardInterval:KinesisStreamingSourceOptions', kinesisStreamingSourceOptions_describeShardInterval - The minimum time interval between two ListShards API calls for your script to consider resharding. The default value is 1s.

$sel:endpointUrl:KinesisStreamingSourceOptions', kinesisStreamingSourceOptions_endpointUrl - The URL of the Kinesis endpoint.

$sel:idleTimeBetweenReadsInMs:KinesisStreamingSourceOptions', kinesisStreamingSourceOptions_idleTimeBetweenReadsInMs - The minimum time delay between two consecutive getRecords operations, specified in ms. The default value is 1000. This option is only configurable for Glue version 2.0 and above.

$sel:maxFetchRecordsPerShard:KinesisStreamingSourceOptions', kinesisStreamingSourceOptions_maxFetchRecordsPerShard - The maximum number of records to fetch per shard in the Kinesis data stream. The default value is 100000.

$sel:maxFetchTimeInMs:KinesisStreamingSourceOptions', kinesisStreamingSourceOptions_maxFetchTimeInMs - The maximum time spent in the job executor to fetch a record from the Kinesis data stream per shard, specified in milliseconds (ms). The default value is 1000.

$sel:maxRecordPerRead:KinesisStreamingSourceOptions', kinesisStreamingSourceOptions_maxRecordPerRead - The maximum number of records to fetch from the Kinesis data stream in each getRecords operation. The default value is 10000.

$sel:maxRetryIntervalMs:KinesisStreamingSourceOptions', kinesisStreamingSourceOptions_maxRetryIntervalMs - The maximum cool-off time period (specified in ms) between two retries of a Kinesis Data Streams API call. The default value is 10000.

$sel:numRetries:KinesisStreamingSourceOptions', kinesisStreamingSourceOptions_numRetries - The maximum number of retries for Kinesis Data Streams API requests. The default value is 3.

$sel:retryIntervalMs:KinesisStreamingSourceOptions', kinesisStreamingSourceOptions_retryIntervalMs - The cool-off time period (specified in ms) before retrying the Kinesis Data Streams API call. The default value is 1000.

$sel:roleArn:KinesisStreamingSourceOptions', kinesisStreamingSourceOptions_roleArn - The Amazon Resource Name (ARN) of the role to assume using AWS Security Token Service (AWS STS). This role must have permissions for describe or read record operations for the Kinesis data stream. You must use this parameter when accessing a data stream in a different account. Used in conjunction with "awsSTSSessionName".

$sel:roleSessionName:KinesisStreamingSourceOptions', kinesisStreamingSourceOptions_roleSessionName - An identifier for the session assuming the role using AWS STS. You must use this parameter when accessing a data stream in a different account. Used in conjunction with "awsSTSRoleARN".

$sel:startingPosition:KinesisStreamingSourceOptions', kinesisStreamingSourceOptions_startingPosition - The starting position in the Kinesis data stream to read data from. The possible values are "latest", "trim_horizon", or "earliest". The default value is "latest".

$sel:streamArn:KinesisStreamingSourceOptions', kinesisStreamingSourceOptions_streamArn - The Amazon Resource Name (ARN) of the Kinesis data stream.

$sel:streamName:KinesisStreamingSourceOptions', kinesisStreamingSourceOptions_streamName - The name of the Kinesis data stream.

kinesisStreamingSourceOptions_addIdleTimeBetweenReads :: Lens' KinesisStreamingSourceOptions (Maybe Bool) Source #

Adds a time delay between two consecutive getRecords operations. The default value is "False". This option is only configurable for Glue version 2.0 and above.

kinesisStreamingSourceOptions_avoidEmptyBatches :: Lens' KinesisStreamingSourceOptions (Maybe Bool) Source #

Avoids creating an empty microbatch job by checking for unread data in the Kinesis data stream before the batch is started. The default value is "False".

kinesisStreamingSourceOptions_classification :: Lens' KinesisStreamingSourceOptions (Maybe Text) Source #

An optional classification.

kinesisStreamingSourceOptions_delimiter :: Lens' KinesisStreamingSourceOptions (Maybe Text) Source #

Specifies the delimiter character.

kinesisStreamingSourceOptions_describeShardInterval :: Lens' KinesisStreamingSourceOptions (Maybe Natural) Source #

The minimum time interval between two ListShards API calls for your script to consider resharding. The default value is 1s.

kinesisStreamingSourceOptions_endpointUrl :: Lens' KinesisStreamingSourceOptions (Maybe Text) Source #

The URL of the Kinesis endpoint.

kinesisStreamingSourceOptions_idleTimeBetweenReadsInMs :: Lens' KinesisStreamingSourceOptions (Maybe Natural) Source #

The minimum time delay between two consecutive getRecords operations, specified in ms. The default value is 1000. This option is only configurable for Glue version 2.0 and above.

kinesisStreamingSourceOptions_maxFetchRecordsPerShard :: Lens' KinesisStreamingSourceOptions (Maybe Natural) Source #

The maximum number of records to fetch per shard in the Kinesis data stream. The default value is 100000.

kinesisStreamingSourceOptions_maxFetchTimeInMs :: Lens' KinesisStreamingSourceOptions (Maybe Natural) Source #

The maximum time spent in the job executor to fetch a record from the Kinesis data stream per shard, specified in milliseconds (ms). The default value is 1000.

kinesisStreamingSourceOptions_maxRecordPerRead :: Lens' KinesisStreamingSourceOptions (Maybe Natural) Source #

The maximum number of records to fetch from the Kinesis data stream in each getRecords operation. The default value is 10000.

kinesisStreamingSourceOptions_maxRetryIntervalMs :: Lens' KinesisStreamingSourceOptions (Maybe Natural) Source #

The maximum cool-off time period (specified in ms) between two retries of a Kinesis Data Streams API call. The default value is 10000.

kinesisStreamingSourceOptions_numRetries :: Lens' KinesisStreamingSourceOptions (Maybe Natural) Source #

The maximum number of retries for Kinesis Data Streams API requests. The default value is 3.

kinesisStreamingSourceOptions_retryIntervalMs :: Lens' KinesisStreamingSourceOptions (Maybe Natural) Source #

The cool-off time period (specified in ms) before retrying the Kinesis Data Streams API call. The default value is 1000.

kinesisStreamingSourceOptions_roleArn :: Lens' KinesisStreamingSourceOptions (Maybe Text) Source #

The Amazon Resource Name (ARN) of the role to assume using AWS Security Token Service (AWS STS). This role must have permissions for describe or read record operations for the Kinesis data stream. You must use this parameter when accessing a data stream in a different account. Used in conjunction with "awsSTSSessionName".

kinesisStreamingSourceOptions_roleSessionName :: Lens' KinesisStreamingSourceOptions (Maybe Text) Source #

An identifier for the session assuming the role using AWS STS. You must use this parameter when accessing a data stream in a different account. Used in conjunction with "awsSTSRoleARN".

kinesisStreamingSourceOptions_startingPosition :: Lens' KinesisStreamingSourceOptions (Maybe StartingPosition) Source #

The starting position in the Kinesis data stream to read data from. The possible values are "latest", "trim_horizon", or "earliest". The default value is "latest".

kinesisStreamingSourceOptions_streamArn :: Lens' KinesisStreamingSourceOptions (Maybe Text) Source #

The Amazon Resource Name (ARN) of the Kinesis data stream.

kinesisStreamingSourceOptions_streamName :: Lens' KinesisStreamingSourceOptions (Maybe Text) Source #

The name of the Kinesis data stream.