Copyright	(c) 2013-2023 Brendan Hay
License	Mozilla Public License, v. 2.0.
Maintainer	Brendan Hay
Stability	auto-generated
Portability	non-portable (GHC extensions)
Safe Haskell	Safe-Inferred
Language	Haskell2010

Amazonka.Glue.Types.SparkSQL

Description

Synopsis

data SparkSQL = SparkSQL' {
- outputSchemas :: Maybe [GlueSchema]
- name :: Text
- inputs :: NonEmpty Text
- sqlQuery :: Text
- sqlAliases :: [SqlAlias]
}
newSparkSQL :: Text -> NonEmpty Text -> Text -> SparkSQL
sparkSQL_outputSchemas :: Lens' SparkSQL (Maybe [GlueSchema])
sparkSQL_name :: Lens' SparkSQL Text
sparkSQL_inputs :: Lens' SparkSQL (NonEmpty Text)
sparkSQL_sqlQuery :: Lens' SparkSQL Text
sparkSQL_sqlAliases :: Lens' SparkSQL [SqlAlias]

Documentation

data SparkSQL Source #

Specifies a transform where you enter a SQL query using Spark SQL syntax to transform the data. The output is a single DynamicFrame.

See: newSparkSQL smart constructor.

Constructors

SparkSQL'

Fields

outputSchemas :: Maybe [GlueSchema]
Specifies the data schema for the SparkSQL transform.
name :: Text
The name of the transform node.
inputs :: NonEmpty Text
The data inputs identified by their node names. You can associate a table name with each input node to use in the SQL query. The name you choose must meet the Spark SQL naming restrictions.
sqlQuery :: Text
A SQL query that must use Spark SQL syntax and return a single data set.
sqlAliases :: [SqlAlias]
A list of aliases. An alias allows you to specify what name to use in the SQL for a given input. For example, you have a datasource named "MyDataSource". If you specify From as MyDataSource, and Alias as SqlName, then in your SQL you can do:
```
select * from SqlName
```
and that gets data from MyDataSource.

Instances

Instances details

FromJSON SparkSQL Source #
Instance details Defined in Amazonka.Glue.Types.SparkSQL Methods parseJSON :: Value -> Parser SparkSQL # parseJSONList :: Value -> Parser [SparkSQL] #
ToJSON SparkSQL Source #
Instance details Defined in Amazonka.Glue.Types.SparkSQL Methods toJSON :: SparkSQL -> Value # toEncoding :: SparkSQL -> Encoding # toJSONList :: [SparkSQL] -> Value # toEncodingList :: [SparkSQL] -> Encoding #
Generic SparkSQL Source #
Instance details Defined in Amazonka.Glue.Types.SparkSQL Associated Types type Rep SparkSQL :: Type -> Type # Methods from :: SparkSQL -> Rep SparkSQL x # to :: Rep SparkSQL x -> SparkSQL #
Read SparkSQL Source #
Instance details Defined in Amazonka.Glue.Types.SparkSQL Methods readsPrec :: Int -> ReadS SparkSQL # readList :: ReadS [SparkSQL] # readPrec :: ReadPrec SparkSQL # readListPrec :: ReadPrec [SparkSQL] #
Show SparkSQL Source #
Instance details Defined in Amazonka.Glue.Types.SparkSQL Methods showsPrec :: Int -> SparkSQL -> ShowS # show :: SparkSQL -> String # showList :: [SparkSQL] -> ShowS #
NFData SparkSQL Source #
Instance details Defined in Amazonka.Glue.Types.SparkSQL Methods rnf :: SparkSQL -> () #
Eq SparkSQL Source #
Instance details Defined in Amazonka.Glue.Types.SparkSQL Methods (==) :: SparkSQL -> SparkSQL -> Bool # (/=) :: SparkSQL -> SparkSQL -> Bool #
Hashable SparkSQL Source #
Instance details Defined in Amazonka.Glue.Types.SparkSQL Methods hashWithSalt :: Int -> SparkSQL -> Int # hash :: SparkSQL -> Int #
type Rep SparkSQL Source #
Instance details Defined in Amazonka.Glue.Types.SparkSQL type Rep SparkSQL = D1 ('MetaData "SparkSQL" "Amazonka.Glue.Types.SparkSQL" "amazonka-glue-2.0-7miPWwBHdfn8N8SvbpLgE0" 'False) (C1 ('MetaCons "SparkSQL'" 'PrefixI 'True) ((S1 ('MetaSel ('Just "outputSchemas") 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedStrict) (Rec0 (Maybe [GlueSchema])) :: S1 ('MetaSel ('Just "name") 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedStrict) (Rec0 Text)) :: (S1 ('MetaSel ('Just "inputs") 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedStrict) (Rec0 (NonEmpty Text)) :: (S1 ('MetaSel ('Just "sqlQuery") 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedStrict) (Rec0 Text) :: S1 ('MetaSel ('Just "sqlAliases") 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedStrict) (Rec0 [SqlAlias])))))

newSparkSQL Source #

Arguments

:: Text	`$sel:name:SparkSQL'`
-> NonEmpty Text	`$sel:inputs:SparkSQL'`
-> Text	`$sel:sqlQuery:SparkSQL'`
-> SparkSQL

Create a value of SparkSQL with all optional fields omitted.

Use generic-lens or optics to modify other optional fields.

The following record fields are available, with the corresponding lenses provided for backwards compatibility:

$sel:outputSchemas:SparkSQL', sparkSQL_outputSchemas - Specifies the data schema for the SparkSQL transform.

$sel:name:SparkSQL', sparkSQL_name - The name of the transform node.

$sel:inputs:SparkSQL', sparkSQL_inputs - The data inputs identified by their node names. You can associate a table name with each input node to use in the SQL query. The name you choose must meet the Spark SQL naming restrictions.

$sel:sqlQuery:SparkSQL', sparkSQL_sqlQuery - A SQL query that must use Spark SQL syntax and return a single data set.

$sel:sqlAliases:SparkSQL', sparkSQL_sqlAliases - A list of aliases. An alias allows you to specify what name to use in the SQL for a given input. For example, you have a datasource named "MyDataSource". If you specify From as MyDataSource, and Alias as SqlName, then in your SQL you can do:

select * from SqlName

and that gets data from MyDataSource.

sparkSQL_outputSchemas :: Lens' SparkSQL (Maybe [GlueSchema]) Source #

Specifies the data schema for the SparkSQL transform.

sparkSQL_name :: Lens' SparkSQL Text Source #

The name of the transform node.

sparkSQL_inputs :: Lens' SparkSQL (NonEmpty Text) Source #

The data inputs identified by their node names. You can associate a table name with each input node to use in the SQL query. The name you choose must meet the Spark SQL naming restrictions.

sparkSQL_sqlQuery :: Lens' SparkSQL Text Source #

A SQL query that must use Spark SQL syntax and return a single data set.

sparkSQL_sqlAliases :: Lens' SparkSQL [SqlAlias] Source #

A list of aliases. An alias allows you to specify what name to use in the SQL for a given input. For example, you have a datasource named "MyDataSource". If you specify From as MyDataSource, and Alias as SqlName, then in your SQL you can do:

select * from SqlName

and that gets data from MyDataSource.