Copyright | (c) 2013-2023 Brendan Hay |
---|---|
License | Mozilla Public License, v. 2.0. |
Maintainer | Brendan Hay |
Stability | auto-generated |
Portability | non-portable (GHC extensions) |
Safe Haskell | Safe-Inferred |
Language | Haskell2010 |
Synopsis
- data SparkSQL = SparkSQL' {
- outputSchemas :: Maybe [GlueSchema]
- name :: Text
- inputs :: NonEmpty Text
- sqlQuery :: Text
- sqlAliases :: [SqlAlias]
- newSparkSQL :: Text -> NonEmpty Text -> Text -> SparkSQL
- sparkSQL_outputSchemas :: Lens' SparkSQL (Maybe [GlueSchema])
- sparkSQL_name :: Lens' SparkSQL Text
- sparkSQL_inputs :: Lens' SparkSQL (NonEmpty Text)
- sparkSQL_sqlQuery :: Lens' SparkSQL Text
- sparkSQL_sqlAliases :: Lens' SparkSQL [SqlAlias]
Documentation
Specifies a transform where you enter a SQL query using Spark SQL syntax
to transform the data. The output is a single DynamicFrame
.
See: newSparkSQL
smart constructor.
SparkSQL' | |
|
Instances
Create a value of SparkSQL
with all optional fields omitted.
Use generic-lens or optics to modify other optional fields.
The following record fields are available, with the corresponding lenses provided for backwards compatibility:
$sel:outputSchemas:SparkSQL'
, sparkSQL_outputSchemas
- Specifies the data schema for the SparkSQL transform.
$sel:name:SparkSQL'
, sparkSQL_name
- The name of the transform node.
$sel:inputs:SparkSQL'
, sparkSQL_inputs
- The data inputs identified by their node names. You can associate a
table name with each input node to use in the SQL query. The name you
choose must meet the Spark SQL naming restrictions.
$sel:sqlQuery:SparkSQL'
, sparkSQL_sqlQuery
- A SQL query that must use Spark SQL syntax and return a single data set.
$sel:sqlAliases:SparkSQL'
, sparkSQL_sqlAliases
- A list of aliases. An alias allows you to specify what name to use in
the SQL for a given input. For example, you have a datasource named
"MyDataSource". If you specify From
as MyDataSource, and Alias
as
SqlName, then in your SQL you can do:
select * from SqlName
and that gets data from MyDataSource.
sparkSQL_outputSchemas :: Lens' SparkSQL (Maybe [GlueSchema]) Source #
Specifies the data schema for the SparkSQL transform.
sparkSQL_inputs :: Lens' SparkSQL (NonEmpty Text) Source #
The data inputs identified by their node names. You can associate a table name with each input node to use in the SQL query. The name you choose must meet the Spark SQL naming restrictions.
sparkSQL_sqlQuery :: Lens' SparkSQL Text Source #
A SQL query that must use Spark SQL syntax and return a single data set.
sparkSQL_sqlAliases :: Lens' SparkSQL [SqlAlias] Source #
A list of aliases. An alias allows you to specify what name to use in
the SQL for a given input. For example, you have a datasource named
"MyDataSource". If you specify From
as MyDataSource, and Alias
as
SqlName, then in your SQL you can do:
select * from SqlName
and that gets data from MyDataSource.