Documentation

Specifies a crawler program that examines a data source and uses classifiers to try to determine its schema. If successful, the crawler records metadata concerning the data source in the Glue Data Catalog.

See: newCrawler smart constructor.

Create a value of Crawler with all optional fields omitted.

Use generic-lens or optics to modify other optional fields.

The following record fields are available, with the corresponding lenses provided for backwards compatibility:

$sel:classifiers:Crawler', crawler_classifiers - A list of UTF-8 strings that specify the custom classifiers that are associated with the crawler.

$sel:configuration:Crawler', crawler_configuration - Crawler configuration information. This versioned JSON string allows users to specify aspects of a crawler's behavior. For more information, see Setting crawler configuration options.

$sel:crawlElapsedTime:Crawler', crawler_crawlElapsedTime - If the crawler is running, contains the total time elapsed since the last crawl began.

$sel:crawlerSecurityConfiguration:Crawler', crawler_crawlerSecurityConfiguration - The name of the SecurityConfiguration structure to be used by this crawler.

$sel:creationTime:Crawler', crawler_creationTime - The time that the crawler was created.

$sel:databaseName:Crawler', crawler_databaseName - The name of the database in which the crawler's output is stored.

$sel:description:Crawler', crawler_description - A description of the crawler.

$sel:lakeFormationConfiguration:Crawler', crawler_lakeFormationConfiguration - Specifies whether the crawler should use Lake Formation credentials for the crawler instead of the IAM role credentials.

$sel:lastCrawl:Crawler', crawler_lastCrawl - The status of the last crawl, and potentially error information if an error occurred.

$sel:lastUpdated:Crawler', crawler_lastUpdated - The time that the crawler was last updated.

$sel:lineageConfiguration:Crawler', crawler_lineageConfiguration - A configuration that specifies whether data lineage is enabled for the crawler.

$sel:name:Crawler', crawler_name - The name of the crawler.

$sel:recrawlPolicy:Crawler', crawler_recrawlPolicy - A policy that specifies whether to crawl the entire dataset again, or to crawl only folders that were added since the last crawler run.

$sel:role':Crawler', crawler_role - The Amazon Resource Name (ARN) of an IAM role that's used to access customer resources, such as Amazon Simple Storage Service (Amazon S3) data.

$sel:schedule:Crawler', crawler_schedule - For scheduled crawlers, the schedule when the crawler runs.

$sel:schemaChangePolicy:Crawler', crawler_schemaChangePolicy - The policy that specifies update and delete behaviors for the crawler.

Crawler, crawler_state - Indicates whether the crawler is running, or whether a run is pending.

$sel:tablePrefix:Crawler', crawler_tablePrefix - The prefix added to the names of tables that are created.

$sel:targets:Crawler', crawler_targets - A collection of targets to crawl.

$sel:version:Crawler', crawler_version - The version of the crawler.

A list of UTF-8 strings that specify the custom classifiers that are associated with the crawler.

Crawler configuration information. This versioned JSON string allows users to specify aspects of a crawler's behavior. For more information, see Setting crawler configuration options.

If the crawler is running, contains the total time elapsed since the last crawl began.

The name of the SecurityConfiguration structure to be used by this crawler.

The time that the crawler was created.

The name of the database in which the crawler's output is stored.

A description of the crawler.

Specifies whether the crawler should use Lake Formation credentials for the crawler instead of the IAM role credentials.

The status of the last crawl, and potentially error information if an error occurred.

The time that the crawler was last updated.

A configuration that specifies whether data lineage is enabled for the crawler.

The name of the crawler.

A policy that specifies whether to crawl the entire dataset again, or to crawl only folders that were added since the last crawler run.

The Amazon Resource Name (ARN) of an IAM role that's used to access customer resources, such as Amazon Simple Storage Service (Amazon S3) data.

For scheduled crawlers, the schedule when the crawler runs.

The policy that specifies update and delete behaviors for the crawler.

Indicates whether the crawler is running, or whether a run is pending.

The prefix added to the names of tables that are created.

A collection of targets to crawl.

The version of the crawler.