Copyright | (c) 2013-2023 Brendan Hay |
---|---|
License | Mozilla Public License, v. 2.0. |
Maintainer | Brendan Hay |
Stability | auto-generated |
Portability | non-portable (GHC extensions) |
Safe Haskell | Safe-Inferred |
Language | Haskell2010 |
Documentation
data SeedUrlConfiguration Source #
Provides the configuration information for the seed or starting point URLs to crawl.
/When selecting websites to index, you must adhere to the Amazon Acceptable Use Policy and all other Amazon terms. Remember that you must only use Amazon Kendra Web Crawler to index your own webpages, or webpages that you have authorization to index./
See: newSeedUrlConfiguration
smart constructor.
SeedUrlConfiguration' | |
|
Instances
newSeedUrlConfiguration :: SeedUrlConfiguration Source #
Create a value of SeedUrlConfiguration
with all optional fields omitted.
Use generic-lens or optics to modify other optional fields.
The following record fields are available, with the corresponding lenses provided for backwards compatibility:
$sel:webCrawlerMode:SeedUrlConfiguration'
, seedUrlConfiguration_webCrawlerMode
- You can choose one of the following modes:
HOST_ONLY
– crawl only the website host names. For example, if the seed URL is "abc.example.com", then only URLs with host name "abc.example.com" are crawled.SUBDOMAINS
– crawl the website host names with subdomains. For example, if the seed URL is "abc.example.com", then "a.abc.example.com" and "b.abc.example.com" are also crawled.EVERYTHING
– crawl the website host names with subdomains and other domains that the webpages link to.
The default mode is set to HOST_ONLY
.
$sel:seedUrls:SeedUrlConfiguration'
, seedUrlConfiguration_seedUrls
- The list of seed or starting point URLs of the websites you want to
crawl.
The list can include a maximum of 100 seed URLs.
seedUrlConfiguration_webCrawlerMode :: Lens' SeedUrlConfiguration (Maybe WebCrawlerMode) Source #
You can choose one of the following modes:
HOST_ONLY
– crawl only the website host names. For example, if the seed URL is "abc.example.com", then only URLs with host name "abc.example.com" are crawled.SUBDOMAINS
– crawl the website host names with subdomains. For example, if the seed URL is "abc.example.com", then "a.abc.example.com" and "b.abc.example.com" are also crawled.EVERYTHING
– crawl the website host names with subdomains and other domains that the webpages link to.
The default mode is set to HOST_ONLY
.
seedUrlConfiguration_seedUrls :: Lens' SeedUrlConfiguration [Text] Source #
The list of seed or starting point URLs of the websites you want to crawl.
The list can include a maximum of 100 seed URLs.