RobotsConfig

Properties

host?

optional host: string | boolean

Default

false

Description

[ Optional ] Some crawlers(Yandex) support and only accept domain names.

Example

integrations:[
 robots({
   host: siteUrl.replace(/^https?://|:\d+/g, "")
 })
]

Defined in

index.ts:32

---

policy?

optional policy: PolicyOptions[]

Description

[ Optional ] List of policy rules.

Default

policy:[
 {
   userAgent: "*",
   allow: "/"
 }
]

For more help, refer to SYNTAX by Yandex.

Defined in

index.ts:60

---

sitemap?

optional sitemap: string | boolean | string[]

Description

[ Optional, zero or more per file ] The location of a sitemap for this website.

Example

sitemap: [
 "https://example.com/sitemap.xml",
 "https://www.example.com/sitemap.xml"
]

The value of the SITEMAP field is case-sensitive.

Defined in

index.ts:45