Update robots.txt

PATCH

Update the robots.txt configuration for various user agents.

Required scope | site_config:write

site_idstringRequired

Unique identifier for a Site

This endpoint expects an object.

ruleslist of objectsOptional

List of rules for user agents.

sitemapstringOptional

URL to the sitemap.

Request was successful

ruleslist of objectsOptional

List of rules for user agents.

sitemapstringOptional

URL to the sitemap.

1	curl -X PATCH https://api.webflow.com/beta/sites/580e63e98c9a982ac9b8b741/robots_txt \
2	-H "Authorization: Bearer <token>" \
3	-H "Content-Type: application/json" \
4	-d '{
5	"rules": [
6	{
7	"userAgent": "googlebot",
8	"allows": [
9	"/public"
10	],
11	"disallows": [
12	"/vogon-poetry",
13	"/total-perspective-vortex"
14	]
15	}
16	],
17	"sitemap": "https://heartofgold.ship/sitemap.xml"
18	}'

1	{
2	"rules": [
3	{
4	"userAgent": "googlebot",
5	"allows": [
6	"/public"
7	],
8	"disallows": [
9	"/vogon-poetry",
10	"/total-perspective-vortex"
11	]
12	}
13	],
14	"sitemap": "https://heartofgold.ship/sitemap.xml"
15	}