Update robots.txt

Update the robots.txt configuration for various user agents.

This endpoint requires an Enterprise workspace.

Required scope | site_config:write

site_idstringRequired

Unique identifier for a Site

This endpoint expects an object.

ruleslist of objectsOptional

List of rules for user agents.

sitemapstringOptional

URL to the sitemap.

Request was successful

ruleslist of objectsOptional

List of rules for user agents.

sitemapstringOptional

URL to the sitemap.

1	curl -X PATCH https://api.webflow.com/beta/sites/580e63e98c9a982ac9b8b741/robots_txt \
2	-H "Authorization: Bearer <token>" \
3	-H "Content-Type: application/json" \
4	-d '{
5	"rules": [
6	{
7	"userAgent": "googlebot",
8	"allows": [
9	"/public"
10	],
11	"disallows": [
12	"/vogon-poetry",
13	"/total-perspective-vortex"
14	]
15	}
16	],
17	"sitemap": "https://heartofgold.ship/sitemap.xml"
18	}'

1	{
2	"rules": [
3	{
4	"userAgent": "googlebot",
5	"allows": [
6	"/public"
7	],
8	"disallows": [
9	"/vogon-poetry",
10	"/total-perspective-vortex"
11	]
12	}
13	],
14	"sitemap": "https://heartofgold.ship/sitemap.xml"
15	}