Replace robots.txt

PUT

Replace the robots.txt configuration for various user agents.

Required scope | site_config:write

site_idstringRequired

Unique identifier for a Site

This endpoint expects an object.

ruleslist of objectsOptional

List of rules for user agents.

sitemapstringOptional

URL to the sitemap.

Request was successful

ruleslist of objectsOptional

List of rules for user agents.

sitemapstringOptional

URL to the sitemap.

1	curl -X PUT https://api.webflow.com/beta/sites/580e63e98c9a982ac9b8b741/robots_txt \
2	-H "Authorization: Bearer <token>" \
3	-H "Content-Type: application/json" \
4	-d '{
5	"rules": [
6	{
7	"userAgent": "googlebot",
8	"allows": [
9	"/public"
10	],
11	"disallows": [
12	"/vogon-poetry",
13	"/total-perspective-vortex"
14	]
15	}
16	],
17	"sitemap": "https://heartofgold.ship/sitemap.xml"
18	}'

1	{
2	"rules": [
3	{
4	"userAgent": "googlebot",
5	"allows": [
6	"/public"
7	],
8	"disallows": [
9	"/vogon-poetry",
10	"/total-perspective-vortex"
11	]
12	}
13	],
14	"sitemap": "https://heartofgold.ship/sitemap.xml"
15	}