EnterpriseSite Configurationrobots.txt

Delete robots.txt

Remove specific rules for a user-agent in your robots.txt file. To delete all rules for a user-agent, provide an empty rule set. This will remove the user-agent’s entry entirely, leaving it subject to your site’s default crawling behavior.

Note: Deleting a user-agent with no rules will make the user-agent’s access unrestricted unless other directives apply.

Enterprise Only
This endpoint requires an Enterprise workspace.

Required scope: site_config:write

Path parameters

site_idstringRequired

Unique identifier for a Site

Headers

AuthorizationstringRequired

Bearer authentication of the form Bearer <token>, where token is your auth token.

Request

This endpoint expects an object.
ruleslist of objectsOptional

List of rules for user agents.

sitemapstringOptional

URL to the sitemap.

Response

Request was successful

ruleslist of objectsOptional

List of rules for user agents.

sitemapstringOptional

URL to the sitemap.

Errors