Setup a website
...
SEO attributes
Set crawling behaviour
1 min
your robots txt file will tell crawlers how to handle your site, while not all crawlers may interpret it in the same way for detailed information on how google interprets robots txt, please refer to this https //developers google com/search/docs/crawling indexing/robots/intro on the matter locate your robots txt in your builders' default/storefront folder as an example, you could prevent most ai models from accessing your site by adding the following to the robots txt user agent bytespider disallow / user agent ccbot disallow / user agent chatgpt user disallow / user agent claudebot disallow / user agent diffbot disallow / user agent google extended disallow / user agent gptbot disallow / user agent omgili disallow / user agent perplexitybot disallow / user agent anthropic ai disallow / user agent claude web disallow / user agent amazonbot disallow / user agent omgilibot disallow / user agent facebookbot disallow / user agent applebot disallow / user agent diffbot disallow / user agent imagesiftbot disallow / user agent youbot disallow /