Cacher is the code snippet organizer for pro developers

User-Agent: *
Disallow: /cgi-bin/

Sitemap: https://www.example.com/sitemap.xml

User-Agent: *
Disallow: /

# Set Disallow in robots.txt if we're not on the live site
RewriteEngine On
RewriteCond %{HTTP_HOST} !^www\.example\.com$ [NC]
RewriteRule ^robots.txt nobots.txt [L]