ericrasch
7/24/2013 - 7:25 PM

Robots.txt files for sites with WordPress.

Robots.txt files for sites with WordPress.

User-agent: *

# Changing the /feed back to include the trailing slash since it otherwise blocks URLs like http://www.thesimpledollar.com/feeding-my-sweet-tooth-without-breaking-my-belly-or-breaking-the-bank/  
Disallow: /feed/
Disallow: /cgi-bin
Disallow: /wp-admin
Disallow: /wp-content/cache
Disallow: /wp-includes
Disallow: /trackback
Disallow: /xmlrpc.php
Disallow: ?wptheme=

Sitemap: http://www.yourwebsite.com/sitemap_index.xml