NeonRant
7/25/2019 - 4:31 AM

robots.txt

A general sitemap showing exclusions and inclusions. Note Sitemap and Disallow.

 www.robotstxt.org/

# Allow crawling of all content
User-agent: *
Disallow: /
Disallow: /aspnet_client/
Disallow: /bin/
Disallow: /config/
Disallow: /data/
Disallow: /install/
Disallow: /masterpages/
Disallow: /python/
Disallow: /umbraco/
Disallow: /umbraco_client/
Disallow: /usercontrols/
Disallow: /xslt/
Sitemap: http://murrays.create-test.com.au/sitemapxml