transmitstudio
3/15/2018 - 5:52 AM

robots.txt to allow Twitter cards

Because Twitter is a pain in the ass. Use Twitter validator to check after setting it up: https://cards-dev.twitter.com/validator

Via: http://www.cesarserna.com/fixed-twitter-card-images-on-wordpress/

twitter-robots.txt
content_copyfile_download
User-agent: Twitterbot
Disallow: *
Allow: /wp-content/uploads/*

User-agent: *
Disallow: /wp-admin*
Disallow: /wp-login.php
Disallow: /wp-content*
Disallow: /*/feed
Disallow: /*/trackback
Disallow: /*?
Disallow: /*.js$
Disallow: /*.inc$
Disallow: /*.css$
Disallow: /category/
Disallow: /page/
Disallow: /pages/
Disallow: /feed/
Disallow: /feed
Disallow: /tag/
Disallow: /archives/