jcadima
5/20/2015 - 6:09 PM

Hiding Development and Staging Sites from Search Engines

Hiding Development and Staging Sites from Search Engines

reference:
http://www.arsnova.cc/articles/2013-10-04/hiding-development-and-staging-sites-search-engines

Restrict Access to Sites by IP Address via Apache Config

# ip restrictions
<Directory /home/example/staging.example.com/>
  order deny,allow
  # comment the following line for world access
  deny from all  
  allow from xxx.xxx.xxx.xxx    # office
  allow from zzz.zzz.zzz.zzz    # client
</Directory>




A better approach is to add X-Robots-Tag within your apache config. This allows you to add the noindex directive in the HTTP headers, rather than in the HTML, for the entire site

Restricting Indexing

# ip restrictions
<Directory /home/example/staging.example.com/>
  Header set X-Robots-Tag "noindex, nofollow"
  order deny,allow
  # comment the following line for world access
  deny from all  
  allow from xxx.xxx.xxx.xxx    # office
  allow from zzz.zzz.zzz.zzz    # client
</Directory>