pyc-website

main website for pyc inc.

git clone https://9o.is/git/pyc-website.git

robots.txt

(280B)

      1 # www.robotstxt.org/
      2 # www.google.com/support/webmasters/bin/answer.py?hl=en&answer=156449
      3 
      4 User-agent: *
      5 
      6 # we want them to follow through main page
      7 Disallow: /login
      8 Disallow: /register
      9 
     10 # people don't want to see this
     11 Disallow: /help/tos
     12 Disallow: /help/privacy
     13 Disallow: /error