# no robots should visit this site further # User-agent: * # Disallow: / # Per Google's FAQs for webmasters, the below is the preferred way to disallow all crawlers from your site EXCEPT Google: User-agent: * Disallow: / User-agent: Googlebot Disallow: /html_template/ Disallow: /bin/ Disallow: /images/ Disallow: /inc/ Disallow: /internal/ Disallow: /style/ Disallow: /usercontrols/ # allow MSNBot: User-agent: MSNBot Disallow: /html_template/ Disallow: /bin/ Disallow: /images/ Disallow: /inc/ Disallow: /internal/ Disallow: /style/ Disallow: /usercontrols/