# Allows only major search engines and known friendly spiders # Major Search Engines and Known Friendly Spiders (allowed) User-agent: Googlebot Disallow: /cgi-bin/ Disallow: /images/ User-agent: MSNBot Disallow: /cgi-bin/ Disallow: /images/ User-agent: Slurp Disallow: /cgi-bin/ Disallow: /images/ User-agent: Teoma Disallow: /cgi-bin/ Disallow: /images/ User-agent: Gigabot Disallow: /cgi-bin/ Disallow: /images/ User-agent: Scrubby Disallow: /cgi-bin/ Disallow: /images/ # Email spambots disallowed User-agent: Robozilla Disallow: User-agent: EmailSiphon Disallow: / User-agent: EmailWolf Disallow: / User-agent: EmailCollector Disallow: / User-agent: NICErsPRO Disallow: / User-agent: CherryPicker Disallow: / User-agent: ExtractorPro Disallow: /