User-Agent: Googlebot Disallow: # New Area of Website not ready for indexing with a search engine # User-Agent: # Disallow: /files to dissalow/ # Most Email Collector Spiders would probably just ignore this file # so just using it for search engine direction # I'll add a few below though, just for the sake of it User-agent: CherryPicker Disallow: / User-agent: EmailCollector Disallow: / User-agent: EmailSiphon Disallow: / User-agent: WebBandit Disallow: / User-agent: EmailWolf Disallow: / User-agent: ExtractorPro Disallow: / User-agent: WWW-Collector-E Disallow: / User-agent: Harvest/1.5 Disallow: / User-agent: CherryPickerSE/1.0 Disallow: / User-agent: CherryPickerElite/1.0 Disallow: /