# # robots.txt for http://www.edixxon.com/ and friends # # Please note: There are a lot of pages on this site, and there are # some misbehaved spiders out there that go _way_ too fast, # or your users are not interested to our pages. # If you're irresponsible, your access to the site may be blocked. # User-Agent: * Disallow: /demo/ User-Agent: * Disallow: /.search User-Agent: * Disallow: /e4search.html # Some bots are known to be trouble, particularly those designed to copy # entire sites. Please obey robots.txt. User-Agent: Zealbot Disallow: / User-Agent: MSIECrawler Disallow: / User-Agent: SiteSnagger Disallow: / User-Agent: WebStripper Disallow: / User-Agent: WebCopier Disallow: / User-Agent: Fetch Disallow: / User-Agent: Offline Explorer Disallow: / User-Agent: Teleport Disallow: / User-Agent: TeleportPro Disallow: / User-Agent: WebZIP Disallow: / User-Agent: linko Disallow: / User-Agent: HTTrack Disallow: / User-Agent: Microsoft.URL.Control Disallow: / User-Agent: Xenu Disallow: / User-Agent: larbin Disallow: / User-Agent: libwww Disallow: / User-Agent: ZyBORG Disallow: / User-Agent: Download Ninja Disallow: / # # Hits many times per second, not acceptable # http://www.nameprotect.com/botinfo.html User-Agent: NPBot Disallow: / # A capture bot, downloads gazillions of pages with no public benefit # http://www.webreaper.net/ User-Agent: WebReaper Disallow: / Sitemap: http://www.edixxon.com/sitemap_index.xml