User-agent: * Crawl-delay: 10 Disallow: /admin/ Disallow: /edit/ Disallow: /editor/ Disallow: /fck/ Disallow: /fckedit/ Disallow: /fckeditor/ Disallow: /include/ Disallow: /manage/ Disallow: /system/ Disallow: /www/idcplg* Disallow: /*~ Disallow: /CofM/* Disallow: /*.xml$ User-agent: Google Disallow: User-agent: Bing Disallow: User-agent: yahoo Disallow: User-agent: msn Disallow: User-agent: 008 Disallow: / User-agent: 80legs Disallow: / User-agent: AddThis.com Disallow: / User-agent: AddThis.com robot tech.support@clearspring.com Disallow: / User-agent: ahrefs Disallow: / User-agent: ia_archiver (+http://www.alexa.com/site/help/webmasters; crawler@alexa.com) Disallow: / User-agent: BaiduSpider Disallow: / User-agent: dotbot Disallow: / User-agent: Feedfetcher-Google; (+http://www.google.com/feedfetcher.html; 1 subscribers; feed-id=14975829261283866692) Disallow: / User-agent: Mail.RU_Bot/2.0; +http://go.mail.ru/help/robots Disallow: / User-agent: seckicks Disallow: / User-agent: Sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm#07) Disallow: / User-agent: turnitin Disallow: / User-agent: warebay Disallow: / User-agent: Wotbox/2.01 (+http://www.wotbox.com/bot/) Disallow: / User-agent: wowrack Disallow: / User-agent: yandex Disallow: / User-agent: YoudaoBot Disallow: / User-agent: XoviBot Disallow: /