| imho.ws |
![]() |
|
|
|
# 21 |
|
МОД-Оператор ЭВМ
Регистрация: 18.04.2002
Адрес: Питер
Сообщения: 4 343
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
Все-таки я не вижу смысла выискивать роботов. Всех не найдешь, а не всех и искать неинтересно. Ведь цель скорей всего убедиться что робот прошел по сайту, а если скрипт не дает 100% гарантии обнаружения, то сомнения не чуть не меньше, что были до скрипта. Впрочем, если интересно:
Хороший список роботов: http://www.robotstxt.org/wc/active/html/index.html Гугл бот: http://www.google.com/bot.html Яндекс бот: http://www.yandex.ru/info/webmaster2.html Рамблер бот: http://www.rambler.ru/doc/robots.shtml |
|
|
|
|
# 22 |
|
Guest
Сообщения: n/a
|
Да, видел я этот список, ток вот они там все исключительно иностранные.... и, например мне, толку от него нету, у меня на сайте находяться те самые популярные 10 роботов, и им соответственно выдается их имя, вместо "Гость". Чтобы потом в поисковиках не было видно такого: [... Вы = Гость ]
))
|
|
|
# 23 |
|
МОД-Оператор ЭВМ
Регистрация: 18.04.2002
Адрес: Питер
Сообщения: 4 343
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
-------------------------------------------------------------------------------
Yandex 213.180.194.* - *.yandex.ru Yandex/1.01.001 (compatible; Win16; I) Yandex/1.03.003 (compatible; Win16; D) Yandex/1.03.000 (compatible; Win16; M) ------------------------------------------------------------------------------- Rambler 81.19.66.* StackRambler StackRambler/1.0 StackRambler/1.4 StackRambler/2.0 ------------------------------------------------------------------------------- 209.185.253.167 - c2.googlebot.com 209.185.253.168 - c3.googlebot.com 209.185.253.169 - c1.googlebot.com 209.185.253.170 - c4.googlebot.com 209.185.253.171 - c5.googlebot.com 209.185.253.172 - c6.googlebot.com 209.185.253.173 - c7.googlebot.com 209.185.253.174 - c8.googlebot.com 209.185.253.175 - c9.googlebot.com 209.185.253.176 - c10.googlebot.com 209.185.253.177 - c11.googlebot.com 209.185.253.178 - c12.googlebot.com 209.185.253.179 - c13.googlebot.com 209.185.253.180 - c14.googlebot.com 209.185.253.181 - c15.googlebot.com 209.185.253.182 - c16.googlebot.com 209.185.253.183 - c17.googlebot.com 209.185.253.184 - c18.googlebot.com 209.185.253.185 - c19.googlebot.com 209.185.253.186 - c20.googlebot.com 209.185.253.187 - c21.googlebot.com 209.185.253.188 - c22.googlebot.com 209.185.108.138 - a1.google.com 209.185.108.139 - a2.google.com 209.185.108.140 - a3.google.com 209.185.108.141 - a4.google.com 209.185.108.142 - a5.google.com 209.185.108.143 - a6.google.com 209.185.108.144 - a7.google.com 209.185.108.145 - a8.google.com 209.185.108.146 - a9.google.com 209.185.108.147 - a10.google.com 209.185.108.148 - a11.google.com 209.185.108.149 - a12.google.com 209.185.108.150 - a13.google.com 209.185.108.151 - a14.google.com 209.185.108.152 - a15.google.com 209.185.108.153 - a16.google.com 209.185.108.154 - a17.google.com 209.185.108.155 - a18.google.com 209.185.108.156 - c2.google.com 209.185.108.157 - c3.google.com 209.185.108.158 - c1.google.com 209.185.108.159 - a19.google.com 209.185.108.160 - a20.google.com 209.185.108.161 - c4.google.com 209.185.108.162 - a21.google.com 209.185.108.163 - a22.google.com 209.185.108.164 - a23.google.com 209.185.108.165 - a24.google.com 209.185.108.134 - ns.google.com 209.185.108.135 - ns2.google.com 64.209.181.52 - crawler1.googlebot.com 64.208.33.33 - crawl1.googlebot.com 64.209.181.53 - crawler2.googlebot.com Googlebot/1.0 (googlebot@googlebot.com http://googlebot.com/) Googlebot/2.0 (googlebot@googlebot.com http://googlebot.com/) Googlebot/2.0 beta (googlebot(at)googlebot.com) Googlebot/2.1 (+http://googlebot.com/bot.html) Googlebot/2.0 (+http://googlebot.com/bot.html) Googlebot/1.0 (googlebot(at)googlebot.com) Googlebot/2.1w (+http://googlebot.com/bot.html) Googlebot-w/2.1 (+http://googlebot.com/bot.html) Googlebot/1.0 ------------------------------------------------------------------------------- Lycos 166.48.225.254 - lycosinc.NorthRoyalton.cw.net 206.79.171.157 - sjc-fe13-1.sjc.lycos.com 206.79.171.18 - sjc-fe2-2.sjc.lycos.com 206.79.171.19 - sjc-fe3-1.sjc.lycos.com 206.79.171.20 - sjc-fe3-2.sjc.lycos.com 206.79.171.21 - sjc-fe4-1.sjc.lycos.com 206.79.171.22 - sjc-fe4-2.sjc.lycos.com 206.79.171.23 - sjc-fe5-1.sjc.lycos.com 206.79.171.24 - sjc-fe5-2.sjc.lycos.com 206.79.171.25 - sjc-accipiter1f.sjc.lycos.com 206.79.171.81 - sjc-fe8-1.sjc.lycos.com 206.79.171.89 - sjc-fe10-1.sjc.lycos.com 206.79.171.97 - sjc-fe12-1.sjc.lycos.com 206.79.171.85 - sjc-fe9-1.sjc.lycos.com 207.77.90.136 - www-jp.lycos.com 207.77.90.14 - icreport.lycos.com 207.77.90.183 - lycosidae.lycos.com 207.77.90.185 - lycosidae.lycos.com 207.77.90.186 - lycosidae.lycos.com 207.77.90.187 - lycosidae.lycos.com 207.77.90.20 - a2z.lycos.com 207.77.91.184 - spider4.srv.pgh.lycos.com 208.146.26.19 - res5.eng.pgh.lycos.com 208.146.26.233 - zebra.eng.pgh.lycos.com 208.146.27.123 - srd09.ma.lycos.com 208.146.27.124 - srd10.ma.lycos.com 208.146.27.94 - srd06.ma.lycos.com 208.146.27.95 - srd07.ma.lycos.com 208.146.27.96 - srd08.ma.lycos.com 209.67.228.109 - bos-fe8-1.bos.lycos.com 209.67.228.15 - bos-fe1-1.bos.lycos.com 209.67.228.159 - bos-fe6-4.bos.lycos.com 209.67.228.166 - bos-fe9-1.bos.lycos.com 209.67.228.17 - bos-fe2-1.bos.lycos.com 209.67.228.174 - bos-fe11-1.bos.lycos.com 209.67.228.178 - bos-fe12-1.bos.lycos.com 209.67.228.182 - bos-fe13-1.bos.lycos.com 209.67.228.19 - bos-fe3-1.bos.lycos.com 209.67.228.21 - bos-fe4-1.bos.lycos.com 209.67.228.23 - bos-fe5-1.bos.lycos.com 209.67.228.42 - bos-fe6-1.bos.lycos.com 209.67.229.100 - bos-spider9b.bos.lycos.com 209.67.229.101 - bos-spider10b.bos.lycos.com 209.67.229.102 - bos-spider11b.bos.lycos.com 209.67.229.137 - bos-catalog1.bos.lycos.com 209.67.229.138 - bos-spider1.bos.lycos.com 209.67.229.139 - bos-spider2.bos.lycos.com 209.67.229.140 - bos-spider3.bos.lycos.com 209.67.229.141 - bos-spider4.bos.lycos.com 209.67.229.142 - bos-spider5.bos.lycos.com 209.67.229.143 - bos-spider7.bos.lycos.com 209.67.229.143 - bos-spider6.bos.lycos.com 209.67.229.144 - bos-spider7.bos.lycos.com 209.67.229.62 - bos-catalog4.bos.lycos.com 209.67.229.85 - bos-catalog2.bos.lycos.com 216.35.194.183 216.35.194.184 216.35.194.185 216.35.194.186 216.35.194.187 216.35.194.188 216.35.194.189 216.35.194.190 Lycos_Spider_(T-Rex) Mozilla/4.0 (compatible; MSIE 5.0; Windows 98;Lycos_Spider_(T-Rex) ; Lycos_Spider_(T-Rex) ) Mozilla/4.0 (compatible; MSIE 5.0; Windows 98;Lycos_Spider_Beta2(T-Rex) ; Lycos_Spider_Beta2(T-Rex) ) ------------------------------------------------------------------------------- FAST (Lycos) 209.67.247.(129-255) - 209.202.148.(0-128) - 209.202.148.13 - crawler3.bos2.fast-search.net 209.202.148.18 - crawler8.bos2.fast-search.net 209.202.148.23 - crawler13.bos2.fast-search.net 209.202.148.25 - crawler15.bos2.fast-search.net 209.202.148.39 - crawler29.bos2.fast-search.net 209.202.148.51 - 209.202.148.53 - 209.202.148.63 - 209.202.148.65 - 209.202.148.74 - 209.67.247.153 - ix1.fast-search.net 209.67.247.158 - ix6.fast-search.net 209.67.247.161 - hurst.fastwebmedia.co.uk 209.67.247.200 - ix8.fast-search.net 209.67.247.202 - ix10.fast-search.net 209.67.247.205 - ix13.fast-search.net 209.67.247.206 - ix14.fast-search.net 209.67.247.207 - ix15.fast-search.net 209.67.247.208 - ix16.fast-search.net 209.67.247.230 - rdcrawler1.uswal.alltheweb.com 209.67.247.231 - rdcrawler2.uswal.alltheweb.com 209.67.247.233 - rdcrawler4.uswal.alltheweb.com 209.67.247.234 - elsevier.alltheweb.com 207.138.42.105 - fcrawler.looksmart.com 209.202.148.1 - exodus-gw.bos2.fastsearch.net 209.202.148.2 209.202.148.3 209.202.148.4 209.202.148.5 209.202.148.6 209.202.148.7 209.202.148.8 209.202.148.9 - admin1.bos2.fastsearch.net 209.202.148.10 - admin2.bos2.fastsearch.net 209.202.148.11 - crawler1.bos2.fast-search.net 209.202.148.12 - crawler2.bos2.fast-search.net 209.202.148.13 - crawler3.bos2.fast-search.net 209.202.148.14 - crawler4.bos2.fast-search.net 209.202.148.15 - crawler5.bos2.fast-search.net 209.202.148.16 - crawler6.bos2.fast-search.net 209.202.148.17 - crawler7.bos2.fast-search.net 209.202.148.18 - crawler8.bos2.fast-search.net 209.202.148.19 - crawler9.bos2.fast-search.net 209.202.148.20 - crawler10.bos2.fast-search.net 209.202.148.21 - crawler11.bos2.fast-search.net 209.202.148.22 - crawler12.bos2.fast-search.net 209.202.148.23 - crawler13.bos2.fast-search.net 209.202.148.24 - crawler14.bos2.fast-search.net 209.202.148.25 - crawler15.bos2.fast-search.net 209.202.148.26 - crawler16.bos2.fast-search.net 209.202.148.27 - crawler17.bos2.fast-search.net 209.202.148.28 - crawler18.bos2.fast-search.net 209.202.148.29 - crawler19.bos2.fast-search.net 209.202.148.30 - crawler20.bos2.fast-search.net 209.202.148.31 - crawler21.bos2.fast-search.net 209.202.148.32 - crawler22.bos2.fast-search.net 209.202.148.33 - crawler23.bos2.fast-search.net 209.202.148.34 - crawler24.bos2.fast-search.net 209.202.148.35 - crawler25.bos2.fast-search.net 209.202.148.36 - crawler26.bos2.fast-search.net 209.202.148.37 - crawler27.bos2.fast-search.net 209.202.148.38 - crawler28.bos2.fast-search.net 209.202.148.39 - crawler29.bos2.fast-search.net 209.202.148.40 - crawler30.bos2.fast-search.net 209.202.148.41 - crawler31.bos2.fast-search.net 209.202.148.42 - crawler32.bos2.fast-search.net 209.202.148.43 209.202.148.44 209.202.148.45 mp3.bos2.fastsearch.net 209.202.148.46 mp3-1.bos2.fastsearch.net 209.202.148.47 mp3-2.bos2.fastsearch.net 209.202.148.48 209.202.148.49 209.202.148.50 - readmin.bos2.fastsearch.net 209.202.148.51 - recrawler1.bos2.fastsearch.net 209.202.148.52 - recrawler2.bos2.fastsearch.net 209.202.148.53 209.202.148.54 - reds.bos2.fastsearch.net 209.202.148.55 - laptop.bos2.fastsearch.net 209.202.148.56 - windoze.bos2.fastsearch.net 209.202.148.57 - recrawler3.bos2.fastsearch.net 209.202.148.58 - recrawler4.bos2.fastsearch.net 209.202.148.59 - reap03.bos2.fastsearch.net 209.202.148.60 209.202.148.61 - mobile.bos2.fastsearch.net 209.202.148.62 209.202.148.63 - mobile03.bos2.fastsearch.net 209.202.148.64 - reds01.bos2.fastsearch.net 209.202.148.65 - reds02.bos2.fastsearch.net 209.202.148.66 209.202.148.67 209.202.148.68 - corpsearch.bos2.fastsearch.net 209.202.148.69 - ibm.bos2.fastsearch.net 209.202.148.70 - ibmcrawler1.bos2.fastsearch.net 209.202.148.71 - ibmcrawler2.bos2.fastsearch.net 209.202.148.72 - sun1.bos2.alltheweb.com 209.202.148.73 - sun2.bos2.alltheweb.com 209.202.148.74 - sun3.bos2.alltheweb.com 209.202.148.75 - sun4.bos2.alltheweb.com 209.202.148.76 - sun5.bos2.alltheweb.com 209.202.148.77 - reap.bos2.fastsearch.net 209.202.148.78 - mobile04.bos2.fastsearch.net 209.202.148.79 - ibm5.bos2.fastsearch.net 209.202.148.80 - webmediamail.bos2.fastsearch.net 209.202.148.81 - webmediavideo.bos2.fastsearch.net 209.202.148.82 - webmediadb01.bos2.fastsearch.net 209.202.148.83 - webmediadb02.bos2.fastsearch.net 209.202.148.84 - webmediaweb01.bos2.fastsearch.net 209.202.148.85 - webmediaweb02.bos2.fastsearch.net 209.202.148.86 - webmediaweb03.bos2.fastsearch.net 209.202.148.87 - webmediaweb.bos2.fastsearch.net 209.202.148.88 209.202.148.89 209.202.148.90 - lycos-gw.bos2.fastsearch.net 209.202.148.91 - ns1.bos2.fastsearch.net 209.202.148.92 - ns2.bos2.fastsearch.net 209.202.148.93 209.202.148.94 209.202.148.95 209.202.148.96 209.202.148.97 209.202.148.98 209.202.148.99 - www.alltheweb.com 209.202.148.100 209.202.148.101 209.202.148.102 209.202.148.103 209.202.148.104 209.202.148.105 209.202.148.106 209.202.148.107 209.202.148.108 209.202.148.109 209.202.148.110 209.202.148.111 209.202.148.112 209.202.148.113 209.202.148.114 209.202.148.115 209.202.148.116 209.202.148.117 209.202.148.118 209.202.148.119 209.202.148.120 209.202.148.121 209.202.148.122 209.202.148.123 209.202.148.124 209.202.148.125 209.202.148.126 209.202.148.127 209.202.148.128 209.202.148.(129-255) 209.67.247.(1-128) 209.67.247.129 209.67.247.130 209.67.247.131 209.67.247.132 209.67.247.133 209.67.247.134 209.67.247.135 - www.alltheweb.com 209.67.247.136 209.67.247.137 209.67.247.138 209.67.247.139 209.67.247.140 - wap-test.alltheweb.com 209.67.247.141 - syncro.alltheweb.com 209.67.247.142 209.67.247.143 209.67.247.144 - as1.alltheweb.com 209.67.247.145 - as2.alltheweb.com 209.67.247.146 - as3.alltheweb.com 209.67.247.147 - as4.alltheweb.com 209.67.247.148 - as5.alltheweb.com 209.67.247.149 - devzero.uswal.alltheweb.com 209.67.247.150 209.67.247.151 - dispatch2.uswal.alltheweb.com 209.67.247.152 - dispatch3.uswal.alltheweb.com 209.67.247.153 - ix1.fast-search.net 209.67.247.154 - ix2.fast-search.net 209.67.247.155 - ix3.fast-search.net 209.67.247.156 - ix4.fast-search.net 209.67.247.157 - ix5.fast-search.net 209.67.247.158 - ix6.fast-search.net 209.67.247.159 - ix7.fast-search.net 209.67.247.160 - moore.fastwebmedia.co.uk 209.67.247.161 - hurst.fastwebmedia.co.uk 209.67.247.162 - charlton.fastwebmedia.co.uk 209.67.247.163 - banks.fastwebmedia.co.uk 209.67.247.164 - dispatch40.uswal.alltheweb.com 209.67.247.165 - dispatch41.uswal.alltheweb.com 209.67.247.166 - dispatch42.uswal.alltheweb.com 209.67.247.167 - dispatch43.uswal.alltheweb.com 209.67.247.168 209.67.247.169 209.67.247.170 - mp3-1.alltheweb.com 209.67.247.171 - listeningroom.lycos.com 209.67.247.172 209.67.247.173 209.67.247.174 209.67.247.175 209.67.247.176 209.67.247.177 209.67.247.178 209.67.247.179 209.67.247.180 - rdsr1r3.uswal.alltheweb.com 209.67.247.181 - ns6.askfast.com 209.67.247.182 - rdsr2r3.uswal.alltheweb.com 209.67.247.183 - rdsr3r3.uswal.alltheweb.com 209.67.247.184 - rdsr4r3.uswal.alltheweb.com 209.67.247.185 - rdsr5r3.uswal.alltheweb.com 209.67.247.186 - fap1.alltheweb.com 209.67.247.187 - fap2.alltheweb.com 209.67.247.188 209.67.247.189 209.67.247.190 - dispatch1.uswal.alltheweb.com 209.67.247.191 - rdsr1r1.uswal.alltheweb.com 209.67.247.192 - rdsr2r1.uswal.alltheweb.com 209.67.247.193 - rdsr3r1.uswal.alltheweb.com 209.67.247.194 - rdsr4r1.uswal.alltheweb.com 209.67.247.195 - rdsr5r1.uswal.alltheweb.com 209.67.247.196 - rdsr6r1.uswal.alltheweb.com 209.67.247.197 - rdsr2r2.uswal.alltheweb.com 209.67.247.198 - rdsr3r2.uswal.alltheweb.com 209.67.247.199 - rdsr4r2.uswal.alltheweb.com 209.67.247.200 - ix8.fast-search.net 209.67.247.201 - ix9.fast-search.net 209.67.247.202 - ix10.fast-search.net 209.67.247.203 - ix11.fast-search.net 209.67.247.204 - ix12.fast-search.net 209.67.247.205 - ix13.fast-search.net 209.67.247.206 - ix14.fast-search.net 209.67.247.207 - ix15.fast-search.net 209.67.247.208 - ix16.fast-search.net 209.67.247.209 - rdsr5r2.uswal.alltheweb.com 209.67.247.210 - rdsr1r2.uswal.alltheweb.com 209.67.247.211 - dispatch5.uswal.alltheweb.com 209.67.247.212 - dispatch6.uswal.alltheweb.com 209.67.247.213 - dispatch7.uswal.alltheweb.com 209.67.247.214 - dispatch8.uswal.alltheweb.com 209.67.247.215 - dispatch9.uswal.alltheweb.com 209.67.247.216 - dispatch10.uswal.alltheweb.com 209.67.247.217 - dispatch11.uswal.alltheweb.com 209.67.247.218 - dispatch12.uswal.alltheweb.com 209.67.247.219 - dispatch13.uswal.alltheweb.com 209.67.247.220 - dispatch30.uswal.alltheweb.com 209.67.247.221 - dispatch31.uswal.alltheweb.com 209.67.247.222 - dispatch32.uswal.alltheweb.com 209.67.247.223 - dispatch33.uswal.alltheweb.com 209.67.247.224 - dispatch34.uswal.alltheweb.com 209.67.247.225 - dispatch35.uswal.alltheweb.com 209.67.247.226 - dispatch36.uswal.alltheweb.com 209.67.247.227 - dispatch37.uswal.alltheweb.com 209.67.247.228 209.67.247.229 209.67.247.230 - rdcrawler1.uswal.alltheweb.com 209.67.247.231 - rdcrawler2.uswal.alltheweb.com 209.67.247.232 - rdcrawler3.uswal.alltheweb.com 209.67.247.233 - rdcrawler4.uswal.alltheweb.com 209.67.247.234 - elsevier.alltheweb.com 209.67.247.235 - corpdemo.alltheweb.com 209.67.247.236 209.67.247.237 - ibm1.alltheweb.com 209.67.247.238 - ibm2.alltheweb.com 209.67.247.239 - logger.uswal.alltheweb.com 209.67.247.240 - wapsearch.alltheweb.com 209.67.247.241 - elsevier2.alltheweb.com 209.67.247.242 209.67.247.243 209.67.247.244 209.67.247.245 209.67.247.246 209.67.247.247 209.67.247.248 209.67.247.249 - license.alltheweb.com 209.67.247.250 - www.fast.no 209.67.247.251 - dispatch38.uswal.alltheweb.com 209.67.247.252 - dispatch39.uswal.alltheweb.com 209.67.247.253 - lycos-new.alltheweb.com 209.67.247.254 209.67.247.255 FAST-WebCrawler/2.1.pre.2000-04-14.1 (ashen@looksmart.net) FAST-WebCrawler/2.1-pre2 (ashen@looksmart.net) FAST-WebCrawler/2.1.prealpha.2000-04-07.1 (ashen@looksmart.net) fastlwspider/1.0 FAST-WebCrawler/2.1.pre.2000-04-18.1 (crawler@fast.no; http://www.fast.no/faq/faqfastwebsea...ebcrawler.html) FAST-WebCrawler/2.0.9 (crawler@fast.no; http://www.fast.no/faq/faqfastwebsea...ebcrawler.html) FAST-SoccerCrawler/2.2-pre-cvs (oyvinda@fast.no; http://www.fast.no/faq/faqfastwebsea...ebcrawler.html) FAST-WebCrawler/2.0.10 (crawler@fast.no; http://www.fast.no/faq/faqfastwebsea...ebcrawler.html) FAST-WebCrawler/2.1-pre4 (crawler@fast.no; http://www.fast.no/faq/faqfastwebsea...ebcrawler.html) FAST-WebCrawler/2.1-pre5 (crawler@fast.no; http://www.fast.no/faq/faqfastwebsea...ebcrawler.html) FAST-WebCrawler/2.1-pre6 (crawler@fast.no; http://www.fast.no/faq/faqfastwebsea...ebcrawler.html) FAST-WebCrawler/2.1-pre7 (crawler@fast.no; http://www.fast.no/faq/faqfastwebsea...ebcrawler.html) FAST-WebCrawler/2.2-pre1 (crawler@fast.no; http://www.fast.no/faq/faqfastwebsea...ebcrawler.html) FAST-WebCrawler/2.2-pre2 (crawler@fast.no; http://www.fast.no/faq/faqfastwebsea...ebcrawler.html) FAST-WebCrawler/2.2-pre3 (crawler@fast.no; http://www.fast.no/faq/faqfastwebsea...ebcrawler.html) FAST-WebCrawler/2.2-pre4 (crawler@fast.no; http://www.fast.no/faq/faqfastwebsea...ebcrawler.html) FAST-WebCrawler/2.2-pre5 (crawler@fast.no; http://www.fast.no/faq/faqfastwebsea...ebcrawler.html) FAST-WebCrawler/2.2-pre8 (crawler@fast.no; http://www.fast.no/faq/faqfastwebsea...ebcrawler.html) FAST-WebCrawler/2.2-pre9 (crawler@fast.no; http://www.fast.no/faq/faqfastwebsea...ebcrawler.html) FAST-WebCrawler/2.1-pre11 (crawler@fast.no; http://www.fast.no/faq/faqfastwebsea...ebcrawler.html) FAST-WebCrawler/2.1-pre12 (crawler@fast.no; http://www.fast.no/faq/faqfastwebsea...ebcrawler.html) FAST-WebCrawler/2.1-pre13 (crawler@fast.no; http://www.fast.no/faq/faqfastwebsea...ebcrawler.html) FAST-WebCrawler/2.1-pre14 (crawler@fast.no; http://www.fast.no/faq/faqfastwebsea...ebcrawler.html) FAST-WebCrawler/2.1-pre10 (crawler@fast.no; http://www.fast.no/faq/faqfastwebsea...ebcrawler.html) ------------------------------------------------------------------------------- Altavista 204.123.9.76 - add-url.altavista.com 204.152.191.47 - scooter.pa.alta-vista.net 204.123.9.123 - test-scooter.av.pa-x.dec.com 204.123.9.18 - av-dev2.av.pa-x.dec.com 204.123.9.19 - scooter2.av.pa-x.dec.com 204.123.9.20 - scooter.pa-x.dec.com 204.123.9.23 - brillo.av.pa-x.dec.com 204.123.9.47 - vscooter.av.pa-x.dec.com 204.123.9.76 - add-url.altavista.com 204.123.9.95 - av-dev3.av.pa-x.dec.com 204.152.191.41 - av-dev4.pa.alta-vista.net 204.152.191.47 - scooter.pa.alta-vista.net 204.152.191.52 - vscooter.pa.alta-vista.net 204.152.190.148 204.152.191.57 128.177.243.155 - scooter2.sv.alta-vista.net 204.152.191.47 - scooter.pa.alta-vista.net 204.152.191.58 128.177.243.82 - test-scooter.sv.alta-vista.net 128.177.243.86 - bigip-nat.sv.alta-vista.net 212.187.226.111 - scooter.aveurope.co.uk Scooter/1.0 Scooter/2.0 G.R.A.B. V1.1.0 vscooter/2.0 G.R.A.B. V1.1.0 |
|
|
|
|
# 25 |
|
Guest
Сообщения: n/a
|
RaZEr
То что надо, спасиибо. Роботы Рамблера при сканировании игнорируют поля <meta name="keywords"...> и все другие поля <meta...>, кроме <meta name="robots"...>. Это связано с тем, что наша система старается индексировать документ таким, какой он есть (то есть таким, каким его видит пользователь). Не секрет, что зачастую создатели интернет-страниц злоупотребляют этими полями, пытаясь заставить поисковые машины находить документ по запросам, не имеющим к нему прямого отношения. А что это за рботс мета тэг? |