si3009.inktomisearch.com - - [12/Jun/2003:16:09:32 +0200] "GET /robots.txt HTTP/1.0" 404 317 "-" "Mozilla/5.0 (Slurp/si;
slurp@inktomi.com;
http://www.inktomi.com/slurp.html)"
66.77.73.66 - - [11/Jun/2003:22:23:25 +0200] "GET /robots.txt HTTP/1.0" 404 321 "-" "FAST-WebCrawler/3.8 (atw-crawler at fast dot no;
http://fast.no/support/crawler.asp)"
64.68.82.12 - - [11/Jun/2003:10:35:05 +0200] "GET /robots.txt HTTP/1.0" 404 317 "-" "Googlebot/2.1 (+
http://www.googlebot.com/bot.html)"
64.68.87.41 - - [11/Jun/2003:18:22:06 +0200] "GET /robots.txt HTTP/1.0" 404 321 "-" "Mediapartners-Google/2.1 (+
http://www.googlebot.com/bot.html)"
213.180.194.163 [06/Jun/2003:00:19:52 +0400] "GET /robots.txt HTTP/1.1" 404 1952 "-" "Yandex/1.01.001 (compatible; Win16; I)"
216.39.48.111 [08/Jun/2003:02:12:48 +0400] "GET /robots.txt HTTP/1.1" 404 1952 "-" "Scooter/3.2"
Из моих логов... потом ещё пороюсь =)