I’ve had it with inktomisearch.com’s crawler bots flooding my access logs with thousands of requests from thousands of IP’s creating thousands of sessions all day, each day, non-stop!
*.inktomisearch.com, you’re dead to me! You’d think Yahoo! (the people who run that circus), could write a decent crawler that knows when to give it a rest, but that thing has been on crawling non-stop for 3 months now (probably more).
I’ll just add this to my robots.txt for now:
User-Agent: Slurp
Disallow: /














