Google Hates Me

*geek alert – You may want to skip down to the more filling parts of this blog*

I’m always amazed how much traffic my site actually gets even though only a fraction of visitors actually comment. On average, I get about 1300 unique hits a day now. Take a peek.

Of course, a big chunk of that is web-bots indexing my site. And while I’m not even close to exceeding my bandwidth, it is frustrating trying to separate the real clicks from the bot clicks. In the past, I’ve been hesitant to set up a robot.txt file¹ as I don’t want to discourage new readers from finding me. However, I think it is time to implement some controls to keep it in check.

The funny thing is I always assumed Google was the biggest indexing bot for my site. Boy was I wrong. Yahoo was way ahead by almost 2/3’s. Google was second with MSN bringing up the rear. I also discovered a sp@m bot indexing me as well. I guess this explains why my sp@m hits have jumped up a lot lately. I googled its IP address and added it to the filter so we’ll see how it goes.