My Advise: Don’t Do It
Experimenting with Googlebot is indeed interesting. However my advise to anyone contemplating to restrict Googleot from your site via robots.txt is simple: DON’T DO IT! Besides the fact that in a short term you will loose some search traffic(no matter how little that traffic is) you may suffer some long term consequences.
Google commands authority whether we like it or not. Once your website is gone from Google index it may be interpreted that your website is not safe to be visited, what is worst many web masters may think that your website is not safe to link to. Mark’s article about his websites being hacked may not be a good example, I believe his websites were not removed from index. But let say your website is 2-3 years old and is not in Google’s index, other webmasters will look at it suspiciously and I do not blame them.
I was lucky enough to generate links from some very influential bloggers even after my website was gone from Google’s index- you may not be so lucky.
Am I letting the Googlebot back? Not yet. My experiment is not over. However, besides learning a few things my experiment is accomplishing nothing. I think engaging in a constructive criticism will bring much better results!
-
Mike
-
Vlad
-
Sapphire
