Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Imitate GoogleBot - Google's URI Crawler
#11
Hey! That ain't fair.
The point of imitating GoogleBot or some other bot, such as Archive.org is to make some webmaster think he's being spidered Tongue

It is fun, as I did it on MyBB Fans, and everyone though that Archive.org was hitting the forum !

Dr Small
Reply
#12
Well, Good One, Googlebot doesn’t traverse the web at all. It functions much like your web browser, by sending a request to a web server for a web page, downloading the entire page, then handing it off to Google’s indexer.
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  'geo:' URI scheme handler 0.0.1 for Firefox Forum Bot 0 404 03-01-2007, 07:34 PM
Last Post: Forum Bot

Forum Jump:


Users browsing this thread: 1 Guest(s)