10-17-2010, 06:27 PM
(10-16-2010, 12:01 AM)L3g1tWa5te Wrote: Hahaha, @DAMINK...
Lol "Google and all spiders scrape up all links on your forum. (unless in robots.txt file as deny)"
Yep exactly what i said in my 2 posts in this thread.
However some bots dont obey this rule. Many actually.
Some actually look for the deny and scrape those.
However that really is getting a little off topic i guess.