Google Spider?

Once I saw 25 google spiders on at one time...about a week ago I think heh. I think the google spiders just search the web and stuff for sites that can go in thier searches.

Yeah, Google Spiders are just little programs that navigate through the internet, looking at different sites, and gathering information for the Google search engine.

Wow, how many time are we going to go over this? It's really not that hard of a concept to grasp.

Eeek! It's a google spider...man. 😊

Rainman wrote:Wow, how many time are we going to go over this? It's really not that hard of a concept to grasp.

Yeah I was just going to say, this was posted AGAIN i think in New Members Forum like 2 weeks ago, maybe. 🙄

I don't go in the NM forum. Sorry. It was just a quick question.

SixTen wrote:I don't go in the NM forum. Sorry. It was just a quick question.

Honest mistake. No need to aplogize to anyone. I would have asked the same thing if I hadn't known what a Google Spider was.

Whoa! Deja Vu! Wasn't there a post about this same subject a couple weeks ago?

Me wrote:Yeah I was just going to say, this was posted AGAIN i think in New Members Forum like 2 weeks ago, maybe. 🙄

Archibaldo wrote:Whoa! Deja Vu! Wasn't there a post about this same subject a couple weeks ago?

Yes, yes in fact there was. I posted about that, as did Rainman.

Don't those things eat up a good deal of bandwidth crawling the site all the time? (and we have a lot of pages to crawl too) If so, I would think it would be wise to set up a robots.txt and restrict their access to only a few pages.