[GeoStL] Re: interesting observation. Non-slaga related
- From: Glenn <Glenn@xxxxxxxxxx>
- To: geocaching@xxxxxxxxxxxxx, <geocaching@xxxxxxxxxxxxx>
- Date: Mon, 23 Aug 2004 18:44:42 -0500
-
i think that they are limited to main pages and none of the private areas.
At 06:19 PM 8/23/2004, Russ Wilner wrote:
>-
>Is it not 'robots.txt' that can be configured to tell those 'bots (at =
>least
>the well-behaved ones) where and where not they can go on any given =
>site?
>
>Maybe you don't want to limit their probing. I'm just not sure I'd want =
>them
>crawling through every nook & crannie.
>
>
> -=3D Russ =3D- (Bot Fee)
>
> > -----Original Message-----
> > From: geocaching-bounce@xxxxxxxxxxxxx=20
> > [mailto:geocaching-bounce@xxxxxxxxxxxxx] On Behalf Of Glenn
> > Sent: Monday, August 23, 2004 18:11
> > To: geocaching@xxxxxxxxxxxxx
> > Subject: [GeoStL] interesting observation. Non-slaga related
> >=20
> >=20
> > -
> > I have been playing with the forum system for mogeo. When I=20
> > am logged in,=20
> > I can see when the google "spiders" and "web crawling bots"=20
> > hit the site ,=20
> > jump from page to page indexing the various pages. It is=20
> > really weird, dont=20
> > know how they figure out what to do and were to go or how the=20
> > site was even=20
> > there. The bots showed up just hours after turning the thing=20
> > on and they=20
> > visit often.
> >=20
> > It has been up just a week and now googling just on "mogeo=20
> > forums" will=20
> > get you
> > mogeo.forumflash.com/index.php?act=3Didx that goes right to=20
> > the site and it=20
> > is 3rd on the list behind 2 other mogeo pages! Pretty neat=20
> > and not bad. I=20
> > thought you had to submit something to have the google bots=20
> > come visit.
> >=20
> > Bots and spiders.... sounds like something that would come=20
> > out from under a=20
> > log..
> >=20
> > ---------------------
> > Glenn
> >=20
> >=20
> >=20
> > ****************************************
> > Our WebPage! Http://WWW.GeoStL.com =20
> > Mail List Info.=20
> > //www.freelists.org/cgi-bin/list?list_id=3Dgeocaching
> > Mail List FAQ's: //www.freelists.org/help/questions.html=20
> > ****************************************
> > To unsubscribe from this list:
> > send an email to geocaching-request@xxxxxxxxxxxxx with=20
> > 'unsubscribe' in the Subject field
> >=20
> >=20
> >=20
> >=20
> >=20
> >=20
>
> ****************************************
> Our WebPage! Http://WWW.GeoStL.com
> Mail List Info. //www.freelists.org/cgi-bin/list?list_id=geocaching
> Mail List FAQ's: //www.freelists.org/help/questions.html
> ****************************************
>To unsubscribe from this list:
> send an email to geocaching-request@xxxxxxxxxxxxx with 'unsubscribe' in
> the Subject field
****************************************
Our WebPage! Http://WWW.GeoStL.com
Mail List Info. //www.freelists.org/cgi-bin/list?list_id=geocaching
Mail List FAQ's: //www.freelists.org/help/questions.html
****************************************
To unsubscribe from this list:
send an email to geocaching-request@xxxxxxxxxxxxx with 'unsubscribe' in the
Subject field
Other related posts:
