Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > HTML > HELLLLP!!! The spiders are out of control!!

Reply
Thread Tools

HELLLLP!!! The spiders are out of control!!

 
 
dE|_
Guest
Posts: n/a
 
      12-14-2007
Searches of my new site is too interested in the descriptive nav bar on all
7 pages and robots.txt will only control indexing of pages as a whole.

Surely someone here has been through this before??!?

Try it if you like but it's not styled yet
http://www.websitefoundry.co.uk/search/search.php
Search for something like multimedia or trading.

---dE|_---


 
Reply With Quote
 
 
 
 
Chaddy2222
Guest
Posts: n/a
 
      12-14-2007
On Dec 15, 3:07 am, "dE|_" <(E-Mail Removed)> wrote:
> Searches of my new site is too interested in the descriptive nav bar on all
> 7 pages and robots.txt will only control indexing of pages as a whole.
>
> Surely someone here has been through this before??!?
>
> Try it if you like but it's not styled yethttp://www.websitefoundry.co.uk/search/search.php
> Search for something like multimedia or trading.
>

What is your problem exactly?
The SE bots are ment to crawl your site. If you have a lot of content
then they will visit more offen. Maybe give it a week or so.
--
Regards Chad. http://freewebdesignonline.org

 
Reply With Quote
 
 
 
 
dE|_
Guest
Posts: n/a
 
      12-14-2007

"Chaddy2222" <(E-Mail Removed)> wrote in message
news:(E-Mail Removed)...
> On Dec 15, 3:07 am, "dE|_" <(E-Mail Removed)> wrote:
>> Searches of my new site is too interested in the descriptive nav bar on
>> all
>> 7 pages and robots.txt will only control indexing of pages as a whole.
>>
>> Surely someone here has been through this before??!?
>>
>> Try it if you like but it's not styled
>> yethttp://www.websitefoundry.co.uk/search/search.php
>> Search for something like multimedia or trading.
>>

> What is your problem exactly?
> The SE bots are ment to crawl your site. If you have a lot of content
> then they will visit more offen. Maybe give it a week or so.


This is a built in SE that I index myself. If it finds a searched-for word
in the nav bar first you just get multiple lines of the same nav bar content
in the results.

http://www.websitefoundry.co.uk/search/search.php

type in production and you'll see.

---dE|_---


 
Reply With Quote
 
bizshop@gmail.com
Guest
Posts: n/a
 
      12-14-2007
On Dec 14, 8:41 am, "dE|_" <(E-Mail Removed)> wrote:
> "Chaddy2222" <(E-Mail Removed)> wrote in message
>
> news:(E-Mail Removed)...
>
> > On Dec 15, 3:07 am, "dE|_" <(E-Mail Removed)> wrote:
> >> Searches of my new site is too interested in the descriptive nav bar on
> >> all
> >> 7 pages and robots.txt will only control indexing of pages as a whole.

>
> >> Surely someone here has been through this before??!?

>
> >> Try it if you like but it's not styled
> >> yethttp://www.websitefoundry.co.uk/search/search.php
> >> Search for something like multimedia or trading.

>
> > What is your problem exactly?
> > The SE bots are ment to crawl your site. If you have a lot of content
> > then they will visit more offen. Maybe give it a week or so.

>
> This is a built in SE that I index myself. If it finds a searched-for word
> in the nav bar first you just get multiple lines of the same nav bar content
> in the results.
>
> http://www.websitefoundry.co.uk/search/search.php
>
> type in production and you'll see.
>
> ---dE|_---


I don't know how you have your indexing set. If set to crawl .php
pages, make your navbar an .inc file (or something else) so it won't
be crawled.


 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Evil Spiders in Macro Vision XhArD The Lounge 19 01-04-2006 03:15 PM
Web user controls & Google SEO spiders? =?Utf-8?B?VGhvbWFzIEFuZGVyc3Nvbg==?= ASP .Net 7 10-25-2004 02:13 PM
User Controls, Google SEO & spiders =?Utf-8?B?VGhvbWFzIEFuZGVyc3Nvbg==?= ASP .Net 2 10-11-2004 06:49 PM
I write virtual cats and spiders in Java ! Brian Raab Java 0 10-06-2004 07:16 AM
How to make dynamic aspx pages indexable by spiders moondaddy ASP .Net 10 04-27-2004 02:40 PM



Advertisments