Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > HTML > Re: robots.txt

Reply
Thread Tools

Re: robots.txt

 
 
lostinspace
Guest
Posts: n/a
 
      08-08-2003
----- Original Message -----
From: David Graham <>
Newsgroups: alt.html
Sent: Friday, August 08, 2003 10:42 AM
Subject: robots.txt


> Hi
> I have the following 2 lines in my robots.txt to prevent search engines
> indexing the files and folders off the sefriendly folder:
>
> User-agent: *
> Disallow: /sefriendly/
>
> To prevent access to another folder off the root folder called www2, do I
> just add another disallow line like below
>
>
> Disallow: /www2/
>
>
> Thanks
> David
>



David,
Prevent and disallow or even allow or not adequate descriptions of
what robots.txt does.
Although those terms are frequently used.

Robots.txt are "suggestions" for compliant bots.

Htaccess is a control which implements prevention, allows and disallows.

hope this helps.


 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off




Advertisments