Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Javascript > How do I hide javascript content from googlebot?

Reply
Thread Tools

How do I hide javascript content from googlebot?

 
 
Ciaran
Guest
Posts: n/a
 
      07-28-2007
I have a piece of code that I'd rather google's spider did not follow. Is
this
possible please?



 
Reply With Quote
 
 
 
 
Evertjan.
Guest
Posts: n/a
 
      07-28-2007
Ciaran wrote on 28 jul 2007 in comp.lang.javascript:

> I have a piece of code that I'd rather google's spider did not follow.
> Is this possible please?


Only link URLs are "FOLLOWED", code is not.

Or do you want it not to be "INDEXED"?

<http://www.google.com/support/webmasters>

Not really on topic for javascript, meseems.

--
Evertjan.
The Netherlands.
(Please change the x'es to dots in my emailaddress)
 
Reply With Quote
 
 
 
 
Ciaran
Guest
Posts: n/a
 
      07-28-2007

"Evertjan." <(E-Mail Removed)> wrote in message
news:Xns997BB67329118eejj99@194.109.133.242...
> Ciaran wrote on 28 jul 2007 in comp.lang.javascript:
>
>> I have a piece of code that I'd rather google's spider did not follow.
>> Is this possible please?

>
> Only link URLs are "FOLLOWED", code is not.
>
> Or do you want it not to be "INDEXED"?
>
> <http://www.google.com/support/webmasters>
>
> Not really on topic for javascript, meseems.
>
> --
> Evertjan.
> The Netherlands.
> (Please change the x'es to dots in my emailaddress)



Sorry, you're right. I have a bit of javascript code that I dont want to
google to index. Is there a Javascript method to hide this code from the
crawler?
Thanks


 
Reply With Quote
 
Evertjan.
Guest
Posts: n/a
 
      07-28-2007
Ciaran wrote on 28 jul 2007 in comp.lang.javascript:

>
> "Evertjan." <(E-Mail Removed)> wrote in message
> news:Xns997BB67329118eejj99@194.109.133.242...
>> Ciaran wrote on 28 jul 2007 in comp.lang.javascript:
>>
>>> I have a piece of code that I'd rather google's spider did not follow.
>>> Is this possible please?

>>
>> Only link URLs are "FOLLOWED", code is not.
>>
>> Or do you want it not to be "INDEXED"?
>>
>> <http://www.google.com/support/webmasters>
>>
>> Not really on topic for javascript, meseems.

>
> Sorry, you're right. I have a bit of javascript code that I dont want to
> google to index. Is there a Javascript method to hide this code from the
> crawler?


No, not with clientside javascript.

You could test for the word "googlebot" with serverside jscriptin
Request.ServerVariables("HTTP_USER_AGENT"),
but I doubt Google will ever visit your site, if they find out.

Anyhow, why would you want to do that?
Is your scripting so unique and valuable,
that anyone would be interested?

If so, use serverside techniques, which codes are hidden by default.


--
Evertjan.
The Netherlands.
(Please change the x'es to dots in my emailaddress)
 
Reply With Quote
 
Ciaran
Guest
Posts: n/a
 
      07-29-2007

"Evertjan." <(E-Mail Removed)> wrote in message
news:Xns997BBBDAD11BFeejj99@194.109.133.242...
> Ciaran wrote on 28 jul 2007 in comp.lang.javascript:
>
>>
>> "Evertjan." <(E-Mail Removed)> wrote in message
>> news:Xns997BB67329118eejj99@194.109.133.242...
>>> Ciaran wrote on 28 jul 2007 in comp.lang.javascript:
>>>
>>>> I have a piece of code that I'd rather google's spider did not follow.
>>>> Is this possible please?
>>>
>>> Only link URLs are "FOLLOWED", code is not.
>>>
>>> Or do you want it not to be "INDEXED"?
>>>
>>> <http://www.google.com/support/webmasters>
>>>
>>> Not really on topic for javascript, meseems.

>>
>> Sorry, you're right. I have a bit of javascript code that I dont want to
>> google to index. Is there a Javascript method to hide this code from the
>> crawler?

>
> No, not with clientside javascript.
>
> You could test for the word "googlebot" with serverside jscriptin
> Request.ServerVariables("HTTP_USER_AGENT"),
> but I doubt Google will ever visit your site, if they find out.
>
> Anyhow, why would you want to do that?
> Is your scripting so unique and valuable,
> that anyone would be interested?
>
> If so, use serverside techniques, which codes are hidden by default.
>
>
> --
> Evertjan.
> The Netherlands.
> (Please change the x'es to dots in my emailaddress)



Ah yes! Good thinkin - I'll do it with PHP. Thanks!


 
Reply With Quote
 
Bart Van der Donck
Guest
Posts: n/a
 
      07-29-2007
"Evertjan." wrote:

> Ciaran wrote:


>> I have a piece of code that I'd rather google's spider did
>> not follow.


> You could test for the word "googlebot" with serverside jscriptin
> Request.ServerVariables("HTTP_USER_AGENT"),
> but I doubt Google will ever visit your site, if they find out.


You shouldn't do that. The Robots Exclusion Protocol contains widely
accepted rules how to keep crawlers away from your website, or from
some parts of it. Just create a file named "robots.txt" in your
website's root directory.

This "robots.txt"-file keeps all robots out of .js files:

User-agent: *
Disallow: /*.js$

This "robots.txt"-file tells (only) Google to stay away from
directory /javascript/:

User-agent: Googlebot
Disallow: /javascript/

Information:

http://en.wikipedia.org/wiki/Robots_Exclusion_Standard
http://www.google.com/support/webmas...y?answer=33570

--
Bart

 
Reply With Quote
 
Evertjan.
Guest
Posts: n/a
 
      07-29-2007
Bart Van der Donck wrote on 29 jul 2007 in comp.lang.javascript:

> "Evertjan." wrote:
>
>> Ciaran wrote:

>
>>> I have a piece of code that I'd rather google's spider did
>>> not follow.

>
>> You could test for the word "googlebot" with serverside jscriptin
>> Request.ServerVariables("HTTP_USER_AGENT"),
>> but I doubt Google will ever visit your site, if they find out.

>
> You shouldn't do that.


Do what? Ever visit his site when you are Google?

> The Robots Exclusion Protocol contains widely
> accepted rules how to keep crawlers away from your website, or from
> some parts of it. Just create a file named "robots.txt" in your
> website's root directory.
>
> This "robots.txt"-file keeps all robots out of .js files:
>
> User-agent: *
> Disallow: /*.js$
>
> This "robots.txt"-file tells (only) Google to stay away from
> directory /javascript/:
>
> User-agent: Googlebot
> Disallow: /javascript/
>
> Information:
>
> http://en.wikipedia.org/wiki/Robots_Exclusion_Standard
> http://www.google.com/support/webmas...y?answer=33570


My interpretation of "a piece of code" is inHTML code,
not a complete .js file.


--
Evertjan.
The Netherlands.
(Please change the x'es to dots in my emailaddress)
 
Reply With Quote
 
Bart Van der Donck
Guest
Posts: n/a
 
      07-29-2007
"Evertjan." wrote:

> Bart Van der Donck wrote on 29 jul 2007 in comp.lang.javascript:
>
> > "Evertjan." wrote:

>
> >> Ciaran wrote:

>
> >>> I have a piece of code that I'd rather google's spider did
> >>> not follow.

>
> >> You could test for the word "googlebot" with serverside jscriptin
> >> Request.ServerVariables("HTTP_USER_AGENT"),
> >> but I doubt Google will ever visit your site, if they find out.

>
> > You shouldn't do that.

>
> Do what? Ever visit his site when you are Google?


I think it's better to use robots.txt rather than reading out the UA,
and then decide to (whether or not) output javascript code based on
that info.

The general goal is not clear for me. Crawlers don't do much with
javascript; at best they may detect some hyperlinks to index them. So
what's the use of hiding js code for crawlers ? Preventing the code to
be displayed in search results ? I'ld say a well-programmed crawler
wouldn't do that. Preventing source code to influence algorithms like
PageRank ?

> My interpretation of "a piece of code" is inHTML code,
> not a complete .js file.


When I read the orignal poster's sentence "I have a piece of code that
I'd rather google's spider did not follow", I surmised he meant a a
separate .js file, because that's the only thing a bot could follow.

--
Bart

 
Reply With Quote
 
Ciaran
Guest
Posts: n/a
 
      07-31-2007

"Bart Van der Donck" <(E-Mail Removed)> wrote in message
news:(E-Mail Removed) ups.com...
> "Evertjan." wrote:
>
>> Bart Van der Donck wrote on 29 jul 2007 in comp.lang.javascript:
>>
>> > "Evertjan." wrote:

>>
>> >> Ciaran wrote:

>>
>> >>> I have a piece of code that I'd rather google's spider did
>> >>> not follow.

>>
>> >> You could test for the word "googlebot" with serverside jscriptin
>> >> Request.ServerVariables("HTTP_USER_AGENT"),
>> >> but I doubt Google will ever visit your site, if they find out.

>>
>> > You shouldn't do that.

>>
>> Do what? Ever visit his site when you are Google?

>
> I think it's better to use robots.txt rather than reading out the UA,
> and then decide to (whether or not) output javascript code based on
> that info.
>
> The general goal is not clear for me. Crawlers don't do much with
> javascript; at best they may detect some hyperlinks to index them. So
> what's the use of hiding js code for crawlers ? Preventing the code to
> be displayed in search results ? I'ld say a well-programmed crawler
> wouldn't do that. Preventing source code to influence algorithms like
> PageRank ?
>
>> My interpretation of "a piece of code" is inHTML code,
>> not a complete .js file.

>
> When I read the orignal poster's sentence "I have a piece of code that
> I'd rather google's spider did not follow", I surmised he meant a a
> separate .js file, because that's the only thing a bot could follow.
>
> --
> Bart
>


Hi guys, I've already sorted this out using PHP but to clarify:
I have a JS link that records votes on my site. googlebot was clicking it
and voting over and over so I wanted to hide it. Now hold that thought! I
know I shouldn't be doing stuff like that with javascript but I did it years
ago when I didnt really know what I was doing. It's not a big important
project and I'm far to lazy to go back and change the whole system now.

Thanks for the help!
Ciarán


 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Re: How include a large array? Edward A. Falk C Programming 1 04-04-2013 08:07 PM
error: Only Content controls are allowed directly in a content page that contains Content controls. hazz ASP .Net 6 06-09-2010 01:54 PM
How to hide Common boarder content via print.css jnm Javascript 3 03-18-2006 02:51 PM
[HIDE LAN] Hide a part of hosts into the LAN at VPN users? Paolo Bresi Cisco 1 04-04-2005 02:41 PM
req: css based tabs that hide/show content mark | r HTML 2 09-17-2003 08:04 AM



Advertisments