Velocity Reviews

Velocity Reviews (http://www.velocityreviews.com/forums/index.php)
-   ASP General (http://www.velocityreviews.com/forums/f65-asp-general.html)
-   -   Ping Aaron - [Rel: Spiders] (http://www.velocityreviews.com/forums/t794808-ping-aaron-rel-spiders.html)

Steven Burn 06-07-2004 12:12 AM

Ping Aaron - [Rel: Spiders]
 
Aaron,
Apologies for the ping but, your the only one I can think of offhand that
will know what I'm looking for.

Basically, to cut a long story short, I run an sURL service (surl.co.uk)
that allows one to convert long URL's to short etc, and am looking to have a
search facility on it that will allow one to locate an sURL for a specific
subject.

Now, the problem I'm running into is, I'd like to have some sort of spider
that will go to each site in the database, and grab the title, keywords, and
description from the meta tags, As I've never written a spider in my life,
and have been unsuccessful in locating info on how to go about doing so, I
was hoping you could point me in the right direction?

--

Regards

Steven Burn
Ur I.T. Mate Group
www.it-mate.co.uk

Keeping it FREE!





Aaron [SQL Server MVP] 06-07-2004 05:18 AM

Re: Ping Aaron - [Rel: Spiders]
 
One way is to use ServerXMLHTTP (see http://www.aspfaq.com/2173). Once you
have the response text back, you can parse it for <title> and </title>, etc.

--
http://www.aspfaq.com/
(Reverse address to reply.)




"Steven Burn" <pvt@noyb.com> wrote in message
news:u51cCRCTEHA.3012@tk2msftngp13.phx.gbl...
> Aaron,
> Apologies for the ping but, your the only one I can think of offhand that
> will know what I'm looking for.
>
> Basically, to cut a long story short, I run an sURL service (surl.co.uk)
> that allows one to convert long URL's to short etc, and am looking to have

a
> search facility on it that will allow one to locate an sURL for a specific
> subject.
>
> Now, the problem I'm running into is, I'd like to have some sort of spider
> that will go to each site in the database, and grab the title, keywords,

and
> description from the meta tags, As I've never written a spider in my life,
> and have been unsuccessful in locating info on how to go about doing so, I
> was hoping you could point me in the right direction?
>
> --
>
> Regards
>
> Steven Burn
> Ur I.T. Mate Group
> www.it-mate.co.uk
>
> Keeping it FREE!
>
>
>
>




Steven Burn 06-07-2004 07:26 AM

Re: Ping Aaron - [Rel: Spiders]
 
Cheers Aaron..... I had a feeling that was going to be the response (I've
got a page on my site that uses it (checks whether a site is online or not),
but it's terribly slow for some reason)

--

Regards

Steven Burn
Ur I.T. Mate Group
www.it-mate.co.uk

Keeping it FREE!


"Aaron [SQL Server MVP]" <ten.xoc@dnartreb.noraa> wrote in message
news:e6tmN8ETEHA.3548@TK2MSFTNGP09.phx.gbl...
> One way is to use ServerXMLHTTP (see http://www.aspfaq.com/2173). Once

you
> have the response text back, you can parse it for <title> and </title>,

etc.
>
> --
> http://www.aspfaq.com/
> (Reverse address to reply.)
>
>
>
>
> "Steven Burn" <pvt@noyb.com> wrote in message
> news:u51cCRCTEHA.3012@tk2msftngp13.phx.gbl...
> > Aaron,
> > Apologies for the ping but, your the only one I can think of offhand

that
> > will know what I'm looking for.
> >
> > Basically, to cut a long story short, I run an sURL service (surl.co.uk)
> > that allows one to convert long URL's to short etc, and am looking to

have
> a
> > search facility on it that will allow one to locate an sURL for a

specific
> > subject.
> >
> > Now, the problem I'm running into is, I'd like to have some sort of

spider
> > that will go to each site in the database, and grab the title, keywords,

> and
> > description from the meta tags, As I've never written a spider in my

life,
> > and have been unsuccessful in locating info on how to go about doing so,

I
> > was hoping you could point me in the right direction?
> >
> > --
> >
> > Regards
> >
> > Steven Burn
> > Ur I.T. Mate Group
> > www.it-mate.co.uk
> >
> > Keeping it FREE!
> >
> >
> >
> >

>
>





All times are GMT. The time now is 01:39 PM.

Powered by vBulletin®. Copyright ©2000 - 2014, vBulletin Solutions, Inc.
SEO by vBSEO ©2010, Crawlability, Inc.