Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Java > Suggestions on a Web Spider that ....

Reply
Thread Tools

Suggestions on a Web Spider that ....

 
 
JeepGary
Guest
Posts: n/a
 
      10-21-2003
Hello,

I am looking for a web spider that downloads the text content of a web
page. Any ideas if one already exist. That would save a lot of time.

Thanks,
Gary
 
Reply With Quote
 
 
 
 
Bjoern
Guest
Posts: n/a
 
      10-21-2003
JeepGary wrote:
> Hello,
>
> I am looking for a web spider that downloads the text content of a web
> page. Any ideas if one already exist. That would save a lot of time.


I've had good results with Pavuk, http://www.idata.sk/~ondrej/pavuk/

Worked better for me than wget.

Bjoern

--


Let's not weep for their evil deeds,
but for their lack of imagination
(Nick Cave)

 
Reply With Quote
 
 
 
 
Roedy Green
Guest
Posts: n/a
 
      10-21-2003
On 20 Oct 2003 18:36:25 -0700, http://www.velocityreviews.com/forums/(E-Mail Removed) (JeepGary) wrote or
quoted :

>I am looking for a web spider that downloads the text content of a web
>page. Any ideas if one already exist. That would save a lot of time.


For link checking, look at xenu. It is much faster than most since it
is many thread. See http://mindprod.com/jgloss/xenu.html

For code to programmatically download a page with an http GET see
http://mindprod.com/fileio.html

--
Canadian Mind Products, Roedy Green.
Coaching, problem solving, economical contract programming.
See http://mindprod.com/jgloss/jgloss.html for The Java Glossary.
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
SEO spider-spider ANY website prasadpelluri5@gmail.com Java 0 10-30-2008 07:04 PM
how google spider access my web site? baroque Chou ASP .Net 7 02-02-2006 09:25 PM
web spider and password protected pages jdonnell Python 5 02-17-2005 01:42 AM
Web Crawler / Spider Commercial Software Info Request Gray Ghost Computer Support 1 11-07-2004 01:48 PM
Web Spider Thomas Lindgaard Python 3 07-07-2004 01:29 PM



Advertisments