Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Perl > Perl Misc > Alternatives to LWP::Parallel

Reply
Thread Tools

Alternatives to LWP::Parallel

 
 
Peter
Guest
Posts: n/a
 
      10-24-2009
I'm in the process of writing a script that will take a whole bunch of
URL's and get their HTTP status. I was going to use
LWP:arallel::UserAgent to handle parallel HTTP requests. However, I
found that this module does not work with later version of libwww and
it looks like this module is no longer maintained.

So I started looking around at other methods/modules I could use.
There are certainly quite a few to choose from and some are quite
complex for my needs.

Some of the modules I looked at were:

IO::Lambda::HTTP (this looked the most robust/well maintained, but has
a learning curve to it)
POE::Component::Client::HTTP (as above)
AnyEvent::HTTP (this looked promising, but I couldn't get any results
with it)
HTTP::Client:arallel (doesn't appear to have any function to limit
number of connections)
HTTP::Async (a little buggy and didn't seem to produce expected
results).

One essential feature I need is the ability to throttle/limit the
number of simultaneous connections or number of connections/requests
per minute.

I'd appreciate any advice as to what method/module might be best for
me. The other governing factor is that I need it to run with
ActivePerl on Linux and Windows.
 
Reply With Quote
 
 
 
 
J. Gleixner
Guest
Posts: n/a
 
      10-26-2009
Peter wrote:
> I'm in the process of writing a script that will take a whole bunch of
> URL's and get their HTTP status. I was going to use
> LWP:arallel::UserAgent to handle parallel HTTP requests. However, I
> found that this module does not work with later version of libwww and
> it looks like this module is no longer maintained.
>
> So I started looking around at other methods/modules I could use.
> There are certainly quite a few to choose from and some are quite
> complex for my needs.
>
> Some of the modules I looked at were:


Why not simply Parallel::ForkManager and LWP::UserAgent?

[...]
 
Reply With Quote
 
 
 
 
Peter
Guest
Posts: n/a
 
      10-26-2009
On 27 Oct, 06:02, "J. Gleixner" <(E-Mail Removed)>
wrote:
> Peter wrote:
> > I'm in the process of writing a script that will take a whole bunch of
> > URL's and get their HTTP status. *I was going to use
> > LWP:arallel::UserAgent to handle parallel HTTP requests. *However, I
> > found that this module does not work with later version of libwww and
> > it looks like this module is no longer maintained.

>
> > So I started looking around at other methods/modules I could use.
> > There are certainly quite a few to choose from and some are quite
> > complex for my needs.

>
> > Some of the modules I looked at were:

>
> Why not simply Parallel::ForkManager and LWP::UserAgent?
>
> [...]



Ah, now that looks to be just what I need. I don't know how I managed
to miss that one. Thanks.
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
BGP alternatives? SP Cisco 3 09-08-2009 02:32 AM
LEAP & ACS Alternatives N. Hall Cisco 2 05-28-2005 08:31 AM
MS Press 2003 books and alternatives Bill Bixby MCSE 7 04-29-2004 05:52 PM
alternatives to accessing PIX via Telnet Anne Robynn Cisco 3 01-03-2004 08:48 AM
Alternatives when user's browser doesn't accept cookies Robert V. Hanson ASP .Net 2 07-03-2003 03:24 AM



Advertisments