Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Perl > Perl Misc > Very Interesting max_size in UserAgent

Reply
Thread Tools

Very Interesting max_size in UserAgent

 
 
Public Interest
Guest
Posts: n/a
 
      10-07-2003
#!/usr/bin/perl
use Net::HTTP;
use LWP::UserAgent;

$ua = LWP::UserAgent->new ( );
$ua->max_size(2000);
$url = 'http://www.yahoo.com';

my $req = HTTP::Request->new(GET => "$url");

$htmlcode = $ua->request($req)->content;
print $htmlcode;

# the problem is if i change 2000 to 1950 in max_size, the $htmlcode is
still the same. Looks to me that the server is flushing and client is using
buffer to hold it and the server decide how much each flushing is giving,
and if each flushing is 1024 bytes, and max_size is checked after each
flushing. So, max size of 1025 to max size of 2047 will give the same
result: both are 2048. Is that right? So max_size in UserAgent is really an
estimated value.


 
Reply With Quote
 
 
 
 
ko
Guest
Posts: n/a
 
      10-07-2003
Public Interest wrote:
> #!/usr/bin/perl
> use Net::HTTP;
> use LWP::UserAgent;
>
> $ua = LWP::UserAgent->new ( );
> $ua->max_size(2000);
> $url = 'http://www.yahoo.com';
>
> my $req = HTTP::Request->new(GET => "$url");
>
> $htmlcode = $ua->request($req)->content;
> print $htmlcode;
>
> # the problem is if i change 2000 to 1950 in max_size, the $htmlcode is
> still the same. Looks to me that the server is flushing and client is using
> buffer to hold it and the server decide how much each flushing is giving,
> and if each flushing is 1024 bytes, and max_size is checked after each
> flushing. So, max size of 1025 to max size of 2047 will give the same
> result: both are 2048. Is that right? So max_size in UserAgent is really an
> estimated value.


No, it is a specific limit. From the LWP::UserAgent docs:

$ua->max_size([$bytes])

Get/set the size limit for response content. The default is undef, which
means that there is no limit. If the returned response content is only
partial, because the size limit was exceeded, then a ``Client-Aborted''
header will be added to the response.

So if you want to check if max_size has been exceeded, you have to test
for if there is a Client-Aborted header:

#!/usr/intel/bin/perl -w
use strict;

use LWP::UserAgent;

my $ua = LWP::UserAgent->new( max_size => 0);
my $url = 'http://www.yahoo.com';
print "Exceeded max_size\n" if $ua->get($url)->header('Client-Aborted');

HTH - keith

 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
std::tr1::array - What's the difference between max_size and size? Sarath C++ 2 08-19-2009 08:46 AM
max_size() t C++ 4 10-03-2007 08:20 AM
method max_size() for map-object (STL) returns -1 alexey_m C++ 2 12-22-2005 11:24 AM
Exceeding container::max_size()? Kevin Goodsell C++ 1 04-03-2004 11:49 PM
UserAgent's Max_Size and HTTP's header field Range Great Deals Perl Misc 1 10-01-2003 08:05 PM



Advertisments