Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Perl > Perl Misc > LWP Post

Reply
Thread Tools

LWP Post

 
 
Dale Gerdemann
Guest
Posts: n/a
 
      10-21-2004
I'm trying to figure out how use LWP::UserAgent to get dynamically
generated Web pages using the POST method. I've followed the examples
in the O'Reilly book: Perl & LWP by Sean M. Burke. There's an example
there of posting to the AltaVista Babelfish translator, and this
example works fine for me. But when I try to do someting similar on
another page, it fails.

For example, I tried to get morphological analysis (word structure
analysis) from http://www.uni-plovdiv.bg/dcs/morphe.htm, and even
though it's a very simple minded form, I had no success. Sean Burke
has a program in his book called formpairs, which allows you to test
what posted info a form is expecting. I used this and checked very
carefully, but still, no success.

Here's what the form looks like:

function FrontPage_Form1_Validator(theForm)
{

if (theForm.D1.selectedIndex < 0)
{
alert("Please select one of the \"D1\" options.");
theForm.D1.focus();
return (false);
}
return (true);
}
//--></script><!--webbot BOT="GeneratedScript" endspan --><form
action="http://rdesc.pu.acad.bg/cgi-bin/cgi_morf.exe" method="POST"
onsubmit="return FrontPage_Form1_Validator(this)"
name="FrontPage_Form1">
<p><textarea name="S1" rows="5" cols="82"></textarea></p>
<p><!--webbot bot="Validation" b-value-required="TRUE" --><select
name="D1" size="3">
<option>1.Windows ( )</option>
<option>2.Macintosh ( abvgde'zijklmnoprstufxchw]=;_q )</option>
<option>3.QWERTY ( abwgdevzijklmnoprstufhc`[]yx\q )</option>
</select></p>

<p><input type="submit" name="B1" value="Analysis"></p>
</form>


You see that there's a bit of Javascript getting in the way, but I
don't think thats a problem here. [You also see that the author of the
page mixed up is terminology and referred to ASCII as QWERTY.
But that's beside the point.] I tried to access this page with the
following:

use strict;
use LWP;
my $browser;
sub do_POST {
# Parameters:
# the URL,
# an arrayref or hashref for the key/value pairs,
# and then, optionally, any header lines: (key,value, key,value)
$browser = LWP::UserAgent->new( ) unless $browser;
my $resp = $browser->post(@_);
return ($resp->content, $resp->status_line, $resp->is_success, $resp)
if wantarray;
return unless $resp->is_success;
return $resp->content;
}



my ($content, $message, $is_success) = do_POST(
'http://www.uni-plovdiv.bg/dcs/morphe.htm',
[ 'S1' => 'ne', 'B1' => "Analysis",
'D1'=>'3.QWERTY ( abwgdevzijklmnoprstufhc`[]yx\q )' ],
);

print "$content\n";



Anybody have any ideas???


Dale Gerdemann
 
Reply With Quote
 
 
 
 
Tim Heaney
Guest
Posts: n/a
 
      10-21-2004
http://www.velocityreviews.com/forums/(E-Mail Removed)-tuebingen.de (Dale Gerdemann) writes:

> my ($content, $message, $is_success) = do_POST(
> 'http://www.uni-plovdiv.bg/dcs/morphe.htm',
> [ 'S1' => 'ne', 'B1' => "Analysis",
> 'D1'=>'3.QWERTY ( abwgdevzijklmnoprstufhc`[]yx\q )' ],
> );
>
> print "$content\n";
>
>
>
> Anybody have any ideas???


You don't want to post to the HTML page with the form on it, you want
to post to their program. Try something like

my ($content, $message, $is_success) = do_POST(
'http://rdesc.pu.acad.bg/cgi-bin/cgi_morf.exe',
[ 'S1' => 'ne',
'D1' => '3.QWERTY ( abwgdevzijklmnoprstufhc`[]yx\q )',
'B1' => "Analysis",
],
);

instead (note the different URL). When I do that, I get

<HTML><HEAD><TITLE>DELPHI CGI TEST</TITLE></HEAD>
<BODY bgcolor="#FFFFFF">
<STRONG>Result:<BR><BR>
<P>ne article, :189 1 base form:ne</P>
</B></P>
</BODY>

I hope this helps,

Tim
 
Reply With Quote
 
 
 
 
Tad McClellan
Guest
Posts: n/a
 
      10-21-2004
Dale Gerdemann <(E-Mail Removed)-tuebingen.de> wrote:

> I'm trying to figure out how use LWP::UserAgent to get dynamically
> generated Web pages using the POST method.


> For example, I tried to get morphological analysis (word structure
> analysis) from http://www.uni-plovdiv.bg/dcs/morphe.htm, and even
> though it's a very simple minded form, I had no success.



> Here's what the form looks like:


[snip]

> <form
> action="http://rdesc.pu.acad.bg/cgi-bin/cgi_morf.exe" method="POST"

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^


There is where the message should be posted to...


> my ($content, $message, $is_success) = do_POST(
> 'http://www.uni-plovdiv.bg/dcs/morphe.htm',

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^


.... but that isn't where you are actually posting it!


> Anybody have any ideas???



Post to the correct URL.



BTW, the "web scraping proxy" is a wildly valuable tool for
reverse-engineering such things:

http://www.research.att.com/~hpk/wsp/

It wrote this Perl code for me when I tried it:

$request = POST "http://rdesc.pu.acad.bg/cgi-bin/cgi_morf.exe" , [
'S1' => "a Bulgarian sentence",
'D1' => "3.QWERTY ( abwgdevzijklmnoprstufhc`[]yx\q )",
'B1' => "Analysis",
] ;


--
Tad McClellan SGML consulting
http://www.velocityreviews.com/forums/(E-Mail Removed) Perl programming
Fort Worth, Texas
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Making urllib2's POST 302 handle same as Perl LWP's behaviour Karra Python 2 12-29-2010 02:03 PM
LWP::UserAgent post to post other than 80 Zapanaz Perl Misc 4 10-12-2005 06:11 PM
LWP::UserAgent to POST and redirect browser at the same time La Jesus Perl 9 10-27-2003 09:55 PM
some proxies post and some won't with lwp Robert Perl Misc 5 09-23-2003 03:13 PM
LWP - Upload via HTTP POST Mark Mykkanen Perl Misc 1 07-02-2003 08:47 PM



Advertisments