Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Perl > Perl Misc > LWP with proxy problem

Reply
Thread Tools

LWP with proxy problem

 
 
Woogie
Guest
Posts: n/a
 
      12-23-2003
When running the sample code below without a proxy the GET returns the
expected data. When run with the $proxy uncommented the GET returns
the content of the login page for the site being accessed. The site
in the code is valid for ease of testing. I also am including the LWP
debug info for each attempt.

Can anyone explain this behavior and what can I do to correct it?

Thanks in advance

Trace without proxy:

LWP::UserAgent::new: ()
LWP::UserAgent::request: ()
LWP::UserAgent::send_request: GET
https://squid.servebeer.com/getservi...ing&format=csv
LWP::UserAgent::_need_proxy: Not proxied
LWP:rotocol::http::request: ()
LWP:rotocol::collect: read 28 bytes
LWP::UserAgent::request: Simple response: OK

Home
PE


Trace with proxy:

LWP::UserAgent::new: ()
LWP::UserAgent:roxy: https http://148.245.207.85:8080
LWP::UserAgent::request: ()
LWP::UserAgent::send_request: GET
https://squid.servebeer.com/getservi...ing&format=csv
LWP::UserAgent::_need_proxy: Proxied to http://148.245.207.85:8080
LWP:rotocol::http::request: ()
LWP:rotocol::collect: read 236 bytes
LWP:rotocol::collect: read 594 bytes
LWP:rotocol::collect: read 416 bytes
LWP:rotocol::collect: read 450 bytes
LWP:rotocol::collect: read 1017 bytes
LWP:rotocol::collect: read 443 bytes
LWP:rotocol::collect: read 643 bytes
LWP::UserAgent::request: Simple response: OK

<html lang="en">

<!-- Start Head -->
<head>
<title>

Error

</title>
<script language="JavaScript">
....



Here is the sample code:


#!/usr/bin/perl -w

use LWP::UserAgent;
use HTTP::Request;
use HTTP::Response;
use Crypt::SSLeay;

LWP:ebug::level('+');

$url = "https://squid.servebeer.com/getservices.do?user=Guest&password=JustLooking&for mat=csv";
#$proxy="http://xxx.xxx.xxx.xxx:8080";

$ua = LWP::UserAgent->new();

if (defined $proxy)
{
$ENV{HTTPS_PROXY} = $proxy;

# initialize from environment variables
$ua->env_proxy;
}

$req = HTTP::Request->new(GET => $url);
$response = $ua->request($req);
if ($response->is_error())
{
printf " %s\n", $response->status_line;
}

else
{
$content = $response->content();
print $content;
}

exit;
 
Reply With Quote
 
 
 
 
Sara
Guest
Posts: n/a
 
      12-24-2003
http://www.velocityreviews.com/forums/(E-Mail Removed) (Woogie) wrote in message news:<(E-Mail Removed). com>...
> When running the sample code below without a proxy the GET returns the
> expected data. When run with the $proxy uncommented the GET returns
> the content of the login page for the site being accessed. The site
> in the code is valid for ease of testing. I also am including the LWP
> debug info for each attempt.
>
> Can anyone explain this behavior and what can I do to correct it?
>
> Thanks in advance
>
> Trace without proxy:
>
> LWP::UserAgent::new: ()
> LWP::UserAgent::request: ()
> LWP::UserAgent::send_request: GET
> https://squid.servebeer.com/getservi...ing&format=csv
> LWP::UserAgent::_need_proxy: Not proxied
> LWP:rotocol::http::request: ()
> LWP:rotocol::collect: read 28 bytes
> LWP::UserAgent::request: Simple response: OK
>
> Home
> PE
>
>
> Trace with proxy:
>
> LWP::UserAgent::new: ()
> LWP::UserAgent:roxy: https http://148.245.207.85:8080
> LWP::UserAgent::request: ()
> LWP::UserAgent::send_request: GET
> https://squid.servebeer.com/getservi...ing&format=csv
> LWP::UserAgent::_need_proxy: Proxied to http://148.245.207.85:8080
> LWP:rotocol::http::request: ()
> LWP:rotocol::collect: read 236 bytes
> LWP:rotocol::collect: read 594 bytes
> LWP:rotocol::collect: read 416 bytes
> LWP:rotocol::collect: read 450 bytes
> LWP:rotocol::collect: read 1017 bytes
> LWP:rotocol::collect: read 443 bytes
> LWP:rotocol::collect: read 643 bytes
> LWP::UserAgent::request: Simple response: OK
>
> <html lang="en">
>
> <!-- Start Head -->
> <head>
> <title>
>
> Error
>
> </title>
> <script language="JavaScript">
> ...
>
>
>
> Here is the sample code:
>
>
> #!/usr/bin/perl -w
>
> use LWP::UserAgent;
> use HTTP::Request;
> use HTTP::Response;
> use Crypt::SSLeay;
>
> LWP:ebug::level('+');
>
> $url = "https://squid.servebeer.com/getservices.do?user=Guest&password=JustLooking&for mat=csv";
> #$proxy="http://xxx.xxx.xxx.xxx:8080";
>
> $ua = LWP::UserAgent->new();
>
> if (defined $proxy)
> {
> $ENV{HTTPS_PROXY} = $proxy;
>
> # initialize from environment variables
> $ua->env_proxy;
> }
>
> $req = HTTP::Request->new(GET => $url);
> $response = $ua->request($req);
> if ($response->is_error())
> {
> printf " %s\n", $response->status_line;
> }
>
> else
> {
> $content = $response->content();
> print $content;
> }
>
> exit;


might want to try on one of the cgi groups or even comp.perl.modules ?
 
Reply With Quote
 
 
 
 
Kevin Collins
Guest
Posts: n/a
 
      12-24-2003
(E-Mail Removed) (Sara) wrote in message news:<(E-Mail Removed). com>...
> (E-Mail Removed) (Woogie) wrote in message news:<(E-Mail Removed). com>...
> > When running the sample code below without a proxy the GET returns the
> > expected data. When run with the $proxy uncommented the GET returns
> > the content of the login page for the site being accessed. The site
> > in the code is valid for ease of testing. I also am including the LWP
> > debug info for each attempt.
> >
> > Can anyone explain this behavior and what can I do to correct it?
> >
> > Thanks in advance
> >


-snip-
>
> might want to try on one of the cgi groups or even comp.perl.modules ?


Why? This has nothing (necessarily) to do with CGI. CGI is for web
clients and web servers talking to each other. And although LWP is
actually a web client, the example code does not appear related to any
CGI...

Kevin
 
Reply With Quote
 
Kevin Collins
Guest
Posts: n/a
 
      12-24-2003
(E-Mail Removed) (Woogie) wrote in message news:<(E-Mail Removed). com>...
> When running the sample code below without a proxy the GET returns the
> expected data. When run with the $proxy uncommented the GET returns
> the content of the login page for the site being accessed. The site
> in the code is valid for ease of testing. I also am including the LWP
> debug info for each attempt.
>
> Can anyone explain this behavior and what can I do to correct it?
>
> Thanks in advance
>
> Trace without proxy:
>
> LWP::UserAgent::new: ()
> LWP::UserAgent::request: ()
> LWP::UserAgent::send_request: GET
> https://squid.servebeer.com/getservi...ing&format=csv
> LWP::UserAgent::_need_proxy: Not proxied
> LWP:rotocol::http::request: ()
> LWP:rotocol::collect: read 28 bytes
> LWP::UserAgent::request: Simple response: OK
>
> Home
> PE
>
>
> Trace with proxy:
>
> LWP::UserAgent::new: ()
> LWP::UserAgent:roxy: https http://148.245.207.85:8080
> LWP::UserAgent::request: ()
> LWP::UserAgent::send_request: GET
> https://squid.servebeer.com/getservi...ing&format=csv
> LWP::UserAgent::_need_proxy: Proxied to http://148.245.207.85:8080
> LWP:rotocol::http::request: ()
> LWP:rotocol::collect: read 236 bytes
> LWP:rotocol::collect: read 594 bytes
> LWP:rotocol::collect: read 416 bytes
> LWP:rotocol::collect: read 450 bytes
> LWP:rotocol::collect: read 1017 bytes
> LWP:rotocol::collect: read 443 bytes
> LWP:rotocol::collect: read 643 bytes
> LWP::UserAgent::request: Simple response: OK
>
> <html lang="en">
>
> <!-- Start Head -->
> <head>
> <title>
>
> Error
>
> </title>
> <script language="JavaScript">
> ...
>
>
>
> Here is the sample code:
>
>
> #!/usr/bin/perl -w
>
> use LWP::UserAgent;
> use HTTP::Request;
> use HTTP::Response;
> use Crypt::SSLeay;
>
> LWP:ebug::level('+');
>
> $url = "https://squid.servebeer.com/getservices.do?user=Guest&password=JustLooking&for mat=csv";
> #$proxy="http://xxx.xxx.xxx.xxx:8080";
>
> $ua = LWP::UserAgent->new();
>
> if (defined $proxy)
> {
> $ENV{HTTPS_PROXY} = $proxy;
>
> # initialize from environment variables
> $ua->env_proxy;
> }
>
> $req = HTTP::Request->new(GET => $url);
> $response = $ua->request($req);
> if ($response->is_error())
> {
> printf " %s\n", $response->status_line;
> }
>
> else
> {
> $content = $response->content();
> print $content;
> }
>
> exit;


Your code has the definition of $proxy commented out, so I am not sure
your example is relevant to your output.

BUT, if it were uncommented, you are not including the ID and password
in the proxy URL (as you are in the "real" URL), so it makes some
sense that are being prompted for it.

Additionally, why do you want to redirect your request through a proxy
when you can get to it directly?

Kevin
 
Reply With Quote
 
Woogie
Guest
Posts: n/a
 
      12-27-2003
(E-Mail Removed) (Kevin Collins) wrote in message news:<(E-Mail Removed). com>...
> (E-Mail Removed) (Woogie) wrote in message news:<(E-Mail Removed). com>...
> > When running the sample code below without a proxy the GET returns the
> > expected data. When run with the $proxy uncommented the GET returns
> > the content of the login page for the site being accessed. The site
> > in the code is valid for ease of testing. I also am including the LWP
> > debug info for each attempt.
> >
> > Can anyone explain this behavior and what can I do to correct it?
> >
> > Thanks in advance
> >
> > Trace without proxy:
> >
> > LWP::UserAgent::new: ()
> > LWP::UserAgent::request: ()
> > LWP::UserAgent::send_request: GET
> > https://squid.servebeer.com/getservi...ing&format=csv
> > LWP::UserAgent::_need_proxy: Not proxied
> > LWP:rotocol::http::request: ()
> > LWP:rotocol::collect: read 28 bytes
> > LWP::UserAgent::request: Simple response: OK
> >
> > Home
> > PE
> >
> >
> > Trace with proxy:
> >
> > LWP::UserAgent::new: ()
> > LWP::UserAgent:roxy: https http://148.245.207.85:8080
> > LWP::UserAgent::request: ()
> > LWP::UserAgent::send_request: GET
> > https://squid.servebeer.com/getservi...ing&format=csv
> > LWP::UserAgent::_need_proxy: Proxied to http://148.245.207.85:8080
> > LWP:rotocol::http::request: ()
> > LWP:rotocol::collect: read 236 bytes
> > LWP:rotocol::collect: read 594 bytes
> > LWP:rotocol::collect: read 416 bytes
> > LWP:rotocol::collect: read 450 bytes
> > LWP:rotocol::collect: read 1017 bytes
> > LWP:rotocol::collect: read 443 bytes
> > LWP:rotocol::collect: read 643 bytes
> > LWP::UserAgent::request: Simple response: OK
> >
> > <html lang="en">
> >
> > <!-- Start Head -->
> > <head>
> > <title>
> >
> > Error
> >
> > </title>
> > <script language="JavaScript">
> > ...
> >
> >
> >
> > Here is the sample code:
> >
> >
> > #!/usr/bin/perl -w
> >
> > use LWP::UserAgent;
> > use HTTP::Request;
> > use HTTP::Response;
> > use Crypt::SSLeay;
> >
> > LWP:ebug::level('+');
> >
> > $url = "https://squid.servebeer.com/getservices.do?user=Guest&password=JustLooking&for mat=csv";
> > #$proxy="http://xxx.xxx.xxx.xxx:8080";
> >
> > $ua = LWP::UserAgent->new();
> >
> > if (defined $proxy)
> > {
> > $ENV{HTTPS_PROXY} = $proxy;
> >
> > # initialize from environment variables
> > $ua->env_proxy;
> > }
> >
> > $req = HTTP::Request->new(GET => $url);
> > $response = $ua->request($req);
> > if ($response->is_error())
> > {
> > printf " %s\n", $response->status_line;
> > }
> >
> > else
> > {
> > $content = $response->content();
> > print $content;
> > }
> >
> > exit;

>
> Your code has the definition of $proxy commented out, so I am not sure
> your example is relevant to your output.
>
> BUT, if it were uncommented, you are not including the ID and password
> in the proxy URL (as you are in the "real" URL), so it makes some
> sense that are being prompted for it.
>
> Additionally, why do you want to redirect your request through a proxy
> when you can get to it directly?
>
> Kevin



As stated in the comments at the beginning I provided output with and
without the $proxy set. The example trace outputs were derived from
two different environments. The first where a proxy was not required
and the second where one was.

As far as the ID and password are concerned, the code was intended to
support multiple environments, the proxy I was behind for the example
did not require ID or password.

While it is true that the request can be accessed in an open
environment directly, there are times when the code will be run in an
environment that has a proxy to the web. I've tried to write a
relatively generic solution.

Kent
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Proxy and LWP::UserAgent Mike Perl Misc 1 06-30-2005 06:03 PM
sanity checking proxy URL before passing to LWP buildmorelines Perl Misc 2 11-06-2004 05:02 PM
Perl LWP content request behind firewall and proxy server demonhunter Perl 3 08-16-2004 05:01 PM
LWP with proxy problem Woogie Perl 1 12-22-2003 07:51 PM
How to use proxy in Net::HTTP, not in LWP::UserAgent? Great Deals Perl Misc 1 10-02-2003 07:27 PM



Advertisments