Velocity Reviews

Velocity Reviews (http://www.velocityreviews.com/forums/index.php)
-   Perl Misc (http://www.velocityreviews.com/forums/f67-perl-misc.html)
-   -   LWP with proxy problem (http://www.velocityreviews.com/forums/t884322-lwp-with-proxy-problem.html)

Woogie 12-23-2003 08:43 PM

LWP with proxy problem
 
When running the sample code below without a proxy the GET returns the
expected data. When run with the $proxy uncommented the GET returns
the content of the login page for the site being accessed. The site
in the code is valid for ease of testing. I also am including the LWP
debug info for each attempt.

Can anyone explain this behavior and what can I do to correct it?

Thanks in advance

Trace without proxy:

LWP::UserAgent::new: ()
LWP::UserAgent::request: ()
LWP::UserAgent::send_request: GET
https://squid.servebeer.com/getservi...ing&format=csv
LWP::UserAgent::_need_proxy: Not proxied
LWP::Protocol::http::request: ()
LWP::Protocol::collect: read 28 bytes
LWP::UserAgent::request: Simple response: OK

Home
PE


Trace with proxy:

LWP::UserAgent::new: ()
LWP::UserAgent::proxy: https http://148.245.207.85:8080
LWP::UserAgent::request: ()
LWP::UserAgent::send_request: GET
https://squid.servebeer.com/getservi...ing&format=csv
LWP::UserAgent::_need_proxy: Proxied to http://148.245.207.85:8080
LWP::Protocol::http::request: ()
LWP::Protocol::collect: read 236 bytes
LWP::Protocol::collect: read 594 bytes
LWP::Protocol::collect: read 416 bytes
LWP::Protocol::collect: read 450 bytes
LWP::Protocol::collect: read 1017 bytes
LWP::Protocol::collect: read 443 bytes
LWP::Protocol::collect: read 643 bytes
LWP::UserAgent::request: Simple response: OK

<html lang="en">

<!-- Start Head -->
<head>
<title>

Error

</title>
<script language="JavaScript">
....



Here is the sample code:


#!/usr/bin/perl -w

use LWP::UserAgent;
use HTTP::Request;
use HTTP::Response;
use Crypt::SSLeay;

LWP::Debug::level('+');

$url = "https://squid.servebeer.com/getservices.do?user=Guest&password=JustLooking&for mat=csv";
#$proxy="http://xxx.xxx.xxx.xxx:8080";

$ua = LWP::UserAgent->new();

if (defined $proxy)
{
$ENV{HTTPS_PROXY} = $proxy;

# initialize from environment variables
$ua->env_proxy;
}

$req = HTTP::Request->new(GET => $url);
$response = $ua->request($req);
if ($response->is_error())
{
printf " %s\n", $response->status_line;
}

else
{
$content = $response->content();
print $content;
}

exit;

Sara 12-24-2003 01:59 PM

Re: LWP with proxy problem
 
kent_zunker@bmc.com (Woogie) wrote in message news:<4a31f52f.0312231243.25828446@posting.google. com>...
> When running the sample code below without a proxy the GET returns the
> expected data. When run with the $proxy uncommented the GET returns
> the content of the login page for the site being accessed. The site
> in the code is valid for ease of testing. I also am including the LWP
> debug info for each attempt.
>
> Can anyone explain this behavior and what can I do to correct it?
>
> Thanks in advance
>
> Trace without proxy:
>
> LWP::UserAgent::new: ()
> LWP::UserAgent::request: ()
> LWP::UserAgent::send_request: GET
> https://squid.servebeer.com/getservi...ing&format=csv
> LWP::UserAgent::_need_proxy: Not proxied
> LWP::Protocol::http::request: ()
> LWP::Protocol::collect: read 28 bytes
> LWP::UserAgent::request: Simple response: OK
>
> Home
> PE
>
>
> Trace with proxy:
>
> LWP::UserAgent::new: ()
> LWP::UserAgent::proxy: https http://148.245.207.85:8080
> LWP::UserAgent::request: ()
> LWP::UserAgent::send_request: GET
> https://squid.servebeer.com/getservi...ing&format=csv
> LWP::UserAgent::_need_proxy: Proxied to http://148.245.207.85:8080
> LWP::Protocol::http::request: ()
> LWP::Protocol::collect: read 236 bytes
> LWP::Protocol::collect: read 594 bytes
> LWP::Protocol::collect: read 416 bytes
> LWP::Protocol::collect: read 450 bytes
> LWP::Protocol::collect: read 1017 bytes
> LWP::Protocol::collect: read 443 bytes
> LWP::Protocol::collect: read 643 bytes
> LWP::UserAgent::request: Simple response: OK
>
> <html lang="en">
>
> <!-- Start Head -->
> <head>
> <title>
>
> Error
>
> </title>
> <script language="JavaScript">
> ...
>
>
>
> Here is the sample code:
>
>
> #!/usr/bin/perl -w
>
> use LWP::UserAgent;
> use HTTP::Request;
> use HTTP::Response;
> use Crypt::SSLeay;
>
> LWP::Debug::level('+');
>
> $url = "https://squid.servebeer.com/getservices.do?user=Guest&password=JustLooking&for mat=csv";
> #$proxy="http://xxx.xxx.xxx.xxx:8080";
>
> $ua = LWP::UserAgent->new();
>
> if (defined $proxy)
> {
> $ENV{HTTPS_PROXY} = $proxy;
>
> # initialize from environment variables
> $ua->env_proxy;
> }
>
> $req = HTTP::Request->new(GET => $url);
> $response = $ua->request($req);
> if ($response->is_error())
> {
> printf " %s\n", $response->status_line;
> }
>
> else
> {
> $content = $response->content();
> print $content;
> }
>
> exit;


might want to try on one of the cgi groups or even comp.perl.modules ?

Kevin Collins 12-24-2003 07:21 PM

Re: LWP with proxy problem
 
genericax@hotmail.com (Sara) wrote in message news:<776e0325.0312240559.613c4dd4@posting.google. com>...
> kent_zunker@bmc.com (Woogie) wrote in message news:<4a31f52f.0312231243.25828446@posting.google. com>...
> > When running the sample code below without a proxy the GET returns the
> > expected data. When run with the $proxy uncommented the GET returns
> > the content of the login page for the site being accessed. The site
> > in the code is valid for ease of testing. I also am including the LWP
> > debug info for each attempt.
> >
> > Can anyone explain this behavior and what can I do to correct it?
> >
> > Thanks in advance
> >


-snip-
>
> might want to try on one of the cgi groups or even comp.perl.modules ?


Why? This has nothing (necessarily) to do with CGI. CGI is for web
clients and web servers talking to each other. And although LWP is
actually a web client, the example code does not appear related to any
CGI...

Kevin

Kevin Collins 12-24-2003 07:32 PM

Re: LWP with proxy problem
 
kent_zunker@bmc.com (Woogie) wrote in message news:<4a31f52f.0312231243.25828446@posting.google. com>...
> When running the sample code below without a proxy the GET returns the
> expected data. When run with the $proxy uncommented the GET returns
> the content of the login page for the site being accessed. The site
> in the code is valid for ease of testing. I also am including the LWP
> debug info for each attempt.
>
> Can anyone explain this behavior and what can I do to correct it?
>
> Thanks in advance
>
> Trace without proxy:
>
> LWP::UserAgent::new: ()
> LWP::UserAgent::request: ()
> LWP::UserAgent::send_request: GET
> https://squid.servebeer.com/getservi...ing&format=csv
> LWP::UserAgent::_need_proxy: Not proxied
> LWP::Protocol::http::request: ()
> LWP::Protocol::collect: read 28 bytes
> LWP::UserAgent::request: Simple response: OK
>
> Home
> PE
>
>
> Trace with proxy:
>
> LWP::UserAgent::new: ()
> LWP::UserAgent::proxy: https http://148.245.207.85:8080
> LWP::UserAgent::request: ()
> LWP::UserAgent::send_request: GET
> https://squid.servebeer.com/getservi...ing&format=csv
> LWP::UserAgent::_need_proxy: Proxied to http://148.245.207.85:8080
> LWP::Protocol::http::request: ()
> LWP::Protocol::collect: read 236 bytes
> LWP::Protocol::collect: read 594 bytes
> LWP::Protocol::collect: read 416 bytes
> LWP::Protocol::collect: read 450 bytes
> LWP::Protocol::collect: read 1017 bytes
> LWP::Protocol::collect: read 443 bytes
> LWP::Protocol::collect: read 643 bytes
> LWP::UserAgent::request: Simple response: OK
>
> <html lang="en">
>
> <!-- Start Head -->
> <head>
> <title>
>
> Error
>
> </title>
> <script language="JavaScript">
> ...
>
>
>
> Here is the sample code:
>
>
> #!/usr/bin/perl -w
>
> use LWP::UserAgent;
> use HTTP::Request;
> use HTTP::Response;
> use Crypt::SSLeay;
>
> LWP::Debug::level('+');
>
> $url = "https://squid.servebeer.com/getservices.do?user=Guest&password=JustLooking&for mat=csv";
> #$proxy="http://xxx.xxx.xxx.xxx:8080";
>
> $ua = LWP::UserAgent->new();
>
> if (defined $proxy)
> {
> $ENV{HTTPS_PROXY} = $proxy;
>
> # initialize from environment variables
> $ua->env_proxy;
> }
>
> $req = HTTP::Request->new(GET => $url);
> $response = $ua->request($req);
> if ($response->is_error())
> {
> printf " %s\n", $response->status_line;
> }
>
> else
> {
> $content = $response->content();
> print $content;
> }
>
> exit;


Your code has the definition of $proxy commented out, so I am not sure
your example is relevant to your output.

BUT, if it were uncommented, you are not including the ID and password
in the proxy URL (as you are in the "real" URL), so it makes some
sense that are being prompted for it.

Additionally, why do you want to redirect your request through a proxy
when you can get to it directly?

Kevin

Woogie 12-27-2003 03:21 PM

Re: LWP with proxy problem
 
spamtotrash@toomuchfiction.com (Kevin Collins) wrote in message news:<a6882f32.0312241132.55fb4a0f@posting.google. com>...
> kent_zunker@bmc.com (Woogie) wrote in message news:<4a31f52f.0312231243.25828446@posting.google. com>...
> > When running the sample code below without a proxy the GET returns the
> > expected data. When run with the $proxy uncommented the GET returns
> > the content of the login page for the site being accessed. The site
> > in the code is valid for ease of testing. I also am including the LWP
> > debug info for each attempt.
> >
> > Can anyone explain this behavior and what can I do to correct it?
> >
> > Thanks in advance
> >
> > Trace without proxy:
> >
> > LWP::UserAgent::new: ()
> > LWP::UserAgent::request: ()
> > LWP::UserAgent::send_request: GET
> > https://squid.servebeer.com/getservi...ing&format=csv
> > LWP::UserAgent::_need_proxy: Not proxied
> > LWP::Protocol::http::request: ()
> > LWP::Protocol::collect: read 28 bytes
> > LWP::UserAgent::request: Simple response: OK
> >
> > Home
> > PE
> >
> >
> > Trace with proxy:
> >
> > LWP::UserAgent::new: ()
> > LWP::UserAgent::proxy: https http://148.245.207.85:8080
> > LWP::UserAgent::request: ()
> > LWP::UserAgent::send_request: GET
> > https://squid.servebeer.com/getservi...ing&format=csv
> > LWP::UserAgent::_need_proxy: Proxied to http://148.245.207.85:8080
> > LWP::Protocol::http::request: ()
> > LWP::Protocol::collect: read 236 bytes
> > LWP::Protocol::collect: read 594 bytes
> > LWP::Protocol::collect: read 416 bytes
> > LWP::Protocol::collect: read 450 bytes
> > LWP::Protocol::collect: read 1017 bytes
> > LWP::Protocol::collect: read 443 bytes
> > LWP::Protocol::collect: read 643 bytes
> > LWP::UserAgent::request: Simple response: OK
> >
> > <html lang="en">
> >
> > <!-- Start Head -->
> > <head>
> > <title>
> >
> > Error
> >
> > </title>
> > <script language="JavaScript">
> > ...
> >
> >
> >
> > Here is the sample code:
> >
> >
> > #!/usr/bin/perl -w
> >
> > use LWP::UserAgent;
> > use HTTP::Request;
> > use HTTP::Response;
> > use Crypt::SSLeay;
> >
> > LWP::Debug::level('+');
> >
> > $url = "https://squid.servebeer.com/getservices.do?user=Guest&password=JustLooking&for mat=csv";
> > #$proxy="http://xxx.xxx.xxx.xxx:8080";
> >
> > $ua = LWP::UserAgent->new();
> >
> > if (defined $proxy)
> > {
> > $ENV{HTTPS_PROXY} = $proxy;
> >
> > # initialize from environment variables
> > $ua->env_proxy;
> > }
> >
> > $req = HTTP::Request->new(GET => $url);
> > $response = $ua->request($req);
> > if ($response->is_error())
> > {
> > printf " %s\n", $response->status_line;
> > }
> >
> > else
> > {
> > $content = $response->content();
> > print $content;
> > }
> >
> > exit;

>
> Your code has the definition of $proxy commented out, so I am not sure
> your example is relevant to your output.
>
> BUT, if it were uncommented, you are not including the ID and password
> in the proxy URL (as you are in the "real" URL), so it makes some
> sense that are being prompted for it.
>
> Additionally, why do you want to redirect your request through a proxy
> when you can get to it directly?
>
> Kevin



As stated in the comments at the beginning I provided output with and
without the $proxy set. The example trace outputs were derived from
two different environments. The first where a proxy was not required
and the second where one was.

As far as the ID and password are concerned, the code was intended to
support multiple environments, the proxy I was behind for the example
did not require ID or password.

While it is true that the request can be accessed in an open
environment directly, there are times when the code will be run in an
environment that has a proxy to the web. I've tried to write a
relatively generic solution.

Kent


All times are GMT. The time now is 09:08 PM.

Powered by vBulletin®. Copyright ©2000 - 2014, vBulletin Solutions, Inc.
SEO by vBSEO ©2010, Crawlability, Inc.