«

I'm trying to access a site with data that needs to be paged through, one
page at a time. It won't allow back buttons and you have to use the menu
links to get through. In other words, it is run through CGI (ASP, I think)
and must be able to keep track of sessions.

I have a simple program I've written using Perl and LWP. I tried a totally
innocuous site (TVLand.com) that I found had cookies. When I read in 1
page from this site, I got a cookie and it showed up in lwpcookies.txt.
Whenever I try this other site (which requires a password and account, so I
can't list it here, according to registration agreement -- and no, it isn't
pr0n!), I find that there is a line in the header to set a cookie in both
web pages. The only difference is that the one where the cookie isn't
storied doesn't list a domain name in the cookie line. Here's the header
lines with the cookie info (1st is TVLand, 2nd is private site):

Set-Cookie: JSESSIONID=M1LHYKX2DVXRMCQBAFML3UQ; domain=.tvland.com; path=/
Set-Cookie: JSESSIONID=0000CSPVV3Q5TXU2BUPIIDEWOCY:ulnfn1uq;Pa th=/

my program (listed below) prints out that I have a cookie. (Whenever I try
the site with Galeon, I can read a cookie for the site, too), but no cookie
shows up in lwpcookie.txt (the cookie file) at all.

Am I doing something wrong? I need to be sure that the cookie is persistant
in my program, but it never shows up in the cookie file. Is there a reason
for that, or am I doing something wrong? What can I do to make sure the
cookie from the 2nd site is stored for later -- and also read back when
needed by other pages?

Thanks!

Hal
--------------------------------------------------------------------
Program listing:

use LWP::UserAgent;
use HTTP::Cookies;

our $domain = "tvland.com";
our $locmenu = "schedule";
our $ua = LWP::UserAgent->new;
$ua->agent("Mozilla/4.0");
$ua->agent("MSIE/6.0");
$ua->cookie_jar(HTTP::Cookies->new(file =>"lwpcookies.txt", autosave =>
1));
$url = "HTTP://".$domain."/".$locmenu;
print "Url: $url\n";
$req = HTTP::Request->new(GET => $url);
$req->content_type("application/x-www-form-urlencoded");
$req->header('Accept' => 'text/html');
$res = $ua->request($req);
print "Cookie: ".$res->status_line."\n";
$page = $res->as_string;
# print "Page: $page\n";

Hal Vaughan wrote:
> I'm trying to access a site with data that needs to be paged through, one
> page at a time. It won't allow back buttons and you have to use the menu
> links to get through. In other words, it is run through CGI (ASP, I think)
> and must be able to keep track of sessions.
>
> I have a simple program I've written using Perl and LWP. I tried a totally
> innocuous site (TVLand.com) that I found had cookies. When I read in 1
> page from this site, I got a cookie and it showed up in lwpcookies.txt.
> Whenever I try this other site (which requires a password and account, so I
> can't list it here, according to registration agreement -- and no, it isn't
> pr0n!), I find that there is a line in the header to set a cookie in both
> web pages. The only difference is that the one where the cookie isn't
> storied doesn't list a domain name in the cookie line. Here's the header
> lines with the cookie info (1st is TVLand, 2nd is private site):
>
> Set-Cookie: JSESSIONID=M1LHYKX2DVXRMCQBAFML3UQ; domain=.tvland.com; path=/
> Set-Cookie: JSESSIONID=0000CSPVV3Q5TXU2BUPIIDEWOCY:ulnfn1uq;Pa th=/
>
> my program (listed below) prints out that I have a cookie. (Whenever I try
> the site with Galeon, I can read a cookie for the site, too), but no cookie
> shows up in lwpcookie.txt (the cookie file) at all.
>
> Am I doing something wrong? I need to be sure that the cookie is persistant
> in my program, but it never shows up in the cookie file. Is there a reason
> for that, or am I doing something wrong? What can I do to make sure the
> cookie from the 2nd site is stored for later -- and also read back when
> needed by other pages?
>
> Thanks!
>
> Hal
> --------------------------------------------------------------------
> Program listing:
>
> use LWP::UserAgent;
> use HTTP::Cookies;
>
> our $domain = "tvland.com";
> our $locmenu = "schedule";
> our $ua = LWP::UserAgent->new;
> $ua->agent("Mozilla/4.0");
> $ua->agent("MSIE/6.0");
> $ua->cookie_jar(HTTP::Cookies->new(file =>"lwpcookies.txt", autosave =>
> 1));
> $url = "HTTP://".$domain."/".$locmenu;
> print "Url: $url\n";
> $req = HTTP::Request->new(GET => $url);
> $req->content_type("application/x-www-form-urlencoded");
> $req->header('Accept' => 'text/html');
> $res = $ua->request($req);
> print "Cookie: ".$res->status_line."\n";
> $page = $res->as_string;
> # print "Page: $page\n";

$ua = LWP::UserAgent->new( cookie_jar =>HTTP::Cookies->new( file =>
'/tmp/cookies.txt', autosave => 1, ignore_discard => 1 ));

try ignore_discard => 1