Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Perl > Perl Misc > Storing downloaded webpages into files

Reply
Thread Tools

Storing downloaded webpages into files

 
 
PuZzLeD
Guest
Posts: n/a
 
      05-19-2004
Hi,
I have a problem here. I am supposed to be saving downloaded web pages
on to a a local cache. I have written a forking proxy server that
reads the request frm the browser and passes the request to the
Internet Gateway and then gets back the data from the internet gateway
and then passes it on to the browser. I need to save the files that
are associated with each request into a local cache. I have attached
the server code below...


server_listenport = IO::Socket::INET->new(
LocalPort => $server_listenport,
Listen => $listen_queuesize)
or die "couldnt be a tcp server on port $server_listenport:
$@\n";


while($browser_request=$server_listenport->accept())
{
if (fork == 0)
{
#child
print " \n\nUsing Child Proceess...\n";
close ($server_listenport);
sysread($browser_request,$req_webpage,204;
$date_stamp= `date`;
$internet_gateway =IO::Socket::INET->new(
PeerPort=>$IGway_listenport,
PeerAddr=>$IGway_hostname) or die " cannot
connect to gateway";

syswrite($internet_gateway,$req_webpage);

while(sysread($internet_gateway, $recv_webpage, 20480) > 0)
{
syswrite($browser_request,$recv_webpage);
}
print "\n\n\tWeb page is now downloaded!!\n\n";
close($internet_gateway);

exit;
}
}
else
{
#parent
print " \n\nUsing Parent Process...\n";
close($browser_request);
}
} #end of while


Can some1 help me ?
 
Reply With Quote
 
 
 
 
Ben Morrow
Guest
Posts: n/a
 
      05-19-2004

Quoth http://www.velocityreviews.com/forums/(E-Mail Removed) (PuZzLeD):
> Hi,
> I have a problem here. I am supposed to be saving downloaded web pages
> on to a a local cache. I have written a forking proxy server that
> reads the request frm the browser and passes the request to the
> Internet Gateway and then gets back the data from the internet gateway
> and then passes it on to the browser. I need to save the files that
> are associated with each request into a local cache. I have attached
> the server code below...


What is your problem? You don't know how to save the data?

You should rewrite this using HTTP:aemon and LWP, or indeed
HTTP:roxy.

Ben

--
The cosmos, at best, is like a rubbish heap scattered at random.
- Heraclitus
(E-Mail Removed)
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
User Images: Storing in Files VS Storing in Database Jonathan Wood ASP .Net 1 06-02-2008 05:56 PM
Where is Redlasso video site storing files? Not found in temp.internet files. Doc Computer Support 1 03-12-2008 07:00 AM
Problem in downloading js files specified in our webpages Venkatesh Java 4 03-01-2007 06:58 PM
storing pointer vs storing object toton C++ 11 10-13-2006 11:08 AM
Mozilla Adds ".jfif" Extension to all JPEG Files Downloaded? cvnorris@charterinternet.com Firefox 1 10-28-2003 05:36 AM



Advertisments