Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Perl > Perl Misc > Automatically save a webpage as a text file

Reply
Thread Tools

Automatically save a webpage as a text file

 
 
Chris Petersen
Guest
Posts: n/a
 
      08-20-2003
OS: XP Pro with IE 6.0

Every morning I come in, open 2 websites, and save their content as
TXT files, one is a TAB file the other is a CSV file.

Example addresses:
http://somesite/reports/date.tab
http://somesite/reports/date.csv

I then save them as:
http://somesite/reports/date.tab.txt
http://somesite/reports/date.csv.txt

I would like to write a program to automate this, I was wondering
which language would be best, and maybe get a couple of quick and
dirty examples.

Thanks in advance
 
Reply With Quote
 
 
 
 
Wolfgang 'Dreamguard' Nagele
Guest
Posts: n/a
 
      08-21-2003
> I would like to write a program to automate this, I was wondering
> which language would be best, and maybe get a couple of quick and
> dirty examples.

as you describe i think you got shell access to that machine?
if so - just make a cronjob (google be your friend) and simply copy those
files with bash 'cp' command.

yours, dreamguard.


 
Reply With Quote
 
 
 
 
Ted Davis
Guest
Posts: n/a
 
      08-21-2003
On 20 Aug 2003 16:44:24 -0700, http://www.velocityreviews.com/forums/(E-Mail Removed) (Chris
Petersen) wrote:

>OS: XP Pro with IE 6.0
>
>Every morning I come in, open 2 websites, and save their content as
>TXT files, one is a TAB file the other is a CSV file.
>
>Example addresses:
>http://somesite/reports/date.tab
>http://somesite/reports/date.csv
>
>I then save them as:
>http://somesite/reports/date.tab.txt
>http://somesite/reports/date.csv.txt
>
>I would like to write a program to automate this, I was wondering
>which language would be best, and maybe get a couple of quick and
>dirty examples.
>
>Thanks in advance



Either wget or Lynx will do this as a simple command

wget http://somesite/reports/date.tab
wget http://somesite/reports/date.csv

lynx -dump http://somesite/reports/date.tab > date.tab
lynx -dump http://somesite/reports/date.csv > date.csv


<http://unxutils.sourceforge.net/> and <http://lynx.isc.org/release/>

You can put the commands in a batch file and do the whole thing with
one click. Of you log in each morning, you can put the batch file in
your startup folder, though you might want to check their dates before
downloading, even though double downloads (in case of reboot) would
not likely be a problem.


T.E.D. ((E-Mail Removed) - e-mail must contain "T.E.D." or my .sig in the body)
 
Reply With Quote
 
Tad McClellan
Guest
Posts: n/a
 
      08-21-2003
Chris Petersen <(E-Mail Removed)> wrote:

> open 2 websites, and save their content as
> TXT files



> I would like to write a program to automate this, I was wondering
> which language would be best, and maybe get a couple of quick and
> dirty examples.



Here it is in Perl (untested):

use LWP::Simple;
getstore( 'http://somesite/reports/date.tab',
'http://somesite/reports/date.tab.txt' );
getstore( 'http://somesite/reports/date.csv',
'http://somesite/reports/date.csv.txt' );


--
Tad McClellan SGML consulting
(E-Mail Removed) Perl programming
Fort Worth, Texas
 
Reply With Quote
 
Bob Walton
Guest
Posts: n/a
 
      08-21-2003
Chris Petersen wrote:

> OS: XP Pro with IE 6.0
>
> Every morning I come in, open 2 websites, and save their content as
> TXT files, one is a TAB file the other is a CSV file.
>
> Example addresses:
> http://somesite/reports/date.tab
> http://somesite/reports/date.csv
>
> I then save them as:
> http://somesite/reports/date.tab.txt
> http://somesite/reports/date.csv.txt
>
> I would like to write a program to automate this, I was wondering
> which language would be best, and maybe get a couple of quick and
> dirty examples.

....


Assuming you want to do it in Perl since you asked here:

#!usr/bin/perl -w
use strict;
use LWP::Simple;
getstore($ARGV[0],$ARGV[1]);

Invoke as something like:

perl file.pl http://xyz.com/path/etc file_to_store_it.in

Fancy it up with checks to see if the arguments were supplied, and if
the web page was successfully retrieved and stored.

--
Bob Walton

 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
How to save a webpage contents to a file ( with LWP ) Jack Perl Misc 6 02-21-2008 12:50 AM
how to automatically "Save " a page after certain intervals without clicking "Save Page As..." subhadip Java 0 03-28-2007 04:15 PM
How to Save a Webpage as a local file in ASP.net ? Kylin ASP .Net 1 05-16-2005 03:01 AM
Can I find text automatically in a linked webpage Les Juby Javascript 1 11-25-2004 10:11 AM
how to save the out html to a html file on server disk automatically ? sincethe2003 ASP .Net 2 07-14-2004 05:18 PM



Advertisments