Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Perl > Perl Misc > Opening files on the web for reading

Reply
Thread Tools

Opening files on the web for reading

 
 
Graham Stow
Guest
Posts: n/a
 
      09-24-2008
Can anyone give me some Perl code to open an html file on the web (i.e. an
html file stored on somebody elses web server and not mine), for reading. Or
is it more complicated than that?


 
Reply With Quote
 
 
 
 
Peter Makholm
Guest
Posts: n/a
 
      09-24-2008
"Graham Stow" <(E-Mail Removed)> writes:

> Can anyone give me some Perl code to open an html file on the web (i.e. an
> html file stored on somebody elses web server and not mine), for reading. Or
> is it more complicated than that?


You can use the LWP::Simple module. The example in the documentation
should tell you how to do it.

//Makholm
 
Reply With Quote
 
 
 
 
Jürgen Exner
Guest
Posts: n/a
 
      09-24-2008
"Graham Stow" <(E-Mail Removed)> wrote:
>Can anyone give me some Perl code to open an html file on the web (i.e. an
>html file stored on somebody elses web server and not mine), for reading. Or
>is it more complicated than that?


Is there anything wrong with the answer in "perldoc -q HTML":

How do I fetch an HTML file?

jue
 
Reply With Quote
 
xhoster@gmail.com
Guest
Posts: n/a
 
      09-24-2008
Jürgen Exner <(E-Mail Removed)> wrote:
> "Graham Stow" <(E-Mail Removed)> wrote:
> >Can anyone give me some Perl code to open an html file on the web (i.e.
> >an html file stored on somebody elses web server and not mine), for
> >reading. Or is it more complicated than that?

>
> Is there anything wrong with the answer in "perldoc -q HTML":
>
> How do I fetch an HTML file?


Other than it not answering the question? At least on my Perl version,
none of the answers there return a file handle opened for reading. Now
maybe he is fine with downloading the entire file (either to disk or to
memory) and then reading from that, but I'd be inclined to give the benefit
of the doubt that he meant what he asked.

LWP::UserAgent using a callback with for example :content_cb would "stream"
the data back, but not via a file handle. One could probably come up with
an adaptor that ties a file handle front end to the callback backend.

There might be a more direct way, but I don't know what it is.




Xho

--
-------------------- http://NewsReader.Com/ --------------------
The costs of publication of this article were defrayed in part by the
payment of page charges. This article must therefore be hereby marked
advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate
this fact.
 
Reply With Quote
 
Jürgen Exner
Guest
Posts: n/a
 
      09-24-2008
http://www.velocityreviews.com/forums/(E-Mail Removed) wrote:
>Jürgen Exner <(E-Mail Removed)> wrote:
>> "Graham Stow" <(E-Mail Removed)> wrote:
>> >Can anyone give me some Perl code to open an html file on the web (i.e.
>> >an html file stored on somebody elses web server and not mine), for
>> >reading. Or is it more complicated than that?

>>
>> Is there anything wrong with the answer in "perldoc -q HTML":
>>
>> How do I fetch an HTML file?

>
>Other than it not answering the question? At least on my Perl version,
>none of the answers there return a file handle opened for reading. Now
>maybe he is fine with downloading the entire file (either to disk or to
>memory) and then reading from that, but I'd be inclined to give the benefit
>of the doubt that he meant what he asked.


Fair enough. I interpreted "to open an html file on the web [...] for
reading" as he just wants to get he content of that file (which as we
all know may not be a file in the first place), not to actually have a
read file handle to a URL.
At the very least his terminology is sloppy and your interpretation may
very well be closer to his intentions.

jue
 
Reply With Quote
 
Ben Morrow
Guest
Posts: n/a
 
      09-24-2008

Quoth (E-Mail Removed):
> Jürgen Exner <(E-Mail Removed)> wrote:
> > "Graham Stow" <(E-Mail Removed)> wrote:
> > >Can anyone give me some Perl code to open an html file on the web (i.e.
> > >an html file stored on somebody elses web server and not mine), for
> > >reading. Or is it more complicated than that?

> >
> > Is there anything wrong with the answer in "perldoc -q HTML":
> >
> > How do I fetch an HTML file?

>
> Other than it not answering the question? At least on my Perl version,
> none of the answers there return a file handle opened for reading. Now
> maybe he is fine with downloading the entire file (either to disk or to
> memory) and then reading from that, but I'd be inclined to give the benefit
> of the doubt that he meant what he asked.
>
> LWP::UserAgent using a callback with for example :content_cb would "stream"
> the data back, but not via a file handle. One could probably come up with
> an adaptor that ties a file handle front end to the callback backend.
>
> There might be a more direct way, but I don't know what it is.


IO::All::LWP

Ben

--
The Earth is degenerating these days. Bribery and corruption abound.
Children no longer mind their parents, every man wants to write a book,
and it is evident that the end of the world is fast approaching.
Assyrian stone tablet, c.2800 BC (E-Mail Removed)
 
Reply With Quote
 
Tim Greer
Guest
Posts: n/a
 
      09-24-2008
Graham Stow wrote:

> Can anyone give me some Perl code to open an html file on the web
> (i.e. an html file stored on somebody elses web server and not mine),
> for reading. Or is it more complicated than that?


Are you just looking to read it and maybe check something, or parse it,
or download it/save it? There are many methods, but the best one could
depend on what your goals are.
--
Tim Greer, CEO/Founder/CTO, BurlyHost.com, Inc.
Shared Hosting, Reseller Hosting, Dedicated & Semi-Dedicated servers
and Custom Hosting. 24/7 support, 30 day guarantee, secure servers.
Industry's most experienced staff! -- Web Hosting With Muscle!
 
Reply With Quote
 
C.DeRykus
Guest
Posts: n/a
 
      09-25-2008
On Sep 24, 8:47 am, (E-Mail Removed) wrote:
> ...
>
> LWP::UserAgent using a callback with for example :content_cb would "stream"
> the data back, but not via a file handle. One could probably come up with
> an adaptor that ties a file handle front end to the callback backend.
>
> There might be a more direct way, but I don't know what it is.
>

S
Another possibility but still indirect
(and w/o graceful error handling):

use LWP::Simple;
my $pid = open( my $fh, "-|" );
die "fork: $!" unless defined $pid;
if ($pid ) { while <$fh> { ... } }
else { getprint( ...); }
....



--
Charles DeRykus
 
Reply With Quote
 
Ted Zlatanov
Guest
Posts: n/a
 
      09-25-2008
On Wed, 24 Sep 2008 18:25:48 +0100 Ben Morrow <(E-Mail Removed)> wrote:

BM> Quoth (E-Mail Removed):
>> Jürgen Exner <(E-Mail Removed)> wrote:
>> > "Graham Stow" <(E-Mail Removed)> wrote:
>> > >Can anyone give me some Perl code to open an html file on the web (i.e.
>> > >an html file stored on somebody elses web server and not mine), for
>> > >reading. Or is it more complicated than that?
>> >
>> > Is there anything wrong with the answer in "perldoc -q HTML":
>> >
>> > How do I fetch an HTML file?

>>
>> Other than it not answering the question? At least on my Perl version,
>> none of the answers there return a file handle opened for reading. Now
>> maybe he is fine with downloading the entire file (either to disk or to
>> memory) and then reading from that, but I'd be inclined to give the benefit
>> of the doubt that he meant what he asked.
>>
>> LWP::UserAgent using a callback with for example :content_cb would "stream"
>> the data back, but not via a file handle. One could probably come up with
>> an adaptor that ties a file handle front end to the callback backend.
>>
>> There might be a more direct way, but I don't know what it is.


BM> IO::All::LWP

Unfortunately, the docs say "The bad news is that the whole file is
stored in memory after getting it or before putting it. This may cause
problems if you are dealing with multi-gigabyte files!"

It would be nice to have a buffered reader/writer which wouldn't grab
the whole file, using the LWP callbacks, as xhoster suggests... I
haven't seen such a module.

Ted
 
Reply With Quote
 
xhoster@gmail.com
Guest
Posts: n/a
 
      09-25-2008
Ted Zlatanov <(E-Mail Removed)> wrote:
> On Wed, 24 Sep 2008 18:25:48 +0100 Ben Morrow <(E-Mail Removed)> wrote:
>
> BM> IO::All::LWP
>
> Unfortunately, the docs say "The bad news is that the whole file is
> stored in memory after getting it or before putting it. This may cause
> problems if you are dealing with multi-gigabyte files!"
>
> It would be nice to have a buffered reader/writer which wouldn't grab
> the whole file, using the LWP callbacks, as xhoster suggests... I
> haven't seen such a module.


And it doesn't seem as easy as I thought. In order for the callback to be
invoked, the thing invoking the callback has to be "in control". But to
read from a file handle, the thing reading is in control. You'd have to
fork a process and in one have the callback invoker in control, streaming
data to the other process as it comes in and the callback is invoked. So
then you would have portability problems.

It seems like it is easy to write a wrapper that turns an iterator into a
callback, but vice versa is not easy.

Xho

--
-------------------- http://NewsReader.Com/ --------------------
The costs of publication of this article were defrayed in part by the
payment of page charges. This article must therefore be hereby marked
advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate
this fact.
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
UnauthorizedAccessException when reading XML files (no problem when reading other file-types) blabla120@gmx.net ASP .Net 0 09-15-2006 02:08 PM
Anyone resolved links opening blank page? or box opening in Outlook 2000? S.Rodgers Firefox 13 12-16-2005 03:07 PM
Word file not opening from Adm -My recent docs but opening from file menu of Word Thaqalain Computer Support 0 06-30-2005 02:20 AM
need some help with dial box opening when opening a folder John Computer Information 0 11-13-2003 08:38 PM
Reading WORD files without opening? Serendipity Computer Support 2 10-19-2003 06:07 PM



Advertisments