«

I'm use IO::Socket on linux to read from a webpage. I don't want to use
LWP::Simple. How would I find the EOF when the web server is finished
sending me the webpage? Here's what I'm using right now:

while (my $res = sysread($sock_in,$buf,$bufsize)) {
syswrite($sock_out,$buf,length($buf));
last if ($res < $bufsize);
}

It works, but obviously, if a webpage has a size that is a multiple of
$bufsize, this will loop forever.

Regards,
--
lucas
-------------------------
Perl Coder since 2001
shift || die;
-------------------------

Quoth :
> I'm use IO::Socket on linux to read from a webpage. I don't want to use
> LWP::Simple. How would I find the EOF when the web server is finished
> sending me the webpage? Here's what I'm using right now:
>
> while (my $res = sysread($sock_in,$buf,$bufsize)) {
> syswrite($sock_out,$buf,length($buf));
> last if ($res < $bufsize);
> }
>
> It works, but obviously, if a webpage has a size that is a multiple of
> $bufsize, this will loop forever.

I don't know what makes you think that. Did you try it? sysread returns
0 on EOF.

Note that you should also be checking defined($res), as sysread returns
undef on errors. Depending on how you've opened the socket and your
system, some errors (EINTR and EAGAIN, specifically) are not in fact
errors, but indications that you need to retry the read.

I think you need to read

perldoc -f sysread

and

man 2 read

before you go much further (the latter assumes you are on Unix; this is
the documentation for the C read function which is what Perl's sysread
calls. If you are on some other system you will need to find the
appropriate documentation for yourself).

Ben

--
I must not fear. Fear is the mind-killer. I will face my fear and
I will let it pass through me. When the fear is gone there will be
nothing. Only I will remain.
Frank Herbert, 'Dune'

Ben Morrow wrote:

> Quoth :
> I don't know what makes you think that. Did you try it? sysread returns
> 0 on EOF.
>
> Note that you should also be checking defined($res), as sysread returns
> undef on errors. Depending on how you've opened the socket and your
> system, some errors (EINTR and EAGAIN, specifically) are not in fact
> errors, but indications that you need to retry the read.
>
> I think you need to read
>
> perldoc -f sysread
>
> and
>
> man 2 read
>
> before you go much further (the latter assumes you are on Unix; this is
> the documentation for the C read function which is what Perl's sysread
> calls. If you are on some other system you will need to find the
> appropriate documentation for yourself).

In my original post, I mentioned I am using Linux; Slackware in particular.
I have read the perldoc for sysread. The problem is sysread blocks. So,
when the last of the data is transmitted, $res has a value, the data is
processed, and when the loop goes back to read more data, there is none,
and sysread just sits, waiting for more.

So, I figure, the only way to get sysread to do this is with non-blocking.
Using IO::Select, I came up with this:

my $select = IO::Select->new($socket);

while (1) {
if (my $ready = $select->can_read(1)) {
my $res = sysread($socket,$_,4096);
last if(!defined($res)); #never used
print "RES:$res\nLEN:",length($_),"\n$_\n";
}
else { print "!READY\n"; last; }
}

I tried using "can_read(0)", so that it wouldn't wait, but it was too fast
and the "else { print "!READY\n"; last; }" statement would catch. Without
the else statement, the loop never exists. The "last if(!defined($res));"
is never used because the can_read line doesn't enter the loop if there
isn't any data.

I guess this is the best way to get this to work. It just seems a bit
clunky.

--
lucas
-------------------------
Perl Coder since 2001
shift || die;
-------------------------

lucas <> wrote:
> Ben Morrow wrote:
>
> In my original post, I mentioned I am using Linux; Slackware in
> particular. I have read the perldoc for sysread. The problem is sysread
> blocks.

It blocks waiting on more data. It doesn't block on eof, as there is no
more data to wait on.

> So, when the last of the data is transmitted, $res has a value,
> the data is processed, and when the loop goes back to read more data,
> there is none, and sysread just sits, waiting for more.

If that is actually the case, then the server is not sending an eof when it
should (or maybe it isn't supposed to send an eof and you are
misunderstanding the protocol).

.....
>
> I guess this is the best way to get this to work. It just seems a bit
> clunky.

That often happens when you decline to use the existing modules and insist
on rolling your own.

Xho

--
-------------------- http://NewsReader.Com/ --------------------
Usenet Newsgroup Service $9.95/Month 30GB

lucas wrote:
> I'm use IO::Socket on linux to read from a webpage. I don't want to use
> LWP::Simple. [...]

Of course not, otherwise you'd be finished and working on something else
by now.