Really unbuffered reads...
 

Hi all,

I'm using GnuPG::Interface to verify some PGP signed data, and for the
most part it all seems to be working nicely.

I have come across a small problem though. Sometimes (not always, which
makes this nicely annoying) the function I use to read through the
output blocks indefinitely.

The segment of code that causes the problem is:

# $fh is a filehandle passed in to the function -
# its an IO::Handle object that's been used as the
# stderr handle to GnuPG::Interface::verify
my ($buffer, $ok) = ('', undef);
while (1) {
if (not sysread($fh, $buffer, 1024, length($buffer)) {
last if not length $buffer;
$ok = [1, $1] if $buffer =~ PGP_SIGOK;
last;
}
# Do some stuff with buffer to look for individual lines...
}

This _almost_ always works fine. Rarely, that sysread hangs. If i'm
looking at the process table at the time and I kill off the gpg process,
everything trucks along as it should.

Is there a way to fix this. Maybe a different method for reading in the
data? I'm guessing that the problem is that sometimes gpg doesn't spit
out a newline at the end of its stderr messages, but I can't find any
obvious differences between messages that work and those that block.

MB

Re: Really unbuffered reads...
 

Matthew Braid <mb@uq.net.au.invalid> wrote in comp.lang.perl.misc:
> Hi all,
>
> I'm using GnuPG::Interface to verify some PGP signed data, and for the
> most part it all seems to be working nicely.
>
> I have come across a small problem though. Sometimes (not always, which
> makes this nicely annoying) the function I use to read through the
> output blocks indefinitely.
>
> The segment of code that causes the problem is:
>
> # $fh is a filehandle passed in to the function -
> # its an IO::Handle object that's been used as the
> # stderr handle to GnuPG::Interface::verify
> my ($buffer, $ok) = ('', undef);
> while (1) {
> if (not sysread($fh, $buffer, 1024, length($buffer)) {
> last if not length $buffer;

"last unless ..." reads better here.

> $ok = [1, $1] if $buffer =~ PGP_SIGOK;
^^^^^^^^^
Uh... I suppose that's a constant that returns a regex.

> last;
> }
> # Do some stuff with buffer to look for individual lines...
> }
>
> This _almost_ always works fine. Rarely, that sysread hangs. If i'm

Use select() or IO::Select to see if the filehandle is ready for reading.
You could also make it non-blocking, but I don't know how well that works
with sysread().

In both cases you'll have to decide what to do if the handle never
becomes ready, and how long "never" is.

Anno

Re: Really unbuffered reads...
 

>>>>> "AS" == Anno Siegel <anno4000@lublin.zrz.tu-berlin.de> writes:

AS> Use select() or IO::Select to see if the filehandle is ready for reading.
AS> You could also make it non-blocking, but I don't know how well that works
AS> with sysread().

sysread/write work fine with non-blocking as long as you handle the buffering
yourself. in fact they usually required for non-blocking sockets as you
could do a large print to such a socket and not all of the data will be
written and the rest will sit in the buffer forever until you flush
it. and you have no control then over how much gets written each time.

uri

--
Uri Guttman ------ uri@stemsystems.com -------- http://www.stemsystems.com
--Perl Consulting, Stem Development, Systems Architecture, Design and Coding-
Search or Offer Perl Jobs ---------------------------- http://jobs.perl.org

Re: Really unbuffered reads...
 

Anno Siegel wrote:

> Matthew Braid <mb@uq.net.au.invalid> wrote in comp.lang.perl.misc:
<snip>
>> $ok = [1, $1] if $buffer =~ PGP_SIGOK;
>
> ^^^^^^^^^
> Uh... I suppose that's a constant that returns a regex.

Whoops - sorry, yeah it does.

> Use select() or IO::Select to see if the filehandle is ready for reading.
> You could also make it non-blocking, but I don't know how well that works
> with sysread().
>
> In both cases you'll have to decide what to do if the handle never
> becomes ready, and how long "never" is.

As a quick fix I used a couple of surrounding alarm statements, which
worked for a while until another test case came along that led to the
solution.

My first two test cases were very similar, the third was much larger. It
seems the first two were just on either side of a boundary problem (they
differed in size by 23 bytes, so it was a close thing :) )

The problem was the output handle (not the error handle). The function
that called the problem code was trying to handle verification,
encryption and decryption. When it came to verification it sometimes
decrypted the message instead of just verifying it so that the
signatures on encrypted messages could be checked. Unfortunately, this
meant that it 'threw away' the decrypted output but supplying an
IO::Handle object that was never read from. It was also not closed until
_after_ the error handle was scanned.

This meant that for large enough messages whatever underlying buffer
IO::Handle has was filling up. What I was seeing in top was gpg sitting
in pipe-write state, and my script sitting in pipe-read - but gpg was
trying to write to the output handle, while my script was trying to read
from its error handle. I had thought that the two were stuck on the same
handle.

Once I changed the code so that if decryption is required for
verification the output handle is undef (and thus the output disappears
into the void) everything suddenly worked perfectly. It also explains
why the problem only came up with verification - on encryption and
decryption the output filehandle was opened to a temp file (I needed to
keep it, and I needed it seekable), so the data could pretty much flow
straight through unhindered.

Phew :)

Thanks for mentioning select though - looks a little cleaner than alarms...

MB