Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Perl > Perl Misc > Fast pipe communication

Reply
Thread Tools

Fast pipe communication

 
 
Andrei Alexandrescu (See Website For Email)
Guest
Posts: n/a
 
      02-03-2006
Hello everybody,


I've written a small utility that allows Perl to communicate with TeX
(knowledge of TeX is not necessary to understand my question). Basically
I run perl to communicate with two pipes: perlin.tex and perlout.tex.
Perl reads a line off perlin.tex, evaluates the string, and writes it to
perlout.tex. Here's what I came up with:

===========================================
#!/bin/env perl
use strict;
use warnings;

system("rm -f perlin.tex perlout.tex && mkfifo perlin.tex perlout.tex") == 0
or die "Couldn't create pipes";

while () {
print "Opening perlin.tex...\n";
open(my $perlin, "perlin.tex") or die "Can't open: $!";
print "Opened perlin.tex\n";
print "Opening perlout.tex...\n";
open(my $perlout, ">perlout.tex") or die "Can't open: $!";
print "Opened perlout.tex\n";
$| = 1;
while (<$perlin>) {
chomp;
print "Read a line: `$_'\n";
next if $_ eq "";
my $r = eval($_);
die $@ if $@;
chomp $r;
print $perlout "$r\n";
}
print "Input pipe finished.\n";
}
===========================================

Test code:

$ perl perlengine.pl &
$ echo "1+1" > perlin.tex
$ cat perlout.tex
2
$ _

(Ignoring, of course, all of the debug messages.)

The question is, how can I make the code more efficient? I noticed that
every evaluation is really a pass through the outer loop. In other
words, I was unable to convince perl to block in <$perlin> once the
process writing to perlin.tex has finished. I need to reopen perlin.tex
for that; the call to open will block until someone wrote to perlin.tex.

Is there any chance of avoiding the expensive trip of opening and
closing the whole thing over and over again? Can I tell perl, "read to
the EOF of this pipe, and then rewind and read again, blocking if nobody
wrote to it"?


Thanks,

Andrei

 
Reply With Quote
 
 
 
 
xhoster@gmail.com
Guest
Posts: n/a
 
      02-03-2006
"Andrei Alexandrescu (See Website For Email)"
<(E-Mail Removed)> wrote:
> Hello everybody,
>
> I've written a small utility that allows Perl to communicate with TeX
> (knowledge of TeX is not necessary to understand my question). Basically
> I run perl to communicate with two pipes: perlin.tex and perlout.tex.
> Perl reads a line off perlin.tex, evaluates the string, and writes it to
> perlout.tex. Here's what I came up with:
>
> ===========================================
> #!/bin/env perl
> use strict;
> use warnings;
>
> system("rm -f perlin.tex perlout.tex && mkfifo perlin.tex perlout.tex")
> == 0
> or die "Couldn't create pipes";
>
> while () {
> print "Opening perlin.tex...\n";
> open(my $perlin, "perlin.tex") or die "Can't open: $!";
> print "Opened perlin.tex\n";
> print "Opening perlout.tex...\n";
> open(my $perlout, ">perlout.tex") or die "Can't open: $!";
> print "Opened perlout.tex\n";
> $| = 1;


This turns on autoflush only for STDOUT. You probably need to turn it on
for $perlout as well (although that is not the cause of the current
problem.)

use IO::Handle; # This should go near the top of the script, not here
$perlout->autoflush(1);

> while (<$perlin>) {
> chomp;
> print "Read a line: `$_'\n";
> next if $_ eq "";
> my $r = eval($_);
> die $@ if $@;
> chomp $r;
> print $perlout "$r\n";
> }
> print "Input pipe finished.\n";
> }
> ===========================================
>
> Test code:
>
> $ perl perlengine.pl &
> $ echo "1+1" > perlin.tex
> $ cat perlout.tex
> 2
> $ _
>
> (Ignoring, of course, all of the debug messages.)
>
> The question is, how can I make the code more efficient? I noticed that
> every evaluation is really a pass through the outer loop. In other
> words, I was unable to convince perl to block in <$perlin> once the
> process writing to perlin.tex has finished.


That's correct. Perl will not block on handles which are closed on the
other end.

> I need to reopen perlin.tex
> for that; the call to open will block until someone wrote to perlin.tex.


So it blocks at a slightly different place. Why is that a problem?

> Is there any chance of avoiding the expensive trip of opening and
> closing the whole thing over and over again?


You could move the opening of the perlout handle out of the loop, I see no
reason it needs to be repeated each time.

Anyway, assuming there were a way to hook a new process up to an existing
but closed pipe on one end, why should that be significantly more efficient
than re-opening the pipe? Both operations seem to be of about the same
complexity. The way to improve efficiency is to make the writing process
stay on the line rather than hanging up every time.

> Can I tell perl, "read to
> the EOF of this pipe, and then rewind and read again, blocking if nobody
> wrote to it"?


I know of no way to do it. It seems like that more a matter for the OS
than for Perl.

Xho

--
-------------------- http://NewsReader.Com/ --------------------
Usenet Newsgroup Service $9.95/Month 30GB
 
Reply With Quote
 
 
 
 
Andrei Alexandrescu (See Website For Email)
Guest
Posts: n/a
 
      02-04-2006
http://www.velocityreviews.com/forums/(E-Mail Removed) wrote:
>> I need to reopen perlin.tex
>> for that; the call to open will block until someone wrote to perlin.tex.

>
> So it blocks at a slightly different place. Why is that a problem?


You're right that the complexity of operation is the same, but generally
it's good to get away with as few system calls as it gets.

>> Is there any chance of avoiding the expensive trip of opening and
>> closing the whole thing over and over again?

>
> You could move the opening of the perlout handle out of the loop, I see no
> reason it needs to be repeated each time.


I tried, but then TeX becomes unhappy (hangs), and I have exceedingly
little control over the way TeX manipulates files...

> Anyway, assuming there were a way to hook a new process up to an existing
> but closed pipe on one end, why should that be significantly more efficient
> than re-opening the pipe? Both operations seem to be of about the same
> complexity. The way to improve efficiency is to make the writing process
> stay on the line rather than hanging up every time.
>
>> Can I tell perl, "read to
>> the EOF of this pipe, and then rewind and read again, blocking if nobody
>> wrote to it"?

>
> I know of no way to do it. It seems like that more a matter for the OS
> than for Perl.


Thanks for your input, Xho. Made me feel better ). After all, the
system runs very smooth as is.


Andrei
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Bidirectional communication over unix socket (named pipe) J Rice Python 6 03-11-2006 02:40 AM
[named pipe] i wanna know about validate of pipe handle of client lee, wonsun C++ 1 11-02-2004 04:29 AM
Why does IO::Pipe::END generate an EXCEPT pipe message? lvirden@gmail.com Perl Misc 1 06-02-2004 02:17 PM
fast communication with c++ H Brett Bolen Java 9 10-18-2003 03:16 PM
DVD Verdict reviews: ATANARJUAT (THE FAST RUNNER), PIPE DREAM, and more! DVD Verdict DVD Video 0 07-03-2003 09:03 AM



Advertisments