Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Ruby > Piping binary data to an external program

Reply
Thread Tools

Piping binary data to an external program

 
 
|MKSM|
Guest
Posts: n/a
 
      03-06-2006
Hello,

I'm working on a logparser and i've run into some issues. It will
parse OpenBSD PF logs. They are tcpdump format logs and BSD normally
compress them.

Here is the usage I have in mind:

"gzip -cd log.gz | ruby logparser.rb --today"

I have the following code:

Open3.popen3("/usr/sbin/tcpdump -nettr -") { |in_io, out_io, err_io|
=09in_io.write($stdin.read)
=09in_io.close
=09$log =3D out_io.read
}

The script freezes on the open3 line and doesn't continue. I've tested
several other methods but it doesn't seem to work.

Any suggestions on how this can be done?

Regards,

Ricardo.


 
Reply With Quote
 
 
 
 
Robert Klemme
Guest
Posts: n/a
 
      03-06-2006
|MKSM| wrote:
> Hello,
>
> I'm working on a logparser and i've run into some issues. It will
> parse OpenBSD PF logs. They are tcpdump format logs and BSD normally
> compress them.
>
> Here is the usage I have in mind:
>
> "gzip -cd log.gz | ruby logparser.rb --today"
>
> I have the following code:
>
> Open3.popen3("/usr/sbin/tcpdump -nettr -") { |in_io, out_io, err_io|
> in_io.write($stdin.read)
> in_io.close
> $log = out_io.read
> }
>
> The script freezes on the open3 line and doesn't continue. I've tested
> several other methods but it doesn't seem to work.
>
> Any suggestions on how this can be done?


It's likely that you run into a deadlock caused by pipe buffer sizes. I
suggest to not read and write the whole content but to do it in chunks.
Also, I'd separate the reading and writing code into two threads.

Untested:

Open3.popen3("/usr/sbin/tcpdump -nettr -") { |in_io, out_io, err_io|
t = Thread.new(in_io) do |out|
while ( buff = $stdin.read( 1024 ) )
out.write(buff)
end
out.close
end
$log = ""
while ( b = out_io.read(1024))
$log << b
end
}


HTH

Kind regards

robert

 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
piping input to an external script Tim Arnold Python 9 05-12-2009 05:24 PM
Piping println() through an external pager program (more, less) Thomas Kellerer Java 47 04-12-2009 08:45 PM
Piping binary files Bazsl Ruby 1 10-15-2007 12:00 AM
Java: Using Piping or any other methods to use the output from one program as input in another. julielaurek@gmail.com Java 12 07-04-2007 01:06 AM
Piping external commands saibotorama@gmail.com Python 1 07-18-2006 05:43 PM



Advertisments