Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > C++ > spawn process with bidirectional pipe (std/stdout)

Reply
Thread Tools

spawn process with bidirectional pipe (std/stdout)

 
 
Gernot Frisch
Guest
Posts: n/a
 
      12-14-2010
Hi,

I want to create a process and get a (or 2) FILE* to fwrite() to its stdin and fread() its stdout, but cross platform.

I found a windows only way:
http://support.microsoft.com/kb/190351/en-us

But all unix version seem to rely on fork(), which is total overkill if you fork() a big process (or am I wrong? - I'm a windows
guy).

So, is there a way to get a read/write handle to a process without fork?

Thank you,


------------------------------------
Gernot Frisch
http://www.glbasic.com

 
Reply With Quote
 
 
 
 
red floyd
Guest
Posts: n/a
 
      12-14-2010
On Dec 14, 9:04*am, "Gernot Frisch" <(E-Mail Removed)> wrote:
> Hi,
>
> I want to create a process and get a (or 2) FILE* to fwrite() to its stdin and fread() its stdout, but cross platform.


1. By definition that's OS dependent. You're kind of OT here. But
for half-duplex
(read only or write only), look at popen() on unix.
>
> I found a windows only way:http://support.microsoft.com/kb/190351/en-us
>
> But all unix version seem to rely on fork(), which is total overkill if you fork() a big process (or am I wrong? - I'm a windows
> guy).
>


2. And how, pray tell, do you plan on creating a new process, other
than fork()?
 
Reply With Quote
 
 
 
 
Jens Thoms Toerring
Guest
Posts: n/a
 
      12-14-2010
Gernot Frisch <(E-Mail Removed)> wrote:
> I want to create a process and get a (or 2) FILE* to fwrite() to its stdin and fread() its stdout, but cross platform.


> I found a windows only way:
> http://support.microsoft.com/kb/190351/en-us


> But all unix version seem to rely on fork(), which is total overkill if
> you fork() a big process (or am I wrong? - I'm a windows guy).


fork() nowadays (and already for quite some time) uses copy-
on-write, i.e. only those pages in memory that are being writ-
ten to by one of the processes are copied, so fork() does not
copy the whole process memory but only sucessively and as far
as necessary. So it's not the overkill you may suspect it to
be.

> So, is there a way to get a read/write handle to a process without fork?


You can't. If you need a new process you need to call fork(),
that's how processes are created (well, there's also vfork(),
but that you definitely don't want to use).

For the details of the redirections of the file descriptors
and then the FILE pointers etc. involved I would think it
would be wiser to ask in e.g. comp.unix.programmer where it
is as on-topic as you can get
Regards, Jens
--
\ Jens Thoms Toerring ___ http://www.velocityreviews.com/forums/(E-Mail Removed)
\__________________________ http://toerring.de
 
Reply With Quote
 
Joshua Maurice
Guest
Posts: n/a
 
      12-14-2010
On Dec 14, 9:04*am, "Gernot Frisch" <(E-Mail Removed)> wrote:
> Hi,
>
> I want to create a process and get a (or 2) FILE* to fwrite() to its stdin and fread() its stdout, but cross platform.
>
> I found a windows only way:http://support.microsoft.com/kb/190351/en-us
>
> But all unix version seem to rely on fork(), which is total overkill if you fork() a big process (or am I wrong? - I'm a windows
> guy).
>
> So, is there a way to get a read/write handle to a process without fork?
>
> Thank you,


For POSIX systems, your only real option is fork. (There's a couple
more obscure ones which you don't want to use, or which are basically
wrappers on top of fork, such as popen and posix_spawn.)

IMHO, this is a broken state of affairs for a variety of reasons.

My biggest annoyance is that it's incredibly hard to ensure no
resource is "leaked" across a fork in a quick and portable way, and
the extra memory required for forking. Almost the entire point of
processes is fault isolation. The entire OS and the hardware chip
itself is built with this goal in mind, e.g. kernel mode and user
mode. The default usage of fork makes it incredibly easy, standard
practice even, to leak resources across a fork, which is quite
contradictory. I think the most notable thing is that it's a huge
gaping security hole when files, mmaps, etc., can be leaked from one
process to another so easily.

As mentioned else-thread, most windows / mac / unix systems implement
fork so that its virtual pages are copy on write, and this helps a
lot, but I would mention that still has some negative side effects.
Either you have overcommit on, in which case if your system is loaded,
random programs will be killed with great prejudice (aka very bad), or
you have overcommit off, which means that while a fork will not copy
all of the contents of the virtual pages, it will commit those virtual
pages, which can fail if your system is loaded and near the commit
limit. At least with overcommit off, the process gets an error when it
tries to get a new virtual page as opposed to a random process just
being instantly arbitrarily killed. I suppose the problem with
overcommit off can be solved with a larger swap partition size if you
know the size of your workload, so I guess that it's not that big of a
deal. I just wanted to make it clear.

To finish this rant, I don't mind fork existing. I just want another
primitive which spawns a process from an executable file in a clean
environment, so I can choose to be easily leak-free if so desired. I'd
prefer this new spawn would be the default for developers as well. In
the end, what fork has over this new method is simply ease of use.
With my spawn primitive, you would have to put the code for the child
process which sets up the environment into its own cpp file and
compile it into its own executable, whereas with the fork you can put
the code inline. Saves some keystrokes only.

Off topic, I know. Sorry. Rant done.

PS: Is windows broken in the same way with regards to file handles? I
can't parse the msdn website very well in this regard, but from what I
can gather, /I think that/ windows has the same resource leak problem
as unix style forks - all file handles are inherited by default into
the child, and there's no easy way to say "Only inherit these specific
X file handles".
 
Reply With Quote
 
Marc
Guest
Posts: n/a
 
      12-15-2010
Joshua Maurice wrote:

> For POSIX systems, your only real option is fork. (There's a couple
> more obscure ones which you don't want to use, or which are basically
> wrappers on top of fork, such as popen and posix_spawn.)


Why discard popen and posix_spawn so fast? Properly implemented, they
address some of your concerns, like the one about overcommit.
 
Reply With Quote
 
Miles Bader
Guest
Posts: n/a
 
      12-15-2010
"Gernot Frisch" <(E-Mail Removed)> writes:
> But all unix version seem to rely on fork(), which is total overkill
> if you fork() a big process (or am I wrong? - I'm a windows guy).


Fork+exec is quite efficient. You shouldn't worry about it unless you
actually have specific data showing a problem for your usage.

> So, is there a way to get a read/write handle to a process without fork?


You can use "sendmsg" to send an open file descriptor to
an existing process, but of course you've got to create the receiving
process first...

-Miles

--
Mad, adj. Affected with a high degree of intellectual independence; not
conforming to standards of thought, speech, and action derived by the
conformants [sic] from study of themselves; at odds with the majority; in
short, unusual. It is noteworthy that persons are pronounced mad by officials
destitute of evidence that they themselves are sane.
 
Reply With Quote
 
Joshua Maurice
Guest
Posts: n/a
 
      12-15-2010
On Dec 15, 2:20*am, Marc <(E-Mail Removed)> wrote:
> Joshua Maurice *wrote:
> > For POSIX systems, your only real option is fork. (There's a couple
> > more obscure ones which you don't want to use, or which are basically
> > wrappers on top of fork, such as popen and posix_spawn.)

>
> Why discard popen and posix_spawn so fast? Properly implemented, they
> address some of your concerns, like the one about overcommit.


True, but they do nothing for my resource "leaking" / "bleeding"
problem. I was straightforward about my concerns, I think.
 
Reply With Quote
 
Jorgen Grahn
Guest
Posts: n/a
 
      12-18-2010
On Tue, 2010-12-14, Gernot Frisch wrote:
> Hi,
>
> I want to create a process and get a (or 2) FILE* to
> fwrite() to its stdin and fread() its stdout, but cross platform.
>
> I found a windows only way:
> http://support.microsoft.com/kb/190351/en-us
>
> But all unix version seem to rely on fork(), which is total overkill
> if you fork() a big process (or am I wrong? - I'm a windows
> guy).


There is some misunderstanding here. The way to create a true process
on Unix /is/ fork(); it cannot by definition be overkill to create a
process by that method. Did you mean a thread, of do you confuse
fork() with exec()?

Also note that it's very easy to run into a deadlock when you set up
double pipes like this -- especially with the buffering done by FILE*.
You may end up with process A being blocked when writing to B, because
B is blocked trying to write to A.

And finally ... all this is offtopic here.

/Jorgen

--
// Jorgen Grahn <grahn@ Oo o. . .
\X/ snipabacken.se> O o .
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Spawn a process, then exit, whilst leaving process running? Victor Hooi Python 1 02-10-2013 03:50 AM
Logging stdout/stderr/stdin of an spawn process (Open4::spawn) Edgardo Hames Ruby 1 05-06-2008 08:17 PM
Logging stdout/stderr/stdin of an spawn process (Open4::spawn) Ed Hames Ruby 0 04-16-2008 04:21 PM
bidirectional connection between two bidirectional ports Manfred Balik VHDL 12 09-10-2006 08:53 PM
spawn syntax + os.P_WAIT mode behavior + spawn stdout redirection Derek Basch Python 2 01-21-2005 05:37 AM



Advertisments