Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > os.pipe and subprocess under Windows

Reply
Thread Tools

os.pipe and subprocess under Windows

 
 
Jeremy Sanders
Guest
Posts: n/a
 
      11-17-2008
Hi - I have some code which works under linux. It starts a remote python
process using subprocess and communicates to it via a pipe created by
os.pipe. As far as I understand, child processes should inherit file
descriptors from the parent if close_fds=False on the suprocess.Popen
command line.

This code doesn't work under Window, but gives "bad file descriptor" when
trying to read from the pipe in the child process. I have some example code
which fails. It consists of two files, master.py and slave.py. The file
descriptor for reading from the pipe is passed as a argument to slave.py.

----------------------------------------------------------------
#!/usr/bin/env python
# This is master.py

import os
import os.path
import sys
import subprocess

def runMaster():
# create pipe to communicate with remote process
rpipe, wpipe = os.pipe()

# start remote process
cmdline = [sys.executable,
os.path.join( os.path.dirname(
os.path.abspath(__file__)), 'slave.py' ),
str(rpipe) ]

remote = subprocess.Popen(cmdline, shell=False, bufsize=0,
close_fds=False)

# send text to remote process via pipe
os.write(wpipe, 'hi there$')

# wait until remote exit
remote.wait()

if __name__ == '__main__':
runMaster()

------------------------------------------------------------------

# This is slave.py

import sys
import os

def runSlave(fd):
"""Copy text to stderr from file descriptor until a $ symbol."""
while True:
intext = os.read(fd, 1)
if intext == '$':
break
elif intext:
# write text from pipe to stderr
sys.stderr.write('* %s\n' % intext)

if __name__ == '__main__':
fd = int(sys.argv[1])
runSlave(fd)

-------------------------------------------------------------------

Does anyone have any ideas how to get this to work under Windows? Is it
correct code under unix?

Thanks

Jeremy

--
Jeremy Sanders
http://www.jeremysanders.net/
 
Reply With Quote
 
 
 
 
Lawrence D'Oliveiro
Guest
Posts: n/a
 
      11-17-2008
Jeremy Sanders wrote:

> As far as I understand, child processes should inherit file
> descriptors from the parent if close_fds=False on the suprocess.Popen
> command line.
>
> This code doesn't work under Window, but gives "bad file descriptor" when
> trying to read from the pipe in the child process.


<http://docs.python.org/library/subprocess.html>:

If close_fds is true, all file descriptors except 0, 1 and 2 will be
closed before the child process is executed. (Unix only). Or, on
Windows, if close_fds is true then no handles will be inherited by the
child process.

Windows has no fork(2).
 
Reply With Quote
 
 
 
 
Jeremy Sanders
Guest
Posts: n/a
 
      11-17-2008
Lawrence D'Oliveiro wrote:

> <http://docs.python.org/library/subprocess.html>:
>
> If close_fds is true, all file descriptors except 0, 1 and 2 will be
> closed before the child process is executed. (Unix only). Or, on
> Windows, if close_fds is true then no handles will be inherited by the
> child process.
>
> Windows has no fork(2).


Yes - I saw that - thanks. This suggests that as I have used
closed_fds=False, then the child process will inherit the handles under
Windows, which it doesn't seem to. This documentation looks wrong to me.

I know Windows has no fork - that's why I used subprocess. This MSDN page
suggests you you need to do something with SetHandleInformation to get them
to be inherited. Doesn't subprocess do that?

http://msdn.microsoft.com/en-us/libr...99(VS.85).aspx

--
Jeremy Sanders
http://www.jeremysanders.net/
 
Reply With Quote
 
Jeremy Sanders
Guest
Posts: n/a
 
      11-17-2008
Jeremy Sanders wrote:

> Hi - I have some code which works under linux. It starts a remote python
> process using subprocess and communicates to it via a pipe created by
> os.pipe. As far as I understand, child processes should inherit file
> descriptors from the parent if close_fds=False on the suprocess.Popen
> command line.


Hmm... examining the code for os.pipe in posixmodule.c, it looks like pipes
are create specifically to be non-inheritable in Windows. I can't see why
you would want a non-inheritable pipe, so I would call this a bug.

I suppose I could try this trick from subprocess.py to make the pipes
inheritable:

def _make_inheritable(self, handle):
"""Return a duplicate of handle, which is inheritable"""
return DuplicateHandle(GetCurrentProcess(), handle,
GetCurrentProcess(), 0, 1,
DUPLICATE_SAME_ACCESS)

Pretty nasty to have to do this though, and I would have to add a win32api
dependency, or hack around with the _subprocess module.

Jeremy

--
Jeremy Sanders
http://www.jeremysanders.net/
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
subprocess.Popen under windows 7 Frank van den Boom Python 5 12-09-2011 09:47 AM
Re: subprocess module under python 2.7 Chris Rebert Python 0 07-27-2010 11:36 PM
subprocess module under python 2.7 Timothy W. Grove Python 0 07-27-2010 11:12 PM
how to import subprocess into my 'subprocess.py' file hiral Python 2 05-05-2010 12:56 PM
[Subprocess/Windows] subprocess module under Windows 98 Andreas Jung Python 2 11-02-2005 05:41 PM



Advertisments