Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > subprocess module and long-lived subprocesses

Reply
Thread Tools

subprocess module and long-lived subprocesses

 
 
skip@pobox.com
Guest
Posts: n/a
 
      01-20-2012

I'm converting some os.popen calls to use subprocess.Popen. I had
previously been ignoring stdout and stderr when using os.popen. The primary
motivation to switch to subprocess.Popen now is that I now want to check
stderr, so would have to make code changes to use os.popen[34] anyway.
Might as well go whole hog and switch to the new API.

The library documentation doesn't talk a lot about long-lived subprocesses
other than the possibility of deadlock when using Popen.wait(). Ideally, I
would write to the subprocess's stdin, check for output on stdout and
stderr, then lather, rinse, repeat. Is it safe to assume that if the stdout
and/or stderr pipes have nothing for me the reads on those file objects (I'm
using PIPE for all three std* files) will return immediately with an empty
string for output? They won't block, will they? Will a broken pipe IOError
get raised as for os.popen() or do I have to call Popen.poll() even in error
situations?

Thanks,

--
Skip Montanaro - http://www.velocityreviews.com/forums/(E-Mail Removed) - http://www.smontanaro.net/
 
Reply With Quote
 
 
 
 
Nobody
Guest
Posts: n/a
 
      01-21-2012
On Fri, 20 Jan 2012 08:42:16 -0600, skip wrote:

> The library documentation doesn't talk a lot about long-lived subprocesses
> other than the possibility of deadlock when using Popen.wait(). Ideally, I
> would write to the subprocess's stdin, check for output on stdout and
> stderr, then lather, rinse, repeat. Is it safe to assume that if the stdout
> and/or stderr pipes have nothing for me the reads on those file objects (I'm
> using PIPE for all three std* files) will return immediately with an empty
> string for output? They won't block, will they?


They will. You need to use either threads, select() or non-blocking I/O in
order to avoid deadlock. See the definitions of subprocess._communicate()
(there's one version for Windows which uses threads and another for Unix
using select()).

> Will a broken pipe IOError get raised as for os.popen()


IOError(EPIPE) will be raised if you write to the stdin pipe when there
are no readers.

> or do I have to call Popen.poll() even in error situations?


Once you're finished with the process, you should close .stdin then
consume all output from .stdout and .stderr until both report EOF, then
call .wait(). That should cover any possible child behaviour (e.g. if
the child explicitly close()s its stdin, getting EPIPE doesn't mean that
you can forget about the process or that .wait() won't deadlock).

 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
how to import subprocess into my 'subprocess.py' file hiral Python 2 05-05-2010 12:56 PM
subprocesses and deadlocks betatim@gmail.com Python 1 08-06-2006 04:19 PM
Confusion with resource.setrlimit() and subprocesses Mitja Trampus Python 0 07-05-2006 11:20 AM
subprocesses, stdin/out, ttys, and beating insubordinate processesinto the ground Jonathan Smith Python 0 06-06-2006 08:52 PM
[Subprocess/Windows] subprocess module under Windows 98 Andreas Jung Python 2 11-02-2005 05:41 PM



Advertisments