Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > Re: Subprocess does not return for longer-running process

Reply
Thread Tools

Re: Subprocess does not return for longer-running process

 
 
Dennis Lee Bieber
Guest
Posts: n/a
 
      09-22-2010
On Tue, 21 Sep 2010 17:33:25 -0700, Chris Rebert <(E-Mail Removed)>
declaimed the following in gmane.comp.python.general:

> As your Traceback clearly indicates, the Popen() call has already
> completed; it's *the os.waitpid() call* that's blocking, but that's
> entirely to be expected given its defined behavior. If you don't want
> to wait around for the process to die, then don't call waitpid() on it
> in the first place!
>

And a likely reason the waitpid() if blocking is that the
sub-process is blocking trying to write to pipe that is full.

--
Wulfraed Dennis Lee Bieber AF6VN
http://www.velocityreviews.com/forums/(E-Mail Removed) HTTP://wlfraed.home.netcom.com/

 
Reply With Quote
 
 
 
 
Nobody
Guest
Posts: n/a
 
      09-22-2010
On Tue, 21 Sep 2010 23:54:04 -0700, Dennis Lee Bieber wrote:

>> As your Traceback clearly indicates, the Popen() call has already
>> completed; it's *the os.waitpid() call* that's blocking, but that's
>> entirely to be expected given its defined behavior. If you don't want
>> to wait around for the process to die, then don't call waitpid() on it
>> in the first place!
>>

> And a likely reason the waitpid() if blocking is that the
> sub-process is blocking trying to write to pipe that is full.


Which is why you should always use the .communicate() method if you
redirect two or more of std{in,out,err} to a pipe (this is why the Unix
popen() function only lets you redirect one of the standard descriptors).

And I can't think of any reason why you should use os.waitpid() or
similar; use the .wait() method.

 
Reply With Quote
 
 
 
 
Lawrence D'Oliveiro
Guest
Posts: n/a
 
      09-23-2010
In message <(E-Mail Removed)>, Nobody wrote:

> And I can't think of any reason why you should use os.waitpid() or
> similar; use the .wait() method.


I have used WNOHANG to poll for completion of a subprocess while providing
progress updates to the user.
 
Reply With Quote
 
Nobody
Guest
Posts: n/a
 
      09-23-2010
On Thu, 23 Sep 2010 12:25:53 +1200, Lawrence D'Oliveiro wrote:

>> And I can't think of any reason why you should use os.waitpid() or
>> similar; use the .wait() method.

>
> I have used WNOHANG to poll for completion of a subprocess while providing
> progress updates to the user.


This can be done via the .poll() method.

 
Reply With Quote
 
Lawrence D'Oliveiro
Guest
Posts: n/a
 
      09-24-2010
In message <(E-Mail Removed)>, Nobody wrote:

> On Thu, 23 Sep 2010 12:25:53 +1200, Lawrence D'Oliveiro wrote:
>
>>> And I can't think of any reason why you should use os.waitpid() or
>>> similar; use the .wait() method.

>>
>> I have used WNOHANG to poll for completion of a subprocess while
>> providing progress updates to the user.

>
> This can be done via the .poll() method.


And what do you think the poll method uses?
 
Reply With Quote
 
Nobody
Guest
Posts: n/a
 
      09-24-2010
On Fri, 24 Sep 2010 15:18:47 +1200, Lawrence D'Oliveiro wrote:

>>>> And I can't think of any reason why you should use os.waitpid() or
>>>> similar; use the .wait() method.
>>>
>>> I have used WNOHANG to poll for completion of a subprocess while
>>> providing progress updates to the user.

>>
>> This can be done via the .poll() method.

>
> And what do you think the poll method uses?


1. Not relevant; use the defined interface.

2. It currently uses waitpid() on Unix, WaitForSingleObject() on Windows,
maybe other functions if subprocess gets ported to other platforms or
extended. If you use the .poll() method, you don't need to worry about any
of this.

More importantly the .poll() method sets the .returncode attribute. Why is
this important? Because if the .returncode attribute is None,
Popen.__del__ will add the object to the _active list. Not only does this
prevent the object from being finalised, it will continue to poll for
termination via _cleanup().

But if you have manually reaped the child, its PID becomes available for
re-use. This can result in a subsequent _cleanup() reaping some other
child process. In the 2.6 implementation, manually setting the .returncode
attribute will solve this. In future implementations ... who knows?

Again: use the .wait() or .poll() methods. These are portable, can be
assumed to do the right thing for subclasses of subprocess.Popen(), and
will perform any necessary maintenance of "internal" state.

If those methods don't suffice, you can't reliably use subprocess. E.g. if
you need to wait until any child terminates, using os.wait() will cause
the child to be reaped, which will confuse Popen().

POSIX (and Linux since 2.6.9) has waitid(), which allows you to wait for a
child *without* reaping it (via the WNOWAIT flag), but Python doesn't
include an interface for this.

 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Subprocess does not return for longer-running process Jason Friedman Python 0 09-21-2010 10:06 PM
how to import subprocess into my 'subprocess.py' file hiral Python 2 05-05-2010 12:56 PM
[Subprocess/Windows] subprocess module under Windows 98 Andreas Jung Python 2 11-02-2005 05:41 PM
what value does lack of return or empty "return;" return Greenhorn C Programming 15 03-06-2005 08:19 PM
Process does not receive EOF when reading from Process.getOutputStream() Reid Madsen Java 8 05-16-2004 01:37 AM



Advertisments