Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > Python deadlocks when Popen in multithread.

Reply
Thread Tools

Python deadlocks when Popen in multithread.

 
 
INADA Naoki
Guest
Posts: n/a
 
      10-11-2010
On Ubuntu 10.10 amd64 and dual-core CPU, following code deadlocks.
What's wrong on this code?

#http://codepad.org/GkoXHbik

#!/usr/bin/env python

from subprocess import *
from threading import *

THREAD_COUNT=50

def worker():
p = Popen(["cat"], stdin=PIPE, stdout=PIPE, stderr=STDOUT)
out = p.communicate("hoge")[0]
print "%s %s" % (current_thread().name, out)

threads = []
for i in range(THREAD_COUNT):
threads.append(Thread(target=worker, name=str(i)))
for t in threads: t.start()
for t in threads: t.join()

 
Reply With Quote
 
 
 
 
Jed Smith
Guest
Posts: n/a
 
      10-11-2010
On Mon, Oct 11, 2010 at 4:19 PM, INADA Naoki <(E-Mail Removed)> wrote:

> def worker():
> * *p = Popen(["cat"], stdin=PIPE, stdout=PIPE, stderr=STDOUT)
> * *out = p.communicate("hoge")[0]
> * *print "%s %s" % (current_thread().name, out)


If, instead of spawning workers you directly call worker(), does it succeed?

I suspect the threading is a red herring here.

--
Jed Smith
http://www.velocityreviews.com/forums/(E-Mail Removed)
 
Reply With Quote
 
 
 
 
INADA Naoki
Guest
Posts: n/a
 
      10-12-2010
On Oct 12, 5:33*am, Jed Smith <(E-Mail Removed)> wrote:
> On Mon, Oct 11, 2010 at 4:19 PM, INADA Naoki <(E-Mail Removed)> wrote:
> > def worker():
> > * *p = Popen(["cat"], stdin=PIPE, stdout=PIPE, stderr=STDOUT)
> > * *out = p.communicate("hoge")[0]
> > * *print "%s %s" % (current_thread().name, out)

>
> If, instead of spawning workers you directly call worker(), does it succeed?


Yes.

> I suspect the threading is a red herring here.


But thread & subprocess is a normal way to invoke outer commands in
parallel.
 
Reply With Quote
 
INADA Naoki
Guest
Posts: n/a
 
      10-12-2010
I've found the bug http://bugs.python.org/issue1731717.
It seems wating subprocess can be multithreaded but creating
subprocess cannot.

On Oct 12, 9:57*am, INADA Naoki <(E-Mail Removed)> wrote:
> On Oct 12, 5:33*am, Jed Smith <(E-Mail Removed)> wrote:
>
> > On Mon, Oct 11, 2010 at 4:19 PM, INADA Naoki <(E-Mail Removed)> wrote:
> > > def worker():
> > > * *p = Popen(["cat"], stdin=PIPE, stdout=PIPE, stderr=STDOUT)
> > > * *out = p.communicate("hoge")[0]
> > > * *print "%s %s" % (current_thread().name, out)

>
> > If, instead of spawning workers you directly call worker(), does it succeed?

>
> Yes.
>
> > I suspect the threading is a red herring here.

>
> But thread & subprocess is a normal way to invoke outer commands in
> parallel.


 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
subprocess.Popen deadlocks eraserix Python 2 11-12-2010 02:28 PM
File.popen/IO.popen hariwise@gmail.com Ruby 1 05-20-2006 08:20 AM
Deadlocks Mike Carr ASP .Net 9 07-23-2004 07:31 PM
Automatically detecting deadlocks? Rogan Dawes Java 2 05-06-2004 06:27 AM
ASP.NET Deadlocks Martin Blackstone [MVP - Exchange] ASP .Net 6 08-24-2003 02:56 AM



Advertisments