Velocity Reviews

Velocity Reviews (http://www.velocityreviews.com/forums/index.php)
-   Python (http://www.velocityreviews.com/forums/f43-python.html)
-   -   multiprocessing, pool and process crashes (http://www.velocityreviews.com/forums/t681993-multiprocessing-pool-and-process-crashes.html)

dragonk@gmail.com 04-29-2009 02:50 PM

multiprocessing, pool and process crashes
 
I want to use the multiprocessing.Pool object to run multiple tasks in
separate processes.

The problem is that I want to call an external C function (from a
shared library, with help from ctypes) and this function tends to
crash (SIGSEGV,SIGFPE,etc.) on certain data sets (the purpose of this
thing is to test the behavior of the function and see when it
crashes).

My question is if the Pool object is able to spawn a new process in
place of the crashed one and resume operation. I killed with SIGSEGV
the subprocesses and the script locked up :(

Any ideas/alternatives?

Thanks!


Jesse Noller 04-30-2009 12:58 PM

Re: multiprocessing, pool and process crashes
 
On Wed, Apr 29, 2009 at 10:50 AM, <dragonk@gmail.com> wrote:
> I want to use the multiprocessing.Pool object to run multiple tasks in
> separate processes.
>
> The problem is that I want to call an external C function (from a
> shared library, with help from ctypes) and this function tends to
> crash (SIGSEGV,SIGFPE,etc.) on certain data sets (the purpose of this
> thing is to test the behavior of the function and see when it
> crashes).
>
> My question is if the Pool object is able to spawn a new process in
> place of the crashed one and resume operation. *I killed with SIGSEGV
> the subprocesses and the script locked up :(
>
> Any ideas/alternatives?
>


You're going to want to use a custom pool, not the built in pool. In
your custom pool, you'll need to capture the signals/errors you want
and handle them accordingly. The built in pool was not designed for
your use case.

-jesse


All times are GMT. The time now is 04:22 PM.

Powered by vBulletin®. Copyright ©2000 - 2014, vBulletin Solutions, Inc.
SEO by vBSEO ©2010, Crawlability, Inc.