Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > Terminating a Python program that uses multi-process, multi-threading

Reply
Thread Tools

Terminating a Python program that uses multi-process, multi-threading

 
 
akineko
Guest
Posts: n/a
 
      01-28-2009
Hello Python experts,

I have a program that uses three processes (invoked by
multiprocessing) and several threads.
The program is terminated when ^C is typed (KeyboardInterrupt).
The main process takes the KeyboardInterrupt Exception and it orderly
shutdown the program.

It works fine in normal situation.

However, KeyboardInterrupt is not accepted when, for example, the
program is stuck somewhere due to error in network. I understand that
the KeyboardInterrupt won't be processed until the program moves out
from an atomic step in a Python program.

Probably this is partly my fault as I use some blocking statements
(without timeout) as they should not block the program in normal
situation.

As the program freezes up, I cannot do anything except killing three
processes using kill command.
So, I cannot tell which statement is actually blocking when this
occurs (and there are many suspects).

Is there any good way to deal with this kind of problem?
Killing three processes when it hangs is not a fun thing to do.

Any suggestions will be greatly appreciated.

Best regards,
Aki Niimura

 
Reply With Quote
 
 
 
 
MRAB
Guest
Posts: n/a
 
      01-28-2009
akineko wrote:
> Hello Python experts,
>
> I have a program that uses three processes (invoked by
> multiprocessing) and several threads.
> The program is terminated when ^C is typed (KeyboardInterrupt).
> The main process takes the KeyboardInterrupt Exception and it orderly
> shutdown the program.
>
> It works fine in normal situation.
>
> However, KeyboardInterrupt is not accepted when, for example, the
> program is stuck somewhere due to error in network. I understand that
> the KeyboardInterrupt won't be processed until the program moves out
> from an atomic step in a Python program.
>
> Probably this is partly my fault as I use some blocking statements
> (without timeout) as they should not block the program in normal
> situation.
>
> As the program freezes up, I cannot do anything except killing three
> processes using kill command.
> So, I cannot tell which statement is actually blocking when this
> occurs (and there are many suspects).
>
> Is there any good way to deal with this kind of problem?
> Killing three processes when it hangs is not a fun thing to do.
>
> Any suggestions will be greatly appreciated.
>

The scripts could write to a log around where they could block (write to
the log just before and just after), so that you can then see where they
blocked.

A hacky way of killing the processes would be for the processes to write
their process IDs to the log and then you can write another script which
will parse the log to get the process IDs and then kill the processes
with those IDs. (Left as an exercise for the reader. )
 
Reply With Quote
 
 
 
 
akineko
Guest
Posts: n/a
 
      01-28-2009
On Jan 28, 1:11*pm, MRAB <(E-Mail Removed)> wrote:
> A hacky way of killing the processes would be for the processes to write
> their process IDs to the log and then you can write another script which
> will parse the log to get the process IDs and then kill the processes
> with those IDs. (Left as an exercise for the reader.


Thank you MRAB.
Your suggestion is very reasonable. I think I will do both.
Try to reduce the chance of freeze-up and create a script to clean-up.

Aki-
 
Reply With Quote
 
Jesse Noller
Guest
Posts: n/a
 
      01-29-2009
On Wed, Jan 28, 2009 at 3:46 PM, akineko <(E-Mail Removed)> wrote:
> Hello Python experts,
>
> I have a program that uses three processes (invoked by
> multiprocessing) and several threads.
> The program is terminated when ^C is typed (KeyboardInterrupt).
> The main process takes the KeyboardInterrupt Exception and it orderly
> shutdown the program.
>
> It works fine in normal situation.
>
> However, KeyboardInterrupt is not accepted when, for example, the
> program is stuck somewhere due to error in network. I understand that
> the KeyboardInterrupt won't be processed until the program moves out
> from an atomic step in a Python program.
>
> Probably this is partly my fault as I use some blocking statements
> (without timeout) as they should not block the program in normal
> situation.
>
> As the program freezes up, I cannot do anything except killing three
> processes using kill command.
> So, I cannot tell which statement is actually blocking when this
> occurs (and there are many suspects).
>
> Is there any good way to deal with this kind of problem?
> Killing three processes when it hangs is not a fun thing to do.
>
> Any suggestions will be greatly appreciated.
>
> Best regards,
> Aki Niimura
>
> --
> http://mail.python.org/mailman/listinfo/python-list
>


See also:
http://jessenoller.com/2009/01/08/mu...oardinterrupt/

jesse
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
problem in running a basic code in python 3.3.0 that includes HTML file Satabdi Mukherjee Python 1 04-04-2013 07:48 PM
Terminating python script easily Balban Python 5 10-23-2009 01:47 AM
Terminating a thread when program exits. Ian Wilson Java 4 01-24-2007 10:53 AM
Terminating program with "exit( )" ern C Programming 19 03-24-2006 07:47 PM
Terminating program run from thread (not just the thread) Jeffrey Barish Python 0 05-28-2004 02:02 AM



Advertisments