Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > A little help with child processes.

Reply
Thread Tools

A little help with child processes.

 
 
Rech
Guest
Posts: n/a
 
      06-16-2004
Hi,
I need a little help here managing child processes in Python. I'm not so
skilled in system programming so I hope you can give me some good
suggestions.

I have a very CPU and memory intensive task that has to be repeated many
times (with different input parameters). So I've thought to write a
Python script that create a child process, wait for it to finish and
then starts again another child with different parameters.

The children processes will save all the results to the disk (using
cPickle module), so the parent process has to wait each child only and
then starts the next one. The problem is that I can't work out a
solution. Sorry, but system programming is not my job.

Any suggestions about how to realize that? A skeleton of the script will
just suffice.


Thanks in advance,
Andrea.
 
Reply With Quote
 
 
 
 
Larry Bates
Guest
Posts: n/a
 
      06-16-2004
Unless you are going to start more than one child
process in parallel and there is enough I/O to
make it worthwhile, there's no reason for using
child processes at all. Just program the application
as a single loop. You can't speed up CPU bound
applications with child processes.

HTH,
Larry Bates
Syscon, Inc.
"Rech" <(E-Mail Removed)> wrote in message
news:(E-Mail Removed)...
> Hi,
> I need a little help here managing child processes in Python. I'm not so
> skilled in system programming so I hope you can give me some good
> suggestions.
>
> I have a very CPU and memory intensive task that has to be repeated many
> times (with different input parameters). So I've thought to write a
> Python script that create a child process, wait for it to finish and
> then starts again another child with different parameters.
>
> The children processes will save all the results to the disk (using
> cPickle module), so the parent process has to wait each child only and
> then starts the next one. The problem is that I can't work out a
> solution. Sorry, but system programming is not my job.
>
> Any suggestions about how to realize that? A skeleton of the script will
> just suffice.
>
>
> Thanks in advance,
> Andrea.



 
Reply With Quote
 
 
 
 
Rech
Guest
Posts: n/a
 
      06-16-2004
Yes I know, but the CPU-intensive tasks eat a lot of memory and for
weird reasons they don't free it (I know bad programming, but re-writing
all the code is impratical at this point). Running them in child
processes solves the problem because when a child dies it frees all the
allocated memory.

Andrea.


In article <(E-Mail Removed)>,
"Larry Bates" <(E-Mail Removed)> wrote:

> Unless you are going to start more than one child
> process in parallel and there is enough I/O to
> make it worthwhile, there's no reason for using
> child processes at all. Just program the application
> as a single loop. You can't speed up CPU bound
> applications with child processes.
>
> HTH,
> Larry Bates
> Syscon, Inc.

 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
1 little 2 little 3 little Kennedys dale Digital Photography 0 03-23-2008 01:03 PM
HELP! Child controls of child controls not visible to web app lisa@starways.net ASP .Net Building Controls 0 05-05-2005 09:14 PM
little red X in little white box Puzzled Computer Support 8 12-13-2004 09:11 AM
[HELP] Why the child window reopen a new child window ? bhchng ASP .Net 0 01-13-2004 07:17 AM
How do I: Main thread spawn child threads, which child processes...control those child processes? Jeff Rodriguez C Programming 23 12-09-2003 11:06 PM



Advertisments