Velocity Reviews

Velocity Reviews (http://www.velocityreviews.com/forums/index.php)
-   Python (http://www.velocityreviews.com/forums/f43-python.html)
-   -   Re: Let child process to run while parent is out (multiprocessing) (http://www.velocityreviews.com/forums/t948299-re-let-child-process-to-run-while-parent-is-out-multiprocessing.html)

Kushal Kumaran 07-20-2012 08:48 AM

Re: Let child process to run while parent is out (multiprocessing)
 
On Fri, Jul 20, 2012 at 2:04 AM, John Wong <gokoproject@gmail.com> wrote:
> def main(...):
> build_id = create_build_id(...)
> build_stuff
> return build_id
>
> Suppose build_stuff compiles a C program. It could take days to finish, and
> notify users their builds are ready. I was thinking about using
> mutliprocessing to handle the build_stuff.
>
> So here is a sample:
>
> #!/usr/bin/python
>
> import multiprocessing as mp
> import time
>
> def build():
> print 'I am building HUGE things'
> time.sleep(10)
>
> def main():
> build_p = mp.Process(name='build process', target=build)
> build_p.start()
> return 'abcd12345'
>
> if __name__ == '__main__':
>
> v = main()
> print v
> print 'done'
>
> Here is output:
> yeukhon@fermat:~$ python c2.py
> abcd12345
> done [now hangs for 10 seconds]
> I build things
>
> When I looked at `ps -elf|grep python`, I can see two processes running, and
> one of the python c2.py process is `wait`. But this is not ideal,
> especially this is a web app. I can't implement any advanced queue / event
> system right now (I use Pylon, I believe I have gevent installed). But just
> with multiprocessing, is it possible to send the id first, while running
> child in the backgroud?
>
> Right now it hangs there as long as the child process is alive. Any
> solutions?
>


>From the documentation, there does not seem to be any way of

"detaching" a multiprocessing Process. But it is doable by using the
underlying os.fork directly (CAUTION: not ready for being invoked from
a web app):

#!/usr/bin/python

import os
import time

def build_and_send_email():
print 'I am building HUGE things'
time.sleep(10)

def main():
child_pid = os.fork()
if child_pid == 0:
build_and_send_email()
os._exit(0)
return 'abcd12345'

if __name__ == '__main__':
v = main()
print v
print 'done'

To make it work correctly with web app will require a bit more work.
At the least, you will have to close all file descriptors to make sure
the request processing finishes. You can turn it into a proper
background process (a daemon) using the python-daemon library with
very little code, I think.

--
regards,
kushal


All times are GMT. The time now is 08:55 PM.

Powered by vBulletin®. Copyright ©2000 - 2014, vBulletin Solutions, Inc.
SEO by vBSEO ©2010, Crawlability, Inc.