Velocity Reviews

Velocity Reviews (http://www.velocityreviews.com/forums/index.php)
-   Python (http://www.velocityreviews.com/forums/f43-python.html)
-   -   reloading code and multiprocessing (http://www.velocityreviews.com/forums/t948263-reloading-code-and-multiprocessing.html)

andrea crotti 07-19-2012 10:15 AM

reloading code and multiprocessing
 
We need to be able to reload code on a live system. This live system
has a daemon process always running but it runs many subprocesses with
multiprocessing, and the subprocesses might have a short life...

Now I found a way to reload the code successfully, as you can see from
this testcase:


def func():
from . import a
print(a.ret())


class TestMultiProc(unittest.TestCase):
def setUp(self):
open(path.join(cur_dir, 'a.py'), 'w').write(old_a)

def tearDown(self):
remove(path.join(cur_dir, 'a.py'))

def test_reloading(self):
"""Starting a new process gives a different result
"""
p1 = Process(target=func)
p2 = Process(target=func)
p1.start()
res = p1.join()
open(path.join(cur_dir, 'a.py'), 'w').write(new_a)
remove(path.join(cur_dir, 'a.pyc'))

p2.start()
res = p2.join()


As long as I import the code in the function and make sure to remove the
"pyc" files everything seems to work..
Are there any possible problems which I'm not seeing in this approach or
it's safe?

Any other better ways otherwise?

88888 Dihedral 07-19-2012 03:08 PM

Re: reloading code and multiprocessing
 
andrea crotti於 2012年7月19日星期四UTC+8下午6時15分11秒 寫道:
> We need to be able to reload code on a live system. This live system
> has a daemon process always running but it runs many subprocesses with
> multiprocessing, and the subprocesses might have a short life...
>
> Now I found a way to reload the code successfully, as you can see from
> this testcase:
>
>
> def func():
> from . import a
> print(a.ret())
>
>
> class TestMultiProc(unittest.TestCase):
> def setUp(self):
> open(path.join(cur_dir, 'a.py'), 'w').write(old_a)
>
> def tearDown(self):
> remove(path.join(cur_dir, 'a.py'))
>
> def test_reloading(self):
> """Starting a new process gives a different result
> """
> p1 = Process(target=func)
> p2 = Process(target=func)
> p1.start()
> res = p1.join()
> open(path.join(cur_dir, 'a.py'), 'w').write(new_a)
> remove(path.join(cur_dir, 'a.pyc'))
>
> p2.start()
> res = p2.join()
>
>
> As long as I import the code in the function and make sure to remove the
> "pyc" files everything seems to work..
> Are there any possible problems which I'm not seeing in this approachor
> it's safe?
>
> Any other better ways otherwise?


If a byte code interpreter is embedded in the executable, then the program
can obtain or reload code objects in the run time.

In C/C++, unless one can swap some DLL safely or spawn in another process with
new executables obtained from other places, or a script interpreter is embedded, otherwise it's not easy for an instance in C/C++ to get new methods dynamically.

or


88888 Dihedral 07-19-2012 03:08 PM

Re: reloading code and multiprocessing
 
andrea crotti於 2012年7月19日星期四UTC+8下午6時15分11秒 寫道:
> We need to be able to reload code on a live system. This live system
> has a daemon process always running but it runs many subprocesses with
> multiprocessing, and the subprocesses might have a short life...
>
> Now I found a way to reload the code successfully, as you can see from
> this testcase:
>
>
> def func():
> from . import a
> print(a.ret())
>
>
> class TestMultiProc(unittest.TestCase):
> def setUp(self):
> open(path.join(cur_dir, 'a.py'), 'w').write(old_a)
>
> def tearDown(self):
> remove(path.join(cur_dir, 'a.py'))
>
> def test_reloading(self):
> """Starting a new process gives a different result
> """
> p1 = Process(target=func)
> p2 = Process(target=func)
> p1.start()
> res = p1.join()
> open(path.join(cur_dir, 'a.py'), 'w').write(new_a)
> remove(path.join(cur_dir, 'a.pyc'))
>
> p2.start()
> res = p2.join()
>
>
> As long as I import the code in the function and make sure to remove the
> "pyc" files everything seems to work..
> Are there any possible problems which I'm not seeing in this approachor
> it's safe?
>
> Any other better ways otherwise?


If a byte code interpreter is embedded in the executable, then the program
can obtain or reload code objects in the run time.

In C/C++, unless one can swap some DLL safely or spawn in another process with
new executables obtained from other places, or a script interpreter is embedded, otherwise it's not easy for an instance in C/C++ to get new methods dynamically.

or



All times are GMT. The time now is 12:19 AM.

Powered by vBulletin®. Copyright ©2000 - 2014, vBulletin Solutions, Inc.
SEO by vBSEO ©2010, Crawlability, Inc.