Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > reloading code and multiprocessing

Reply
Thread Tools

reloading code and multiprocessing

 
 
andrea crotti
Guest
Posts: n/a
 
      07-19-2012
We need to be able to reload code on a live system. This live system
has a daemon process always running but it runs many subprocesses with
multiprocessing, and the subprocesses might have a short life...

Now I found a way to reload the code successfully, as you can see from
this testcase:


def func():
from . import a
print(a.ret())


class TestMultiProc(unittest.TestCase):
def setUp(self):
open(path.join(cur_dir, 'a.py'), 'w').write(old_a)

def tearDown(self):
remove(path.join(cur_dir, 'a.py'))

def test_reloading(self):
"""Starting a new process gives a different result
"""
p1 = Process(target=func)
p2 = Process(target=func)
p1.start()
res = p1.join()
open(path.join(cur_dir, 'a.py'), 'w').write(new_a)
remove(path.join(cur_dir, 'a.pyc'))

p2.start()
res = p2.join()


As long as I import the code in the function and make sure to remove the
"pyc" files everything seems to work..
Are there any possible problems which I'm not seeing in this approach or
it's safe?

Any other better ways otherwise?
 
Reply With Quote
 
 
 
 
88888 Dihedral
Guest
Posts: n/a
 
      07-19-2012
andrea crotti於 2012年7月19日星期四UTC+8下午6時15分11秒 寫道:
> We need to be able to reload code on a live system. This live system
> has a daemon process always running but it runs many subprocesses with
> multiprocessing, and the subprocesses might have a short life...
>
> Now I found a way to reload the code successfully, as you can see from
> this testcase:
>
>
> def func():
> from . import a
> print(a.ret())
>
>
> class TestMultiProc(unittest.TestCase):
> def setUp(self):
> open(path.join(cur_dir, 'a.py'), 'w').write(old_a)
>
> def tearDown(self):
> remove(path.join(cur_dir, 'a.py'))
>
> def test_reloading(self):
> """Starting a new process gives a different result
> """
> p1 = Process(target=func)
> p2 = Process(target=func)
> p1.start()
> res = p1.join()
> open(path.join(cur_dir, 'a.py'), 'w').write(new_a)
> remove(path.join(cur_dir, 'a.pyc'))
>
> p2.start()
> res = p2.join()
>
>
> As long as I import the code in the function and make sure to remove the
> "pyc" files everything seems to work..
> Are there any possible problems which I'm not seeing in this approachor
> it's safe?
>
> Any other better ways otherwise?


If a byte code interpreter is embedded in the executable, then the program
can obtain or reload code objects in the run time.

In C/C++, unless one can swap some DLL safely or spawn in another process with
new executables obtained from other places, or a script interpreter is embedded, otherwise it's not easy for an instance in C/C++ to get new methods dynamically.

or

 
Reply With Quote
 
 
 
 
88888 Dihedral
Guest
Posts: n/a
 
      07-19-2012
andrea crotti於 2012年7月19日星期四UTC+8下午6時15分11秒 寫道:
> We need to be able to reload code on a live system. This live system
> has a daemon process always running but it runs many subprocesses with
> multiprocessing, and the subprocesses might have a short life...
>
> Now I found a way to reload the code successfully, as you can see from
> this testcase:
>
>
> def func():
> from . import a
> print(a.ret())
>
>
> class TestMultiProc(unittest.TestCase):
> def setUp(self):
> open(path.join(cur_dir, 'a.py'), 'w').write(old_a)
>
> def tearDown(self):
> remove(path.join(cur_dir, 'a.py'))
>
> def test_reloading(self):
> """Starting a new process gives a different result
> """
> p1 = Process(target=func)
> p2 = Process(target=func)
> p1.start()
> res = p1.join()
> open(path.join(cur_dir, 'a.py'), 'w').write(new_a)
> remove(path.join(cur_dir, 'a.pyc'))
>
> p2.start()
> res = p2.join()
>
>
> As long as I import the code in the function and make sure to remove the
> "pyc" files everything seems to work..
> Are there any possible problems which I'm not seeing in this approachor
> it's safe?
>
> Any other better ways otherwise?


If a byte code interpreter is embedded in the executable, then the program
can obtain or reload code objects in the run time.

In C/C++, unless one can swap some DLL safely or spawn in another process with
new executables obtained from other places, or a script interpreter is embedded, otherwise it's not easy for an instance in C/C++ to get new methods dynamically.

or

 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Re: reloading code and multiprocessing andrea crotti Python 0 07-27-2012 11:01 AM
Re: reloading code and multiprocessing andrea crotti Python 0 07-25-2012 10:46 AM
Re: reloading code and multiprocessing Chris Angelico Python 0 07-23-2012 02:15 PM
Re: reloading code and multiprocessing andrea crotti Python 0 07-23-2012 01:51 PM
Re: reloading code and multiprocessing Chris Angelico Python 0 07-20-2012 08:17 AM



Advertisments