Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > pickling instances of metaclass generated classes

Reply
Thread Tools

pickling instances of metaclass generated classes

 
 
lars van gemerden
Guest
Posts: n/a
 
      12-29-2011
Hello,

Can someone help me with the following:

I am using metaclasses to make classes and these classes to make
instances. Now I want to use multiprocessing, which needs to pickle
these instances.

Pickle cannot find the class definitions of the instances. I am trying
to add a line to the __new__ of the metaclass to add the new class
under the right name in the right module/place, so pickle can find
it.

Is this the right approach? Can anyone explain to me where/how to add
these classes for pickle to find and maybe why?

Thanks in advance,

Lars
 
Reply With Quote
 
 
 
 
Robert Kern
Guest
Posts: n/a
 
      12-29-2011
On 12/29/11 9:55 AM, lars van gemerden wrote:
> Hello,
>
> Can someone help me with the following:
>
> I am using metaclasses to make classes and these classes to make
> instances. Now I want to use multiprocessing, which needs to pickle
> these instances.
>
> Pickle cannot find the class definitions of the instances. I am trying
> to add a line to the __new__ of the metaclass to add the new class
> under the right name in the right module/place, so pickle can find
> it.
>
> Is this the right approach? Can anyone explain to me where/how to add
> these classes for pickle to find and maybe why?


Can you post some code (preferably pared down to a minimal example that fails)?
I'm not really clear on what you are doing. I would expect that a class defined
by a class statement would usually work fine unless if the metaclass is doing
something particularly weird to it.

In any case, you can probably just explicitly register a reduction function for
each type using copy_reg.pickle():

http://docs.python.org/library/copy_reg

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco

 
Reply With Quote
 
 
 
 
Ian Kelly
Guest
Posts: n/a
 
      12-29-2011
On Thu, Dec 29, 2011 at 2:55 AM, lars van gemerden <(E-Mail Removed)> wrote:
> Hello,
>
> Can someone help me with the following:
>
> I am using metaclasses to make classes and these classes to make
> instances. Now I want to use multiprocessing, which needs to pickle
> these instances.
>
> Pickle cannot find the class definitions of the instances. I am trying
> to add a line to the __new__ of the metaclass to add the new class
> under the right name in the right module/place, so pickle can find
> it.
>
> Is this the right approach? Can anyone explain to me where/how to add
> these classes for pickle to find and maybe why?


It sounds like you're trying to do something like this?

>>> class MetaClass(type):

.... pass
....
>>> instance = MetaClass('<Anonymous>', (object,), {})()
>>> instance

<__main__.<Anonymous> object at 0x00CC00F0>
>>> import pickle
>>> pickle.dumps(instance)

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "c:\python27\lib\pickle.py", line 1374, in dumps
Pickler(file, protocol).dump(obj)
File "c:\python27\lib\pickle.py", line 224, in dump
self.save(obj)
File "c:\python27\lib\pickle.py", line 331, in save
self.save_reduce(obj=obj, *rv)
File "c:\python27\lib\pickle.py", line 401, in save_reduce
save(args)
File "c:\python27\lib\pickle.py", line 286, in save
f(self, obj) # Call unbound method with explicit self
File "c:\python27\lib\pickle.py", line 562, in save_tuple
save(element)
File "c:\python27\lib\pickle.py", line 295, in save
self.save_global(obj)
File "c:\python27\lib\pickle.py", line 748, in save_global
(obj, module, name))
pickle.PicklingError: Can't pickle <class '__main__.<Anonymous>'>:
it's not found as __main__.<Anonymous>


Yeah, pickle's not going to work with anonymous classes. As you
suggest, you could dynamically add the classes to the module namespace
so that pickle.dumps will find them, but bear in mind that they will
also have to exist when calling pickle.loads, so you will need to be
able to reconstruct the same anonymous classes before unpickling later
on.

Cheers,
Ian
 
Reply With Quote
 
lars van gemerden
Guest
Posts: n/a
 
      12-30-2011
On Dec 29, 8:55*pm, Ian Kelly <(E-Mail Removed)> wrote:
> On Thu, Dec 29, 2011 at 2:55 AM, lars van gemerden <(E-Mail Removed)>wrote:
>
> > Hello,

>
> > Can someone help me with the following:

>
> > I am using metaclasses to make classes and these classes to make
> > instances. Now I want to use multiprocessing, which needs to pickle
> > these instances.

>
> > Pickle cannot find the class definitions of the instances. I am trying
> > to add a line to the __new__ of the metaclass to add the new class
> > under the right name in the right module/place, so pickle can find
> > it.

>
> > Is this the right approach? Can anyone explain to me where/how to add
> > these classes for pickle to find and maybe why?

>
> It sounds like you're trying to do something like this?
>
> >>> class MetaClass(type):

>
> ... * * pass
> ...>>> instance = MetaClass('<Anonymous>', (object,), {})()
> >>> instance

>
> <__main__.<Anonymous> object at 0x00CC00F0>>>> import pickle
> >>> pickle.dumps(instance)

>
> Traceback (most recent call last):
> * File "<stdin>", line 1, in <module>
> * File "c:\python27\lib\pickle.py", line 1374, in dumps
> * * Pickler(file, protocol).dump(obj)
> * File "c:\python27\lib\pickle.py", line 224, in dump
> * * self.save(obj)
> * File "c:\python27\lib\pickle.py", line 331, in save
> * * self.save_reduce(obj=obj, *rv)
> * File "c:\python27\lib\pickle.py", line 401, in save_reduce
> * * save(args)
> * File "c:\python27\lib\pickle.py", line 286, in save
> * * f(self, obj) # Call unbound method with explicit self
> * File "c:\python27\lib\pickle.py", line 562, in save_tuple
> * * save(element)
> * File "c:\python27\lib\pickle.py", line 295, in save
> * * self.save_global(obj)
> * File "c:\python27\lib\pickle.py", line 748, in save_global
> * * (obj, module, name))
> pickle.PicklingError: Can't pickle <class '__main__.<Anonymous>'>:
> it's not found as __main__.<Anonymous>
>
> Yeah, pickle's not going to work with anonymous classes. *As you
> suggest, you could dynamically add the classes to the module namespace
> so that pickle.dumps will find them, but bear in mind that they will
> also have to exist when calling pickle.loads, so you will need to be
> able to reconstruct the same anonymous classes before unpickling later
> on.
>
> Cheers,
> Ian


Thank you Ian for the minimal example. This is almost the case i was
trying to discribe, however the classes will be named at runtime and
the class definitions will be persisted in a database.

Can you help me with how to add the classes to the correct namespace?
The application spans multiple modules (or compared to the example,
the metaclass definition will be in another module then one where the
class and instance will be generated).

Cheers, Lars
 
Reply With Quote
 
lars van gemerden
Guest
Posts: n/a
 
      12-30-2011
On Dec 29, 8:55*pm, Ian Kelly <(E-Mail Removed)> wrote:
> On Thu, Dec 29, 2011 at 2:55 AM, lars van gemerden <(E-Mail Removed)>wrote:
>
> > Hello,

>
> > Can someone help me with the following:

>
> > I am using metaclasses to make classes and these classes to make
> > instances. Now I want to use multiprocessing, which needs to pickle
> > these instances.

>
> > Pickle cannot find the class definitions of the instances. I am trying
> > to add a line to the __new__ of the metaclass to add the new class
> > under the right name in the right module/place, so pickle can find
> > it.

>
> > Is this the right approach? Can anyone explain to me where/how to add
> > these classes for pickle to find and maybe why?

>
> It sounds like you're trying to do something like this?
>
> >>> class MetaClass(type):

>
> ... * * pass
> ...>>> instance = MetaClass('<Anonymous>', (object,), {})()
> >>> instance

>
> <__main__.<Anonymous> object at 0x00CC00F0>>>> import pickle
> >>> pickle.dumps(instance)

>
> Traceback (most recent call last):
> * File "<stdin>", line 1, in <module>
> * File "c:\python27\lib\pickle.py", line 1374, in dumps
> * * Pickler(file, protocol).dump(obj)
> * File "c:\python27\lib\pickle.py", line 224, in dump
> * * self.save(obj)
> * File "c:\python27\lib\pickle.py", line 331, in save
> * * self.save_reduce(obj=obj, *rv)
> * File "c:\python27\lib\pickle.py", line 401, in save_reduce
> * * save(args)
> * File "c:\python27\lib\pickle.py", line 286, in save
> * * f(self, obj) # Call unbound method with explicit self
> * File "c:\python27\lib\pickle.py", line 562, in save_tuple
> * * save(element)
> * File "c:\python27\lib\pickle.py", line 295, in save
> * * self.save_global(obj)
> * File "c:\python27\lib\pickle.py", line 748, in save_global
> * * (obj, module, name))
> pickle.PicklingError: Can't pickle <class '__main__.<Anonymous>'>:
> it's not found as __main__.<Anonymous>
>
> Yeah, pickle's not going to work with anonymous classes. *As you
> suggest, you could dynamically add the classes to the module namespace
> so that pickle.dumps will find them, but bear in mind that they will
> also have to exist when calling pickle.loads, so you will need to be
> able to reconstruct the same anonymous classes before unpickling later
> on.
>
> Cheers,
> Ian


Ian also wrote:

'''
Actually, I was wrong, you probably don't need to do that. I suggest
going with Robert Kern's suggestion to either register the class with
the copy_reg module, or (perhaps better since it won't leak
registrations) implement a __reduce__ method on the class. For
example, this seems to work:

>>> def reconstructor(*metaclass_args):

.... cls = MetaClass.build_class(*metaclass_args)
.... self = cls.__new__(cls)
.... return self
....
>>> class MetaClass(type):

.... @classmethod
.... def build_class(mcs, arg1, arg2, arg3):
.... # Do something useful with the args...
.... class _AnonymousClass(object):
.... __metaclass__ = mcs
.... def __reduce__(self):
.... return (reconstructor, ('foo', 'bar', 'baz'),
self.__dict__)
.... return _AnonymousClass
....
>>> instance = MetaClass.build_class('foo', 'bar', 'baz')()
>>> instance

<__main__._AnonymousClass object at 0x011DB410>
>>> instance.banana = 42
>>> import pickle
>>> s = pickle.dumps(instance)
>>> s

"c__main__\nreconstructor
\np0\n(S'foo'\np1\nS'bar'\np2\nS'baz'\np3\ntp4\nRp 5\n(dp6\nS'banana'\np7\nI42\nsb."
>>> inst2 = pickle.loads(s)
>>> inst2

<__main__._AnonymousClass object at 0x011DBE90>
>>> inst2.banana

42
>>> inst2.__class__ is instance.__class__

False

Cheers,
Ian

'''
 
Reply With Quote
 
Peter Otten
Guest
Posts: n/a
 
      12-30-2011
lars van gemerden wrote:

> On Dec 29, 8:55 pm, Ian Kelly <(E-Mail Removed)> wrote:
>> On Thu, Dec 29, 2011 at 2:55 AM, lars van gemerden <(E-Mail Removed)>
>> wrote:
>>
>> > Hello,

>>
>> > Can someone help me with the following:

>>
>> > I am using metaclasses to make classes and these classes to make
>> > instances. Now I want to use multiprocessing, which needs to pickle
>> > these instances.

>>
>> > Pickle cannot find the class definitions of the instances. I am trying
>> > to add a line to the __new__ of the metaclass to add the new class
>> > under the right name in the right module/place, so pickle can find
>> > it.

>>
>> > Is this the right approach? Can anyone explain to me where/how to add
>> > these classes for pickle to find and maybe why?

>>
>> It sounds like you're trying to do something like this?
>>
>> >>> class MetaClass(type):

>>
>> ... pass
>> ...>>> instance = MetaClass('<Anonymous>', (object,), {})()
>> >>> instance

>>
>> <__main__.<Anonymous> object at 0x00CC00F0>>>> import pickle
>> >>> pickle.dumps(instance)

>>
>> Traceback (most recent call last):
>> File "<stdin>", line 1, in <module>
>> File "c:\python27\lib\pickle.py", line 1374, in dumps
>> Pickler(file, protocol).dump(obj)
>> File "c:\python27\lib\pickle.py", line 224, in dump
>> self.save(obj)
>> File "c:\python27\lib\pickle.py", line 331, in save
>> self.save_reduce(obj=obj, *rv)
>> File "c:\python27\lib\pickle.py", line 401, in save_reduce
>> save(args)
>> File "c:\python27\lib\pickle.py", line 286, in save
>> f(self, obj) # Call unbound method with explicit self
>> File "c:\python27\lib\pickle.py", line 562, in save_tuple
>> save(element)
>> File "c:\python27\lib\pickle.py", line 295, in save
>> self.save_global(obj)
>> File "c:\python27\lib\pickle.py", line 748, in save_global
>> (obj, module, name))
>> pickle.PicklingError: Can't pickle <class '__main__.<Anonymous>'>:
>> it's not found as __main__.<Anonymous>
>>
>> Yeah, pickle's not going to work with anonymous classes. As you
>> suggest, you could dynamically add the classes to the module namespace
>> so that pickle.dumps will find them, but bear in mind that they will
>> also have to exist when calling pickle.loads, so you will need to be
>> able to reconstruct the same anonymous classes before unpickling later
>> on.
>>
>> Cheers,
>> Ian

>
> Thank you Ian for the minimal example. This is almost the case i was
> trying to discribe, however the classes will be named at runtime and
> the class definitions will be persisted in a database.
>
> Can you help me with how to add the classes to the correct namespace?
> The application spans multiple modules (or compared to the example,
> the metaclass definition will be in another module then one where the
> class and instance will be generated).


If the metaclass is global in whatever module you don't need to bother about
that. The problem is that you cannot access the classes (metaclass
instances) under a dotted name. One workaround is a pseudo-module that does
whatever is needed to recreate the class:

import pickle
import sys

class MetaClass(type):
pass

class M(object):
def __init__(self, module):
self.__module = module
def __getattr__(self, name):
print "creating class", name
class_ = MetaClass(name, (), {"__module__": self.__module})
setattr(self, name, class_)
return class_

sys.modules["m"] = M("m")
import m
c = m.x
s = pickle.dumps(c)
print repr(s)
d = pickle.loads(s)

assert c is d

sys.modules["m"] = M("m")
e = pickle.loads(s)

assert c is not e

The official way is probably what Robert mentioned, via the copy_reg module,
but I didn't get it to work.

 
Reply With Quote
 
lars van gemerden
Guest
Posts: n/a
 
      12-30-2011
On Dec 30, 12:16*pm, lars van gemerden <(E-Mail Removed)> wrote:
> On Dec 29, 8:55*pm, Ian Kelly <(E-Mail Removed)> wrote:
>
>
>
>
>
>
>
>
>
> > On Thu, Dec 29, 2011 at 2:55 AM, lars van gemerden <(E-Mail Removed)> wrote:

>
> > > Hello,

>
> > > Can someone help me with the following:

>
> > > I am using metaclasses to make classes and these classes to make
> > > instances. Now I want to use multiprocessing, which needs to pickle
> > > these instances.

>
> > > Pickle cannot find the class definitions of the instances. I am trying
> > > to add a line to the __new__ of the metaclass to add the new class
> > > under the right name in the right module/place, so pickle can find
> > > it.

>
> > > Is this the right approach? Can anyone explain to me where/how to add
> > > these classes for pickle to find and maybe why?

>
> > It sounds like you're trying to do something like this?

>
> > >>> class MetaClass(type):

>
> > ... * * pass
> > ...>>> instance = MetaClass('<Anonymous>', (object,), {})()
> > >>> instance

>
> > <__main__.<Anonymous> object at 0x00CC00F0>>>> import pickle
> > >>> pickle.dumps(instance)

>
> > Traceback (most recent call last):
> > * File "<stdin>", line 1, in <module>
> > * File "c:\python27\lib\pickle.py", line 1374, in dumps
> > * * Pickler(file, protocol).dump(obj)
> > * File "c:\python27\lib\pickle.py", line 224, in dump
> > * * self.save(obj)
> > * File "c:\python27\lib\pickle.py", line 331, in save
> > * * self.save_reduce(obj=obj, *rv)
> > * File "c:\python27\lib\pickle.py", line 401, in save_reduce
> > * * save(args)
> > * File "c:\python27\lib\pickle.py", line 286, in save
> > * * f(self, obj) # Call unbound method with explicit self
> > * File "c:\python27\lib\pickle.py", line 562, in save_tuple
> > * * save(element)
> > * File "c:\python27\lib\pickle.py", line 295, in save
> > * * self.save_global(obj)
> > * File "c:\python27\lib\pickle.py", line 748, in save_global
> > * * (obj, module, name))
> > pickle.PicklingError: Can't pickle <class '__main__.<Anonymous>'>:
> > it's not found as __main__.<Anonymous>

>
> > Yeah, pickle's not going to work with anonymous classes. *As you
> > suggest, you could dynamically add the classes to the module namespace
> > so that pickle.dumps will find them, but bear in mind that they will
> > also have to exist when calling pickle.loads, so you will need to be
> > able to reconstruct the same anonymous classes before unpickling later
> > on.

>
> > Cheers,
> > Ian

> Ian also wrote:
>
> '''
> Actually, I was wrong, you probably don't need to do that. *I suggest
> going with Robert Kern's suggestion to either register the class with
> the copy_reg module, or (perhaps better since it won't leak
> registrations) implement a __reduce__ method on the class. *For
> example, this seems to work:
>
> >>> def reconstructor(*metaclass_args):

>
> ... * * cls = MetaClass.build_class(*metaclass_args)
> ... * * self = cls.__new__(cls)
> ... * * return self
> ...>>> class MetaClass(type):
>
> ... * * @classmethod
> ... * * def build_class(mcs, arg1, arg2, arg3):
> ... * * * * # Do something useful with the args...
> ... * * * * class _AnonymousClass(object):
> ... * * * * * * __metaclass__ = mcs
> ... * * * * * * def __reduce__(self):
> ... * * * * * * * * return (reconstructor, ('foo', 'bar','baz'),
> self.__dict__)
> ... * * * * return _AnonymousClass
> ...>>> instance = MetaClass.build_class('foo', 'bar', 'baz')()
> >>> instance

>
> <__main__._AnonymousClass object at 0x011DB410>>>> instance.banana = 42
> >>> import pickle
> >>> s = pickle.dumps(instance)
> >>> s

>
> "c__main__\nreconstructor
> \np0\n(S'foo'\np1\nS'bar'\np2\nS'baz'\np3\ntp4\nRp 5\n(dp6\nS'banana'\np7\nI 42\nsb.">>> inst2 = pickle.loads(s)
> >>> inst2

>
> <__main__._AnonymousClass object at 0x011DBE90>>>> inst2.banana
> 42
> >>> inst2.__class__ is instance.__class__

>
> False
>
> Cheers,
> Ian
>
> '''


Interesting, though I cannot say I completely understand this solution
(looked up __reduce__, but still). I am trying to adapt this example
to a situation where the metaclass generated classes are named at
runtime (not anonymous), but cannot figure it out.

Cheers, Lars
 
Reply With Quote
 
lars van gemerden
Guest
Posts: n/a
 
      12-30-2011
On Dec 30, 4:56*pm, lars van gemerden <(E-Mail Removed)> wrote:
> On Dec 30, 12:16*pm, lars van gemerden <(E-Mail Removed)> wrote:
>
>
>
>
>
>
>
>
>
> > On Dec 29, 8:55*pm, Ian Kelly <(E-Mail Removed)> wrote:

>
> > > On Thu, Dec 29, 2011 at 2:55 AM, lars van gemerden <(E-Mail Removed)> wrote:

>
> > > > Hello,

>
> > > > Can someone help me with the following:

>
> > > > I am using metaclasses to make classes and these classes to make
> > > > instances. Now I want to use multiprocessing, which needs to pickle
> > > > these instances.

>
> > > > Pickle cannot find the class definitions of the instances. I am trying
> > > > to add a line to the __new__ of the metaclass to add the new class
> > > > under the right name in the right module/place, so pickle can find
> > > > it.

>
> > > > Is this the right approach? Can anyone explain to me where/how to add
> > > > these classes for pickle to find and maybe why?

>
> > > It sounds like you're trying to do something like this?

>
> > > >>> class MetaClass(type):

>
> > > ... * * pass
> > > ...>>> instance = MetaClass('<Anonymous>', (object,), {})()
> > > >>> instance

>
> > > <__main__.<Anonymous> object at 0x00CC00F0>>>> import pickle
> > > >>> pickle.dumps(instance)

>
> > > Traceback (most recent call last):
> > > * File "<stdin>", line 1, in <module>
> > > * File "c:\python27\lib\pickle.py", line 1374, in dumps
> > > * * Pickler(file, protocol).dump(obj)
> > > * File "c:\python27\lib\pickle.py", line 224, in dump
> > > * * self.save(obj)
> > > * File "c:\python27\lib\pickle.py", line 331, in save
> > > * * self.save_reduce(obj=obj, *rv)
> > > * File "c:\python27\lib\pickle.py", line 401, in save_reduce
> > > * * save(args)
> > > * File "c:\python27\lib\pickle.py", line 286, in save
> > > * * f(self, obj) # Call unbound method with explicit self
> > > * File "c:\python27\lib\pickle.py", line 562, in save_tuple
> > > * * save(element)
> > > * File "c:\python27\lib\pickle.py", line 295, in save
> > > * * self.save_global(obj)
> > > * File "c:\python27\lib\pickle.py", line 748, in save_global
> > > * * (obj, module, name))
> > > pickle.PicklingError: Can't pickle <class '__main__.<Anonymous>'>:
> > > it's not found as __main__.<Anonymous>

>
> > > Yeah, pickle's not going to work with anonymous classes. *As you
> > > suggest, you could dynamically add the classes to the module namespace
> > > so that pickle.dumps will find them, but bear in mind that they will
> > > also have to exist when calling pickle.loads, so you will need to be
> > > able to reconstruct the same anonymous classes before unpickling later
> > > on.

>
> > > Cheers,
> > > Ian

> > Ian also wrote:

>
> > '''
> > Actually, I was wrong, you probably don't need to do that. *I suggest
> > going with Robert Kern's suggestion to either register the class with
> > the copy_reg module, or (perhaps better since it won't leak
> > registrations) implement a __reduce__ method on the class. *For
> > example, this seems to work:

>
> > >>> def reconstructor(*metaclass_args):

>
> > ... * * cls = MetaClass.build_class(*metaclass_args)
> > ... * * self = cls.__new__(cls)
> > ... * * return self
> > ...>>> class MetaClass(type):

>
> > ... * * @classmethod
> > ... * * def build_class(mcs, arg1, arg2, arg3):
> > ... * * * * # Do something useful with the args...
> > ... * * * * class _AnonymousClass(object):
> > ... * * * * * * __metaclass__ = mcs
> > ... * * * * * * def __reduce__(self):
> > ... * * * * * * * * return (reconstructor, ('foo', 'bar', 'baz'),
> > self.__dict__)
> > ... * * * * return _AnonymousClass
> > ...>>> instance = MetaClass.build_class('foo', 'bar', 'baz')()
> > >>> instance

>
> > <__main__._AnonymousClass object at 0x011DB410>>>> instance.banana = 42
> > >>> import pickle
> > >>> s = pickle.dumps(instance)
> > >>> s

>
> > "c__main__\nreconstructor
> > \np0\n(S'foo'\np1\nS'bar'\np2\nS'baz'\np3\ntp4\nRp 5\n(dp6\nS'banana'\np7\nI 42\nsb.">>> inst2 = pickle.loads(s)
> > >>> inst2

>
> > <__main__._AnonymousClass object at 0x011DBE90>>>> inst2.banana
> > 42
> > >>> inst2.__class__ is instance.__class__

>
> > False

>
> > Cheers,
> > Ian

>
> > '''

>
> Interesting, though I cannot say I completely understand this solution
> (looked up __reduce__, but still). I am trying to adapt this example
> to a situation where the metaclass generated classes are named at
> runtime (not anonymous), but cannot figure it out.
>
> Cheers, Lars


Found a way to name the classes:

def reconstructor(*metaclass_args):
cls = MetaClass2.build_class(*metaclass_args)
self = cls.__new__(cls)
return self

class MetaClass(type):
@classmethod
def build_class(mcs, name, arg1, arg2, arg3):
return mcs(name, (object,), {"__reduce__": lambda e:
(reconstructor2, (name, arg1, arg2, arg3), e.__dict__)})

I still wonder whether it might be easier to add the class to the
namespace. Can anyone help me with that?

Regards, Lars
 
Reply With Quote
 
Ian Kelly
Guest
Posts: n/a
 
      12-30-2011
On Fri, Dec 30, 2011 at 9:51 AM, lars van gemerden <(E-Mail Removed)> wrote:
> I still wonder whether it might be easier to add the class to the
> namespace. Can anyone help me with that?


from mypackage import mymodule

setattr(mymodule, myclass.__name__, myclass)
 
Reply With Quote
 
Peter Otten
Guest
Posts: n/a
 
      01-01-2012
lars van gemerden wrote:

>> import pickle
>> import sys
>>
>> class MetaClass(type):
>> pass
>>
>> class M(object):
>> def __init__(self, module):
>> self.__module = module
>> def __getattr__(self, name):
>> print "creating class", name
>> class_ = MetaClass(name, (), {"__module__": self.__module})
>> setattr(self, name, class_)
>> return class_
>>
>> sys.modules["m"] = M("m")
>> import m
>> c = m.x
>> s = pickle.dumps(c)
>> print repr(s)
>> d = pickle.loads(s)
>>
>> assert c is d
>>
>> sys.modules["m"] = M("m")
>> e = pickle.loads(s)
>>
>> assert c is not e
>>
>> The official way is probably what Robert mentioned, via the copy_reg
>> module, but I didn't get it to work.

>
> I will look further into this. does "sys.modules["m"] = M("m")" create
> a new module?


Assigning to sys.modules[modulename] can put arbitrary objects into the
module cache, in this case an M instance. To drive the point home:

>>> import sys
>>> sys.modules["x"] = 42
>>> import x
>>> x

42
>>> sys.modules["x"] = "spam"
>>> import x
>>> x

'spam'

> Cheers, Lars
>
> PS: I get an error when posting this to the usenet group


Sorry, that seems to happen when I post via gmane and don't manually clear
the follow-up that my newsreader helpfully (knode) inserts. I've not yet
found a permanent fix, but if that was the problem you should be able to
answer this post.

 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Metaclass of a metaclass Steven D'Aprano Python 1 06-05-2012 03:30 PM
Pickling classes (not class instances) Nicolas M. Thiéry Python 2 02-14-2009 09:26 AM
Pickling dynamically generated classes George Sakkis Python 0 01-26-2008 07:48 AM
Pickling limitation with instances defining __cmp__/__hash__? Erik Max Francis Python 1 08-09-2005 03:25 AM
metaclass that inherits a class of that metaclass? ironfroggy Python 16 06-03-2005 10:00 AM



Advertisments