Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > Re: Is there anything that pickle + copy_reg cannot serialize?

Reply
Thread Tools

Re: Is there anything that pickle + copy_reg cannot serialize?

 
 
Jean-Paul Calderone
Guest
Posts: n/a
 
      12-08-2005
On Fri, 09 Dec 2005 02:17:10 +0800, Maurice LING <(E-Mail Removed)> wrote:
>
>> Since copy_reg lets you specify arbitrary code to serialize arbitrary
>> objects, you shouldn't run into any single object that you cannot
>> serialize to a pickle.

>
> [snip - example of pickling code objects]
>
>
>I cannot understand 2 things, which I seek assistance for:
>1. Is code object the only thing can cannot be pickled (less facing
>recursion limits)?


No. There are lots of objects that cannot be pickled by default. Any
extension type which does not explicitly support it cannot be pickled.
Generators cannot be pickled. Method descriptors can't be pickled. Et
cetera.

>2. In the above example, how copy_reg works with pickle?


Any time pickle thinks it has found something it cannot pickle, it asks
the copy_reg module for some help. The above example basically teaches
the copy_reg module how to give the pickle module the help it needs for
code objects.

Jean-Paul
 
Reply With Quote
 
 
 
 
Maurice LING
Guest
Posts: n/a
 
      12-09-2005

>>> Since copy_reg lets you specify arbitrary code to serialize arbitrary
>>> objects, you shouldn't run into any single object that you cannot
>>> serialize to a pickle.

>>
>>
>> [snip - example of pickling code objects]
>>
>>
>> I cannot understand 2 things, which I seek assistance for:
>> 1. Is code object the only thing can cannot be pickled (less facing
>> recursion limits)?

>
>
> No. There are lots of objects that cannot be pickled by default. Any
> extension type which does not explicitly support it cannot be pickled.
> Generators cannot be pickled. Method descriptors can't be pickled. Et
> cetera.


Thank Jean-Paul.

Sorry for not specifying clearly enough. Given that copy_reg lets you
specify arbitrary code to serialize arbitrary objects, of which some are
taken care of by pickle, in the set of possible Python types,

>>> import types
>>> dir(types)

['BooleanType', 'BufferType', 'BuiltinFunctionType',
'BuiltinMethodType', 'ClassType', 'CodeType', 'ComplexType',
'DictProxyType', 'DictType', 'DictionaryType', 'EllipsisType',
'FileType', 'FloatType', 'FrameType', 'FunctionType', 'GeneratorType',
'InstanceType', 'IntType', 'LambdaType', 'ListType', 'LongType',
'MethodType', 'ModuleType', 'NoneType', 'NotImplementedType',
'ObjectType', 'SliceType', 'StringType', 'StringTypes', 'TracebackType',
'TupleType', 'TypeType', 'UnboundMethodType', 'UnicodeType',
'XRangeType', '__builtins__', '__doc__', '__file__', '__name__']

What types cannot be serialized by pickle (besides CodeType) and
requires handling using copy_reg?

Thanks
Maurice


>
>> 2. In the above example, how copy_reg works with pickle?

>
>
> Any time pickle thinks it has found something it cannot pickle, it asks
> the copy_reg module for some help. The above example basically teaches
> the copy_reg module how to give the pickle module the help it needs for
> code objects.
>
> Jean-Paul

 
Reply With Quote
 
 
 
 
Fredrik Lundh
Guest
Posts: n/a
 
      12-09-2005
Maurice LING wrote:

> Sorry for not specifying clearly enough. Given that copy_reg lets you
> specify arbitrary code to serialize arbitrary objects, of which some are
> taken care of by pickle, in the set of possible Python types,


the types module contains a selection of type objects; the set of possible
types that you may have in a Python program is unbounded.

> What types cannot be serialized by pickle (besides CodeType) and
> requires handling using copy_reg?


import types, pickle, sys

for t in dir(types):
try:
if t == "BufferType":
o = buffer("hello")
elif t == "EllipsisType":
o = Ellipsis
elif t == "FileType":
o = file("hello.txt", "w")
elif t == "NoneType":
o = None
elif t == "SliceType":
o = slice(0,0,0)
elif t == "XRangeType":
o = xrange(0,0,1)
else:
o = getattr(types, t)()
except:
print t, "FAILED:", sys.exc_value
else:
try:
s = pickle.dumps(o)
except:
print t, "FAILED:", str(sys.exc_value).upper()
else:
print t, "OK"

should give you an idea (feel free to add more object factories)

</F>



 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
problem in running a basic code in python 3.3.0 that includes HTML file Satabdi Mukherjee Python 1 04-04-2013 07:48 PM
pickle error: can't pickle instancemethod objects Michele Simionato Python 2 05-23-2008 08:29 AM
Re: Is there anything that pickle + copy_reg cannot serialize? Jean-Paul Calderone Python 1 12-08-2005 06:17 PM
Is there anything that pickle + copy_reg cannot serialize? Maurice LING Python 0 12-08-2005 02:42 PM
a pickle's pickle temposs@gmail.com Python 4 08-02-2005 07:20 PM



Advertisments