Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > Pickling an extension type subclasses problems

Thread Tools

Pickling an extension type subclasses problems

Gaetan de Menten
Posts: n/a
Hi all,

I am trying to write an (optional!) C extension for SQLAlchemy, and I
am struggling to make an extension type picklable *and* using the same
format as the pure Python version (so that computers with the C
extension can unpickle pickles from computers without it and
vice-versa). Also the extension type (MyBase in the example code
below) only serves as a base class for the type that get pickled
(MyPythonType). MyPythonType also has further subclasses and those
should be picklable too.

My setup is as follow:

On the Python side:

from cextension import MyBase
except ImportError:
class MyBase(object):
__slots__ = ('a', 'b', 'c')
def __init__(self, a, b, c):
self.a, self.b, self.c = a, b, c

def __getstate__(self):

def __setstate__(self, state):

class MyPythonType(MyBase):

On the .c side:
I implemented a reduce method, which returns

Py_BuildValue("(O(s)N)", Py_TYPE(self), "__state__", state);

and in my BaseRowProxy_init method, I check the number of arguments,
whether the "__state__" marker is present and unpack the state if it
is. This makes the type nicely pickleable, but this is obviously not
the same "pickle format" as the Python version.

What would you recommend in this kind of situation? I'm a bit tired of
trial and error... My last hope is to define __reduce__ on the Python
side too, and make it use a module-level "reconstructor" function as

def mypythontype_reconstructor(cls, state):
obj = object.__new__(cls, state):
return obj

Will this work? Any other idea?

Thanks in advance,
GaŽtan de Menten
Reply With Quote

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off

Similar Threads
Thread Thread Starter Forum Replies Last Post
c extension, classes and subclasses Rolando Abarca Ruby 2 07-12-2009 04:17 PM
pickling extension class harold fellermann Python 4 01-18-2005 09:38 PM
pickling extension types Python 0 08-10-2004 11:34 AM
pickling subclasses of dict/list Edward Loper Python 1 07-04-2004 03:28 PM
problem pickling objects created with the type function danny Python 2 04-23-2004 11:34 AM