Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > Generic constructors and duplication of internal Python logic

Reply
Thread Tools

Generic constructors and duplication of internal Python logic

 
 
John J. Lee
Guest
Posts: n/a
 
      04-14-2004
This is one of those things that I can't quite believe I've never
needed to do before.

I've got a set classes, each of which has a set of attributes that all
behave very similarly. So, I have a class attribute (Blah.attr_spec
below), which is used by a mixin class to implement various methods
that would otherwise be highly repetitious across these classes. I'd
like to do the same for the constructor, to avoid this kind of
nonsense:

class Blah(NamesMixin):
attr_spec = ["foo", "bar", "baz",
("optional1", None), ("optional2", None)]
def __init__(self, foo, bar, baz,
optional1=None, optional2=None):
self.foo, self.bar, self.baz = \
foo, bar, baz
self.optional1, self.optional2 = \
optional1, optional2

So, I wrote a mixin class whose __init__ looks at the attr_spec
attribute, and uses args and kwds (below) to assign attributes in the
same sort of way as the special-case code above:

class ArgsMixin:
def __init__(self, *args, **kwds):
# set attributes based on arguments passed in, as done
# manually in Blah.__init__, above
... lots of logic already present in Python goes here...

That immediately leads to duplication of Python's internal logic: I
have to check things like:

-are there too many positional arguments?
-any unexpected keyword arguments?
-multiple keyword arguments?
-any duplication between positional and keyword arguments?

etc.

Surely there's some easy way of making use of Python's internal logic
here? For some reason, I can't see how. Can anybody see a way?


John
 
Reply With Quote
 
 
 
 
Peter Otten
Guest
Posts: n/a
 
      04-15-2004
John J. Lee wrote:

> This is one of those things that I can't quite believe I've never
> needed to do before.
>
> I've got a set classes, each of which has a set of attributes that all
> behave very similarly. So, I have a class attribute (Blah.attr_spec
> below), which is used by a mixin class to implement various methods
> that would otherwise be highly repetitious across these classes. I'd
> like to do the same for the constructor, to avoid this kind of
> nonsense:
>
> class Blah(NamesMixin):
> attr_spec = ["foo", "bar", "baz",
> ("optional1", None), ("optional2", None)]
> def __init__(self, foo, bar, baz,
> optional1=None, optional2=None):
> self.foo, self.bar, self.baz = \
> foo, bar, baz
> self.optional1, self.optional2 = \
> optional1, optional2
>
> So, I wrote a mixin class whose __init__ looks at the attr_spec
> attribute, and uses args and kwds (below) to assign attributes in the
> same sort of way as the special-case code above:
>
> class ArgsMixin:
> def __init__(self, *args, **kwds):
> # set attributes based on arguments passed in, as done
> # manually in Blah.__init__, above
> ... lots of logic already present in Python goes here...
>
> That immediately leads to duplication of Python's internal logic: I
> have to check things like:
>
> -are there too many positional arguments?
> -any unexpected keyword arguments?
> -multiple keyword arguments?
> -any duplication between positional and keyword arguments?
>
> etc.
>
> Surely there's some easy way of making use of Python's internal logic
> here? For some reason, I can't see how. Can anybody see a way?


You could use a noop method check_attrs() to define the argspec.
check_attrs() is then called from the mixin's __init__() just to check that
the parameters comply.

import inspect

def make_attrspec(f):
a = inspect.getargspec(f)
names = a[0]
if names[0] in ["self", "cls"]:
# XXX for now relies on naming convention
del names[0]
defaults = a[3]
for i in range(-1, -len(defaults)-1, -1):
names[i] = names[i], defaults[i]
return names

class ArgsMixin:
def __init__(self, *args, **kwds):
self.check_attrs(*args, **kwds)

class Blah(ArgsMixin):
def check_attrs(self, foo, bar, baz, optional1="first",
optional2="second"):
pass
attr_spec = make_attrspec(check_attrs)


print Blah.attr_spec
Blah(1, 2, 3)
Blah(1, 2, optional1="o1", baz=99)
Blah(1, 2)

I'm not sure whether check_attrs() should be a class or static method, so I
made it a standard method for now.
Todo: automagically "normalize" the argument list, e. g. convert
Blah(1, 2, optional1="o1", baz=99) to Blah(1, 2, 99, optional1="o1").
A workaround would be to make them all keyword arguments

kwds.update(dict(zip(Blah.attr_spec, args)))

after the self.check_attrs() call.

Peter

 
Reply With Quote
 
 
 
 
Michele Simionato
Guest
Posts: n/a
 
      04-15-2004
http://www.velocityreviews.com/forums/(E-Mail Removed) (John J. Lee) wrote in message news:<(E-Mail Removed)>...
> I have to check things like:
>
> -are there too many positional arguments?
> -any unexpected keyword arguments?
> -multiple keyword arguments?
> -any duplication between positional and keyword arguments?
>
> etc.
>
> Surely there's some easy way of making use of Python's internal logic
> here? For some reason, I can't see how. Can anybody see a way?
>
>
> John


Have you thought of performing the checks in the __call__ method
of a custom metaclass?


Michele Simionato
 
Reply With Quote
 
John J. Lee
Guest
Posts: n/a
 
      04-19-2004
Peter Otten <(E-Mail Removed)> writes:

> John J. Lee wrote:

[...]
> You could use a noop method check_attrs() to define the argspec.
> check_attrs() is then called from the mixin's __init__() just to check that
> the parameters comply.
>
> import inspect
>
> def make_attrspec(f):
> a = inspect.getargspec(f)
> names = a[0]
> if names[0] in ["self", "cls"]:
> # XXX for now relies on naming convention
> del names[0]
> defaults = a[3]
> for i in range(-1, -len(defaults)-1, -1):
> names[i] = names[i], defaults[i]
> return names
>
> class ArgsMixin:
> def __init__(self, *args, **kwds):
> self.check_attrs(*args, **kwds)
>
> class Blah(ArgsMixin):
> def check_attrs(self, foo, bar, baz, optional1="first",
> optional2="second"):
> pass
> attr_spec = make_attrspec(check_attrs)


Clever. But how to get __init__ to assign the arguments to the
instance?

b = Blah(foo=1, bar=2, optional1=4)
assert b.foo, b.bar, b.optional1 = 1, 2, 4


That half of the problem is missing in your solution.

[...]
> Todo: automagically "normalize" the argument list, e. g. convert
> Blah(1, 2, optional1="o1", baz=99) to Blah(1, 2, 99, optional1="o1").


As long as baz ends up getting assigned the value 99 and optional1
gets the value "o1", I don't care how that's achieved (except that I'd
like it done without having to write out all the logic as I have ATM
-- I want to reuse Python's own internal knowledge of argument lists
to get my attributes assigned).


> A workaround would be to make them all keyword arguments
>
> kwds.update(dict(zip(Blah.attr_spec, args)))
>
> after the self.check_attrs() call.


Not acceptable in my case.


John
 
Reply With Quote
 
John J. Lee
Guest
Posts: n/a
 
      04-19-2004
(E-Mail Removed) (Michele Simionato) writes:

> (E-Mail Removed) (John J. Lee) wrote in message news:<(E-Mail Removed)>...
> > I have to check things like:
> >
> > -are there too many positional arguments?
> > -any unexpected keyword arguments?
> > -multiple keyword arguments?
> > -any duplication between positional and keyword arguments?
> >
> > etc.
> >
> > Surely there's some easy way of making use of Python's internal logic
> > here? For some reason, I can't see how. Can anybody see a way?
> >
> >
> > John

>
> Have you thought of performing the checks in the __call__ method
> of a custom metaclass?


No. How would that help?


John
 
Reply With Quote
 
Michele Simionato
Guest
Posts: n/a
 
      04-20-2004
(E-Mail Removed) (John J. Lee) wrote in message news:<(E-Mail Removed)>...
> (E-Mail Removed) (Michele Simionato) writes:
> > Have you thought of performing the checks in the __call__ method
> > of a custom metaclass?

>
> No. How would that help?


I had in mind something like that:

class _WithConstructorChecked(type): # helper metaclass
def __call__(cls, *args, **kw):
assert len(args)<=2, "%s called with more than 2 args" % cls
assert kw.has_key("kw"), "%s needs a 'kw=' argument" % cls
return super(_WithConstructorChecked,cls).__call__(*args, **kw)

class WithConstructorChecked(object): # mixin class
__metaclass__ = _WithConstructorChecked

class C(WithConstructorChecked):
def __init__(self, *args, **kw):
pass

c=C(1,2,kw=3) # ok; try different signatures to get assertion errors

In this example the mixin class WithConstructorChecked ensures that C is
called with at least two positional arguments and a keyword argument named
'kw'. The code of class C is not touched at all, you just add
WithConstructorChecked to the list of its bases.

Is that what you are looking for?


Michele Simionato
 
Reply With Quote
 
Peter Otten
Guest
Posts: n/a
 
      04-20-2004
John J. Lee wrote:

> Peter Otten <(E-Mail Removed)> writes:
>
>> John J. Lee wrote:

> [...]
>> You could use a noop method check_attrs() to define the argspec.
>> check_attrs() is then called from the mixin's __init__() just to check
>> that the parameters comply.
>>
>> import inspect
>>
>> def make_attrspec(f):
>> a = inspect.getargspec(f)
>> names = a[0]
>> if names[0] in ["self", "cls"]:
>> # XXX for now relies on naming convention
>> del names[0]
>> defaults = a[3]
>> for i in range(-1, -len(defaults)-1, -1):
>> names[i] = names[i], defaults[i]
>> return names
>>
>> class ArgsMixin:
>> def __init__(self, *args, **kwds):
>> self.check_attrs(*args, **kwds)
>>
>> class Blah(ArgsMixin):
>> def check_attrs(self, foo, bar, baz, optional1="first",
>> optional2="second"):
>> pass
>> attr_spec = make_attrspec(check_attrs)

>
> Clever. But how to get __init__ to assign the arguments to the
> instance?
>
> b = Blah(foo=1, bar=2, optional1=4)
> assert b.foo, b.bar, b.optional1 = 1, 2, 4
>
>
> That half of the problem is missing in your solution.


I'll give it a try (untested):

class ArgsMixin:
def __init__(self, *args, **kwds):
self.check_attrs(*args, **kwds)
# positionals provided
for n, v in zip(self.attr_spec, args):
setattr(self, n, v)
# positionals using defaults
for nv in self.attr_spec[len(args):]:
if not isinstance(nv, basestring):
n, v = nv
if not n in kwds:
setattr(self, n, v)
# keyword args
for n, v in kwds.iteritems():
setattr(self, n, v)

The clumsy basestring check could be avoided if you either split attr_spec
in the tuple/non-tuple parts or precalculate the first non-tuple position.

Peter

 
Reply With Quote
 
John J. Lee
Guest
Posts: n/a
 
      04-22-2004
Peter Otten <(E-Mail Removed)> writes:

> John J. Lee wrote:
>
> > Peter Otten <(E-Mail Removed)> writes:
> >
> >> John J. Lee wrote:

> > [...]
> >> You could use a noop method check_attrs() to define the argspec.
> >> check_attrs() is then called from the mixin's __init__() just to check
> >> that the parameters comply.
> >>
> >> import inspect
> >>
> >> def make_attrspec(f):
> >> a = inspect.getargspec(f)
> >> names = a[0]
> >> if names[0] in ["self", "cls"]:
> >> # XXX for now relies on naming convention
> >> del names[0]
> >> defaults = a[3]
> >> for i in range(-1, -len(defaults)-1, -1):
> >> names[i] = names[i], defaults[i]
> >> return names
> >>
> >> class ArgsMixin:
> >> def __init__(self, *args, **kwds):
> >> self.check_attrs(*args, **kwds)
> >>
> >> class Blah(ArgsMixin):
> >> def check_attrs(self, foo, bar, baz, optional1="first",
> >> optional2="second"):
> >> pass
> >> attr_spec = make_attrspec(check_attrs)

> >
> > Clever. But how to get __init__ to assign the arguments to the
> > instance?
> >
> > b = Blah(foo=1, bar=2, optional1=4)
> > assert b.foo, b.bar, b.optional1 = 1, 2, 4
> >
> >
> > That half of the problem is missing in your solution.

>
> I'll give it a try (untested):
>
> class ArgsMixin:
> def __init__(self, *args, **kwds):
> self.check_attrs(*args, **kwds)
> # positionals provided
> for n, v in zip(self.attr_spec, args):
> setattr(self, n, v)


But the actual args may (quite legally) be longer than the args part
of attr_spec, and then:

>>> attr_spec = ["foo", "bar", ("a", "b")]
>>> args = [1, 2]
>>> zip(attr_spec, args)

[('foo', 1), ('bar', 2)]
>>> args = [1, 2, 3]
>>> zip(attr_spec, args)

[('foo', 1), ('bar', 2), (('a', 'b'), 3)] # wrong, can't do setattr!
>>>


This can be fixed, but then we move towards my clumsy-but-correct (I
hope) implementation. I'm disappointed there's no simple solution
that makes better use of Python's own knowledge of argument
specifications.


> # positionals using defaults
> for nv in self.attr_spec[len(args):]:
> if not isinstance(nv, basestring):
> n, v = nv
> if not n in kwds:
> setattr(self, n, v)
> # keyword args
> for n, v in kwds.iteritems():
> setattr(self, n, v)

[...]


John
 
Reply With Quote
 
John J. Lee
Guest
Posts: n/a
 
      04-22-2004
(E-Mail Removed) (Michele Simionato) writes:

> (E-Mail Removed) (John J. Lee) wrote in message news:<(E-Mail Removed)>...
> > (E-Mail Removed) (Michele Simionato) writes:
> > > Have you thought of performing the checks in the __call__ method
> > > of a custom metaclass?

> >
> > No. How would that help?

>
> I had in mind something like that:
>
> class _WithConstructorChecked(type): # helper metaclass
> def __call__(cls, *args, **kw):
> assert len(args)<=2, "%s called with more than 2 args" % cls
> assert kw.has_key("kw"), "%s needs a 'kw=' argument" % cls
> return super(_WithConstructorChecked,cls).__call__(*args, **kw)
>
> class WithConstructorChecked(object): # mixin class
> __metaclass__ = _WithConstructorChecked
>
> class C(WithConstructorChecked):
> def __init__(self, *args, **kw):
> pass
>
> c=C(1,2,kw=3) # ok; try different signatures to get assertion errors
>
> In this example the mixin class WithConstructorChecked ensures that C is
> called with at least two positional arguments and a keyword argument named
> 'kw'. The code of class C is not touched at all, you just add
> WithConstructorChecked to the list of its bases.
>
> Is that what you are looking for?


No.

See my replies to Peter: I'm trying to re-use Python's own knowledge
of to get attributes assigned, without repetition (see my original
post for the kind of repetition that motivated me), and without simply
liberally assigning as attributes any old arguments that get passed
in. Seems like it's not possible, though. I don't see how a
metaclass helps here.


John
 
Reply With Quote
 
Peter Otten
Guest
Posts: n/a
 
      04-23-2004
John J. Lee wrote:

[my buggy code]

> This can be fixed, but then we move towards my clumsy-but-correct (I
> hope) implementation. I'm disappointed there's no simple solution
> that makes better use of Python's own knowledge of argument
> specifications.


I warned you that my code was not thoroughly tested, and I aggree with you
that this is getting clumsy.
But will I stop trying to come up with something better? no way. Here's my
next try, and I'm likely to present something completely different again if
you don't like it - not sure if that's a promise or a threat

import inspect, sys

def generateSource(m):
a = inspect.getargspec(m)
names = a[0]
if a[1] or a[2]:
raise Execption("*args/**kw not supported")
nself = names[0]
assignments = [" %s.%s = %s" % (nself, n, n) for n in names[1:]]
return "def %s%s:\n%s" % (m.__name__,
inspect.formatargspec(*inspect.getargspec(m)),
"\n".join(assignments))

def generateMethod(m):
"""Generate a method with the same signiture
as m. For every argument arg a corresponding line

self.arg = arg

is added.
"""
# source code generation factored out for easier debugging
ns = {}
exec generateSource(m) in ns
return ns[m.__name__]

class Demo(object):
def __init__(self, foo, bar, baz=2, bang=3):
pass
__init__ = generateMethod(__init__)
def __str__(self):
return ", ".join(["%s=%r" % (n, getattr(self, n))
for n in "foo bar baz bang".split()])

print "--- generatatSource(Demo.__init__) ---"
print generateSource(Demo.__init__)
print "--------------------------------------"

print Demo(1, 2, 3)
print Demo(1, 2)
print Demo(1, 2, bang=99)
print Demo(1, bang=99, bar=11)
print Demo(1, bar=11)

Peter
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Python Logic Map/Logic Flow Chart. (Example Provided) spike Python 8 02-09-2010 12:31 PM
Is the possible to have all the public constructors of the publicbase class as the constructors of a derived class? Peng Yu C++ 5 09-19-2008 10:19 AM
compiler synthesized constructors/copy constructors/assignment operators Jess C++ 5 06-07-2007 11:09 AM
Copy constructors, de/constructors and reference counts Jeremy Smith C++ 2 08-02-2006 11:25 PM
Constructors that call other Constructors Dave Rudolf C++ 12 02-06-2004 03:26 PM



Advertisments