Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > Re: properties + types, implementing meta-class desciptors elegantly?

Reply
Thread Tools

Re: properties + types, implementing meta-class desciptors elegantly?

 
 
Aahz
Guest
Posts: n/a
 
      07-20-2003
In article <(E-Mail Removed)>,
Mike C. Fletcher <(E-Mail Removed)> wrote:
>
> * finally, stores the value
> o tries to do what would have been done if there were no
> descriptor (with the new, coerced value)
> o does *not* create new names in the object's namespace (all
> names are documented w/ descriptors, there's not a lot of
> '_' prefixed names cluttering the namespace)
> o does *not* require a new dictionary/storage-object attribute
> for the object (the descriptor works like any other
> descriptor, a *stand-alone* object that replaces a regular
> attribute)


But this is a recipe for name clashes. If you follow the first bullet,
it's a normal attribute, but everything else says you want a property.
Properties are themselves objects that are attached to names in an
object. You can't have the same name bound to two different objects.
--
Aahz ((E-Mail Removed)) <*> http://www.pythoncraft.com/

This is Python. We don't care much about theory, except where it intersects
with useful practice. --Aahz
 
Reply With Quote
 
 
 
 
Mike C. Fletcher
Guest
Posts: n/a
 
      07-20-2003
Michele Simionato wrote:

>"Mike C. Fletcher" <(E-Mail Removed)> wrote in message news:<(E-Mail Removed)>...
><snip>
>
>As others before me, I am not sure of what you are trying to do
>and why you are doing so. I use descriptors in this way:
>
>class o(object):
> class p( object ): # descriptor class
> def __set__( self, client, value, *args, **named ):
> print '__set__', self, client, value, args, named
> self.v=value
> def __get__(self,obj,cls):
> return self.v
> v = p()
>

You are aware that you have just created a class-attribute, rather than
an instance attribute? The property/descriptor object exists within the
class 'o' namespace, not the instance namespace. Try creating two
instances of your o class and setting v on both of them. There is one
'p' instance "self" to which you are assigning/retrieving an attribute
for all instances of o. Creating class variables is certainly a valid
use for descriptors, but I'm not sure if that's really what you were
trying to do.

What I'm looking for, in terms of your code, is a method/function which
would do the proper thing instead of using "self.v=value", (or rather
client.__dict__['v'] = value (i.e. store the value in the client
dictionary)) for all of the major built-in types, such as
object-instances, classes (and object-instances with slots would be
nice). To the best of my knowledge, such a function does not exist
within Python; for instances, simply doing instance.__dict__[ key ] =
value is sufficient, but classes do not have a method exposed AFAIK
which allows an equivalent setting of a value *without* triggering the
descriptor machinery for the given key.

I'm creating a slightly more involved pattern, by the way:

class plugin( type ):
someHelperClass = common.ClassByNameProperty(
"someHelperClass", """Documentation for this meta-property""",
defaultValue = "some.package.module.ClassName",
setDefaultOnGet = 0,
)

class MyWorkingPlugIn( myBaseImplementation ):
__metaclass__ = plugin

instance = MyWorkingPlugIn()

Where the properties of the plugin meta-class are providing all sorts of
services for the MyWorkingPlugIn class, such as allowing it to find
"someHelperClass" related to the plug-in system while allowing that
property to be set manually if desired, or automatically calculating a
global identifier if the plug-in doesn't currently have a global
identifier. The value is primarily that the meta-class will use exactly
the same mechanisms as the rest of the system, and so will be readily
dealt with by the property-based meta-application system.

There are certainly other ways to get around the particular problem (I
have already done that), I'm looking for the *elegant* solution to the
*general* case.

Enjoy,
Mike

_______________________________________
Mike C. Fletcher
Designer, VR Plumber, Coder
http://members.rogers.com/mcfletch/




 
Reply With Quote
 
 
 
 
Bengt Richter
Guest
Posts: n/a
 
      07-20-2003
On 20 Jul 2003 09:39:51 -0400, http://www.velocityreviews.com/forums/(E-Mail Removed) (Aahz) wrote:

>In article <(E-Mail Removed)>,
>Mike C. Fletcher <(E-Mail Removed)> wrote:
>>
>> * finally, stores the value
>> o tries to do what would have been done if there were no
>> descriptor (with the new, coerced value)
>> o does *not* create new names in the object's namespace (all
>> names are documented w/ descriptors, there's not a lot of
>> '_' prefixed names cluttering the namespace)
>> o does *not* require a new dictionary/storage-object attribute
>> for the object (the descriptor works like any other
>> descriptor, a *stand-alone* object that replaces a regular
>> attribute)

>
>But this is a recipe for name clashes. If you follow the first bullet,
>it's a normal attribute, but everything else says you want a property.
>Properties are themselves objects that are attached to names in an
>object. You can't have the same name bound to two different objects.


I was wondering if Mike was wanting to have class C(...) with property p
(meaning the property object is an attribute of class C or a base), and
wanting also to have the instance dict contain the data as c.p rather
than, say, c._p.

IOW, does he want a sneaky way to avoid seeing the underscored or otherwise
modified names in the object, so as to have a clean dir(c)?

Other than the problem of getting the class defined in a dynamically configurable way
via metaclasses or other machinery, c.__dict__['p'] = value_from_property_fset should
work, IWT. The code below does that, though a little more parameterized

What I'm not getting is why a dict proxy is getting in the way, unless there is an
attempt to use the property mechanism to store properties-as-values in a class-as-instance.

If so, does he (continuing 3rd person since I'm replying to AAhz. Hi Mike really want
all that dynamic meta-stuff to happen every time a class is instatiated, or should the
classes-to-use be created once at some configuration time (maybe system start or other
special times), and thereafter appear to be equivalent to plain valnilla manually written
classes with properties?

If the latter, some kind of factory module that manufactures configured classes with
configured properties and puts them in its global name space to be used in apparently
normal "import m; inst=m.C()" fashion would seem better than metaclass inheritance magic behind C.
If there are other configuration times than first-import, the module could have a special
factory functions for that, IWT (as in cpc.py below). The properties of the final
configured/manufactured classes could of course act with any degree of dynamism desired
when accessed as attributes of the final class instances.

BTW, do I recall somewhere seeing that defining __dict__ as a slot induces creation of
a writable dict, and could that be used to get a dict in place of proxy for the trouble spot?

For Mike:

What does the following not address (I assume you can rearrange things to your taste
that you want to do? (InfoSource is my placeholder as an example for whatever dynamic
configuration of the ultimate class you want to do).


====< cpc.py >================================================= ================
def getCustPropClass(infoSource):
class MC_CustPropBase(type):
"""Metaclass to provide basic customized property methods and data"""
def __new__(cls, name, bases, cdict):
if name == 'CustPropBase': return type.__new__(cls, name, bases, cdict)
cdict['__name__'] = name
cdict['__doc__'] = infoSource[name, '__doc__']
# was raw_input('Base Doc string for property %r: '%name)
# let subclasses "underride" default and config
if 'default' not in cdict:
cdict['default'] = infoSource[name, 'default']
# was raw_input('Base Default for property %r: '%name)
if 'config' not in cdict:
cdict['config'] = infoSource[name, 'config']
# was raw_input('Base Config data for property %r: '%name)
def __set__(self, client, val):
client.__dict__[self.__name__] = val
def __get__(self, client, ctype=None):
if client is None: return self
if self.__name__ not in client.__dict__: return self.default
return client.__dict__[self.__name__]
def __delete__(self, client):
if self.__name__ not in client.__dict__:
raise AttributeError, 'Property %r has no instance data to delete.' % self.__name__
del client.__dict__[self.__name__]
for method in '__set__ __get__ __delete__'.split():
if method in cdict: continue # don't override
cdict[method] = vars()[method]
return type.__new__(cls, name, bases, cdict)

class CustPropBase(object): __metaclass__ = MC_CustPropBase

class MC_ConfProps(type):
"""
Configurable property definitions to be instantiated in
the instances of (this meta) class.
"""
def __new__(cls, name, bases, cdict):
docs = filter(None, [cdict.get('__doc__', '')])
for k,v in MC_ConfProps.__dict__.items(): #walk prop defs in this class
if isinstance(v, type) and (hasattr(v,'__set__') or hasattr(v, '__get__')):
cdict[k] = v()
docs.append('\n see also %s.%s.__doc__ for property %r'%(name, k, k))
cdict['__doc__'] = ''.join(docs)
return type.__new__(cls, name, bases, cdict)

class p(CustPropBase):
"""property p direct doc string will be overwritten by CustPropBase"""
default = infoSource['p','default']
# was raw_input('Property p: "Underride" of Base default value: ')
conf = infoSource['p','conf']
# was raw_input('Property p: Special Config data: ')
def __set__(self, client, val): # special set method differs from CustPropBase
client.__dict__['p'] = (
'(%r) modified w/ overridden base default(%r) data plus\n'
'base config(%r) & special config(%r) data.' %
( val, self.default, self.config, self.conf))

class q(CustPropBase): pass # totally default

class r(CustPropBase): # totally default except read-only
def __set__(self, client, val):
raise AttributeError, 'Property %r is read-only' % self.__name__

class ConfiguredProperties(object): __metaclass__ = MC_ConfProps

class C(ConfiguredProperties):
"""Empty test class C inheriting from ConfiguredProperties"""
return C

class InteractiveInfo(object):
def __getitem__(self, ixtup): # expect (propname, dataname) tuple as key
return raw_input('Enter value for property %r attribute %r: '% ixtup) #(propname,dataname)

testInfo = {
('p','__doc__'): '<p doc string>',
('p','default'): '<p default value>',
('p','config'): '<p config value>',
('p','conf'): '<p special conf value>',
('q','__doc__'): '<q doc string>',
('q','default'): ('<q default value>', (lambda x: 123456789),'(not just strings '),
('q','config'): '<q config value>',
('r','__doc__'): '<r doc string>',
('r','default'): '<r default value>',
('r','config'): '<r config value>',
}

def test():
for info_source in (testInfo, InteractiveInfo()):
C = getCustPropClass(info_source)
c = C()
print 'intitial p:', c.p
print 'intitial q:', c.q
print 'intitial r:', c.r
print 'try to set them to 123 and retrieve'
for prop in 'pqr':
print 'setting %s to 123'%prop
try:
setattr(c, prop, 123)
except Exception, e:
print '%s: %s'% (e.__class__.__name__, e)
print 'getting %s:\n----\n%s\n----' %(prop, getattr(c,prop,'???'))
print 'try to delete them and retrieve'
for prop in 'pqr':
print 'deleting %s'%prop
try:
delattr(c, prop)
except Exception, e:
print '%s: %s'% (e.__class__.__name__, e)
print 'after deleting %s:\n----\n%s\n----' %(prop, getattr(c,prop,'???'))
print 'c.__doc__\n', c.__doc__
for prop in 'pqr':
print getattr(C, prop).__doc__

if __name__ == '__main__':
test()
================================================== =============================
An interactive run (with automatic test data first):

[15:46] C:\pywk\clp\fletcher>cfc.py
intitial p: <p default value>
intitial q: ('<q default value>', <function <lambda> at 0x00800730>, '(not just strings ')
intitial r: <r default value>
try to set them to 123 and retrieve
setting p to 123
getting p:
----
(123) modified w/ overridden base default('<p default value>') data plus
base config('<p config value>') & special config('<p special conf value>') data.
----
setting q to 123
getting q:
----
123
----
setting r to 123
AttributeError: Property 'r' is read-only
getting r:
----
<r default value>
----
try to delete them and retrieve
deleting p
after deleting p:
----
<p default value>
----
deleting q
after deleting q:
----
('<q default value>', <function <lambda> at 0x00800730>, '(not just strings ')
----
deleting r
AttributeError: Property 'r' has no instance data to delete.
after deleting r:
----
<r default value>
----
c.__doc__
Empty test class C inheriting from ConfiguredProperties
see also C.q.__doc__ for property 'q'
see also C.p.__doc__ for property 'p'
see also C.r.__doc__ for property 'r'
<p doc string>
<q doc string>
<r doc string>
Enter value for property 'p' attribute 'default': [p default]
Enter value for property 'p' attribute 'conf': [p conf]
Enter value for property 'p' attribute '__doc__': [p doc]
Enter value for property 'p' attribute 'config': [p config]
Enter value for property 'q' attribute '__doc__': [q doc]
Enter value for property 'q' attribute 'default': [q default]
Enter value for property 'q' attribute 'config': [q config]
Enter value for property 'r' attribute '__doc__': [r doc]
Enter value for property 'r' attribute 'default': [r default]
Enter value for property 'r' attribute 'config': [r config]
intitial p: [p default]
intitial q: [q default]
intitial r: [r default]
try to set them to 123 and retrieve
setting p to 123
getting p:
----
(123) modified w/ overridden base default('[p default]') data plus
base config('[p config]') & special config('[p conf]') data.
----
setting q to 123
getting q:
----
123
----
setting r to 123
AttributeError: Property 'r' is read-only
getting r:
----
[r default]
----
try to delete them and retrieve
deleting p
after deleting p:
----
[p default]
----
deleting q
after deleting q:
----
[q default]
----
deleting r
AttributeError: Property 'r' has no instance data to delete.
after deleting r:
----
[r default]
----
c.__doc__
Empty test class C inheriting from ConfiguredProperties
see also C.q.__doc__ for property 'q'
see also C.p.__doc__ for property 'p'
see also C.r.__doc__ for property 'r'
[p doc]
[q doc]
[r doc]

Regards,
Bengt Richter
 
Reply With Quote
 
Michele Simionato
Guest
Posts: n/a
 
      07-21-2003
"Mike C. Fletcher" <(E-Mail Removed)> wrote in message news:<(E-Mail Removed)>...
> Michele Simionato wrote:
>
> >"Mike C. Fletcher" <(E-Mail Removed)> wrote in message news:<(E-Mail Removed)>...
> ><snip>
> >
> >As others before me, I am not sure of what you are trying to do
> >and why you are doing so. I use descriptors in this way:
> >
> >class o(object):
> > class p( object ): # descriptor class
> > def __set__( self, client, value, *args, **named ):
> > print '__set__', self, client, value, args, named
> > self.v=value
> > def __get__(self,obj,cls):
> > return self.v
> > v = p()
> >

> You are aware that you have just created a class-attribute, rather than
> an instance attribute? The property/descriptor object exists within the
> class 'o' namespace, not the instance namespace. Try creating two
> instances of your o class and setting v on both of them. There is one
> 'p' instance "self" to which you are assigning/retrieving an attribute
> for all instances of o. Creating class variables is certainly a valid
> use for descriptors, but I'm not sure if that's really what you were
> trying to do.


I have just copied your code, I thought you wanted a class attribute.
It is is trivial to do the same for instance attributes:

class o(object):
class p( object ): # descriptor class
def __set__( self, client, value, *args, **named ):
print '__set__', self, client, value, args, named
self.value=value
def __get__(self,obj,cls):
return self.value

def __init__(self):
self.v = self.p()

I understand that you aware of this, but you don't like it. Still I
do not understand why do you feel it to be ugly and/or inelegant. It
seems to me quite idiomatic.

> What I'm looking for, in terms of your code, is a method/function which
> would do the proper thing instead of using "self.v=value", (or rather
> client.__dict__['v'] = value (i.e. store the value in the client
> dictionary)) for all of the major built-in types, such as
> object-instances, classes (and object-instances with slots would be
> nice). To the best of my knowledge, such a function does not exist
> within Python; for instances, simply doing instance.__dict__[ key ] =
> value is sufficient, but classes do not have a method exposed AFAIK
> which allows an equivalent setting of a value *without* triggering the
> descriptor machinery for the given key.


Yes, I understand you want to be able to set class dictionaries just
as object dictionaries, bypassing descriptors. Still, I am not sure
if this would be a good idea.

> I'm creating a slightly more involved pattern, by the way:
>
> class plugin( type ):
> someHelperClass = common.ClassByNameProperty(
> "someHelperClass", """Documentation for this meta-property""",
> defaultValue = "some.package.module.ClassName",
> setDefaultOnGet = 0,
> )
>
> class MyWorkingPlugIn( myBaseImplementation ):
> __metaclass__ = plugin
>
> instance = MyWorkingPlugIn()
>
> Where the properties of the plugin meta-class are providing all sorts of
> services for the MyWorkingPlugIn class, such as allowing it to find
> "someHelperClass" related to the plug-in system while allowing that
> property to be set manually if desired, or automatically calculating a
> global identifier if the plug-in doesn't currently have a global
> identifier. The value is primarily that the meta-class will use exactly
> the same mechanisms as the rest of the system, and so will be readily
> dealt with by the property-based meta-application system.
>
> There are certainly other ways to get around the particular problem (I
> have already done that), I'm looking for the *elegant* solution to the
> *general* case.
>


Elegance is in the eye of the beholder

> Enjoy,
> Mike
>
> _______________________________________
> Mike C. Fletcher
> Designer, VR Plumber, Coder
> http://members.rogers.com/mcfletch/


Good luck with your project,

Michele
 
Reply With Quote
 
Mike C. Fletcher
Guest
Posts: n/a
 
      07-22-2003
Michele Simionato wrote:

>"Mike C. Fletcher" <(E-Mail Removed)> wrote in message news:<(E-Mail Removed)>...
>
>

....

>>You are aware that you have just created a class-attribute, rather than
>>an instance attribute? The property/descriptor object exists within the
>>class 'o' namespace, not the instance namespace. Try creating two
>>instances of your o class and setting v on both of them. There is one
>>'p' instance "self" to which you are assigning/retrieving an attribute
>>for all instances of o. Creating class variables is certainly a valid
>>use for descriptors, but I'm not sure if that's really what you were
>>trying to do.
>>
>>

>
>I have just copied your code, I thought you wanted a class attribute.
>

I should have been more precise. Your value is stored in the
descriptor, so it is not a per-class attribute, it is an attribute
stored in one place for *all* instances of all sub-classes of the
defined class, it is not a property of the class, but an effectively
static variable in the class definition. That is, each instance
modifies the one copy of the value, regardless of the instance' class.
I'm looking for a meta-property, a property of a class, rather than a
property of a class-instance. Changing an instance's value for that
name shouldn't affect this value. Changing the meta-property value
should only occur when you do setattr( cls, key, value ), and should
store the value in the *particular* concrete class having the value set,
not on some ancestor of that class, just as if you were to do setattr(
cls, key, value) without any meta-descriptor being present.

>It is is trivial to do the same for instance attributes:
>
>class o(object):
> class p( object ): # descriptor class
> def __set__( self, client, value, *args, **named ):
> print '__set__', self, client, value, args, named
> self.value=value
> def __get__(self,obj,cls):
> return self.value
>
> def __init__(self):
> self.v = self.p()
>
>I understand that you aware of this, but you don't like it. Still I
>do not understand why do you feel it to be ugly and/or inelegant. It
>seems to me quite idiomatic.
>
>

It's just not what I'm trying to do (create active properties on
meta-classes). It's a perfectly fine way to create class-static
variables shared among a hierarchy of instance objects.

>Yes, I understand you want to be able to set class dictionaries just
>as object dictionaries, bypassing descriptors. Still, I am not sure
>if this would be a good idea.
>

Well, just to clean up the descriptor API it's probably worth it IMO,
but I only seem to have Michael to back me up on that so far .
There's a missing level of functionality/hooks that haven't been exposed
and should be exposed to allow more general & orthogonal operation of
descriptor code.

Basically, it should be possible to mimic the built-in behaviour of
setattr/getattr w/out resorting to horrific hacks just to get that
behaviour. In other words, it should be possible to interpose an
operation in the descriptor-processing stream in much the same way that
super() allows for co-operative mix-in classes to inter-operate by
saying 'okay, do what would have been done if this "filter" method
didn't exist'.

>Elegance is in the eye of the beholder
>

Indeed.

>Good luck with your project,
>

Thanks. Enjoy,
Mike



 
Reply With Quote
 
Aahz
Guest
Posts: n/a
 
      07-25-2003
In article <(E-Mail Removed)>,
Mike C. Fletcher <(E-Mail Removed)> wrote:
>Aahz wrote:
>>In article <(E-Mail Removed)>,
>>Mike C. Fletcher <(E-Mail Removed)> wrote:
>>>
>>> * finally, stores the value
>>> o tries to do what would have been done if there were no
>>> descriptor (with the new, coerced value)
>>> o does *not* create new names in the object's namespace (all
>>> names are documented w/ descriptors, there's not a lot of
>>> '_' prefixed names cluttering the namespace)
>>> o does *not* require a new dictionary/storage-object attribute
>>> for the object (the descriptor works like any other
>>> descriptor, a *stand-alone* object that replaces a regular
>>> attribute)

>>
>>But this is a recipe for name clashes. If you follow the first bullet,
>>it's a normal attribute, but everything else says you want a property.
>>Properties are themselves objects that are attached to names in an
>>object. You can't have the same name bound to two different objects.

>
>It's true that:
>
> * your class has a namespace, and there are objects stored in that
> namespace
> o the objects stored in that namespace often are descriptors
> o they are available to the instances which do not shadow them
> (as I said, I can deal with this, and it's cleaner anyway).
> + if they are descriptors, they modify instance access
> to attributes


That's technically inaccurate as phrased. First of all, not all
descriptors get activated through attribute access, only data
descriptors do. Data descriptors are descriptors that define a ``set``
attribute (which is essentially a method of the descriptor object).
Secondly, it muddies understanding to say that the descriptor modifies
access to attributes, as if the descriptor is a proxy for an attribute.
Data descriptors *replace* attributes, and if you want to store a value,
it's your responsibility to determine where that value is stored (and
how to retrieve it).

> * the metaclass instance's property *values* would be stored in the
> metaclass instance's dictionary, just as is normally done (the
> metaclass' property *descriptors* would be stored in the
> metaclass' dictionary), there's no new conflicts created here,
> everything just works like it does now, a lookup would look
> something like this:
>
> >>> x.y

>x doesn't have a __getattribute__, so see if it has a descriptor for 'y'


Actually, x is the *last* object checked, and __getattribute__() is
never called on instances.

> get type(x).y
>
> type(x) doesn't have a __getattribute__, so see if it has a
> descriptor for 'y'
>
> get type(type(x)).y -> this is just a simple descriptor, as
> there's no higher-level descriptor, has a __get__ method,
> calls it to retrieve type(x).y


That should be type(type(x).y), I think. (I'm not testing this in
code.)

> returns the currently stored entry 'y' in type(x)'s dictionary,
> which happens to be a descriptor
>
> has type(x).y, a descriptor with a __get__ method, calls it, gets
> back value stored in instance's dictionary


That's correct if type(x).y is a data descriptor; other types of
descriptors do get overridden by the instance.

>You wind up with some names that are not available for use *as
>properties* on instances *iff* you set a simple value for the properties
>of the meta-class instance, but you solve that the regular way, by
>making the value stored by the meta-class instance's property a
>descriptor, rather than a simple value. That's just the way classes
>work. Yes, it's a "name clash", but everything in classes/instances is
>name clashes <shrug>.


Instances don't have properties, only types do. A metaclass is "just" a
class that creates classes (which are themselves types with new-style
classes). Problems only show up when you want to access class behavior
*through* a class instance.

>Not a problem I can see in the problem I outlined just yet, there's
>still no low-level value-setting mechanism exposed,


The point is that there's no low-level value. You always have to
explicitly specify the name and namespace when you use properties.
--
Aahz ((E-Mail Removed)) <*> http://www.pythoncraft.com/

This is Python. We don't care much about theory, except where it intersects
with useful practice. --Aahz
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Unix file desciptors Lorenzo Chomp Ruby 0 03-15-2006 03:34 AM
CompositeControls: ViewState properties w/ Mapped properties probl =?Utf-8?B?Q2hyaXN0b3BoZSBQZWlsbGV0?= ASP .Net 1 01-19-2006 09:19 AM
Making Custom Control Properties Visible in Visual Studio's Properties Palette Nathan Sokalski ASP .Net 0 10-17-2005 02:05 AM
Problems parsing when Properties.dtd.properties Kent Lichty Java 0 04-16-2004 03:08 PM
Re: properties + types, implementing meta-class desciptors elegantly? Michael Hudson Python 3 07-22-2003 02:06 AM



Advertisments