Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > Descriptors and side effects

Reply
Thread Tools

Descriptors and side effects

 
 
mrkafk@gmail.com
Guest
Posts: n/a
 
      11-05-2007
Hello everyone,

I'm trying to do seemingly trivial thing with descriptors: have
another attribute updated on dot access in object defined using
descriptors.

For example, let's take a simple example where you set an attribute s
to a string and have another attribute l set automatically to its
length.

>>> class Desc(str):

def __init__(self,val):
self.s=val
self.l=len(val)
print "creating value: ", self.s
print "id(self.l)", id(self.l)
def __set__(self, obj, val):
self.s=val
self.l=len(val)
print "setting value:", self.s, "length:", self.l
def __get__(self, obj, type=None):
print "getting value:", self.s, "length:", self.l
return self.l


>>> class some(str):

m=Desc('abc')
l=m.l


creating value: abc
id(self.l) 10049688
>>> ta=some()
>>> ta.m='test string'

setting value: test string length: 11

However, the attribute ta.l didn't get updated:

>>> ta.l

3

This is so much weirder that object id of ta.l is the same as id of
instance of descriptor:

>>> id(ta.l)

10049688

A setter function should have updated self.l just like it updated
self.s:

def __set__(self, obj, val):
self.s=val
self.l=len(val)
print "setting value:", self.s, "length:", self.l

Yet it didn't happen.

>From my POV, the main benefit of a descriptor lies in its side effect:

on dot access (getting/setting) I can get other attributes updated
automatically: say, in class of Squares I get area automatically
updated on updating side, etc.

Yet, I'm struggling with getting it done in Python. Descriptors are a
great idea, but I would like to see them implemented in Python in a
way that makes it easier to get desireable side effects.

 
Reply With Quote
 
 
 
 
Bruno Desthuilliers
Guest
Posts: n/a
 
      11-05-2007
http://www.velocityreviews.com/forums/(E-Mail Removed) a écrit :
> Hello everyone,
>
> I'm trying to do seemingly trivial thing with descriptors: have
> another attribute updated on dot access in object defined using
> descriptors.
>
> For example, let's take a simple example where you set an attribute s
> to a string and have another attribute l set automatically to its
> length.
>
>>>> class Desc(str):

> def __init__(self,val):
> self.s=val
> self.l=len(val)
> print "creating value: ", self.s
> print "id(self.l)", id(self.l)
> def __set__(self, obj, val):
> self.s=val
> self.l=len(val)
> print "setting value:", self.s, "length:", self.l
> def __get__(self, obj, type=None):
> print "getting value:", self.s, "length:", self.l
> return self.l
>
>
>>>> class some(str):

> m=Desc('abc')
> l=m.l


First point : I don't get why Desc and some derive from str. Second
point: I don't get why you're storing the value and it's length in the
descriptor itself - obviously, you can't expect one descriptor instance
to be mapped to 2 distinct attributes. Third point: you understand that,
the way you wrote it, your descriptor will behave as a class (ie:shared)
attribute, don't you. Fourth point: if you hope some.l to be rebound
when m.l is, then you should learn Python basics before trying to jump
into descriptors.

The obvious, simple way to some your problem is to use a couple of
properties:

class Some(object):
@apply
def m():
def fget(self):
return self._m
def fset(self, val):
self._m = val
self._l = len(val)
return property(**locals())
@apply
def l():
def fget(self):
return self._l
def fset(self):
raise AttributeError("%s.l is readonly" % self)
def __init__(self, m):
self.m = m

Now if you absolutely insist on using custom descriptors, you'll need
two of them: one to manage access to s, and the second to manage access
to l (which btw is a very bad name):

class DescS(object):
def __init__(self, val):
self._default = val

def __set___(self, obj, val):
obj._s = val
obj._l = len(val)

def __get__(self, obj, cls):
if obj is None:
return self # or self._default, or whatever
try:
return obj._s
except AttributeError:
return self._default


class DescL(object):
def __init__(self, descS):
self._descS = descS
def __get__(self, obj, cls):
if obj is None:
return self # or self._default, or whatever
try:
return obj._l
except AttributeError:
return len(self._descS._default)


class Test(object):
m = DescS('abc')
l = DescL(m)

(nb : not tested)
 
Reply With Quote
 
 
 
 
Rich Harkins
Guest
Posts: n/a
 
      11-05-2007
(E-Mail Removed) wrote:
> Hello everyone,
>
> I'm trying to do seemingly trivial thing with descriptors: have
> another attribute updated on dot access in object defined using
> descriptors.


[snip]

> A setter function should have updated self.l just like it updated
> self.s:
>
> def __set__(self, obj, val):
> self.s=val
> self.l=len(val)
> print "setting value:", self.s, "length:", self.l
>
> Yet it didn't happen.
>

[snip]

I noticed that Python will block all attribute overrides (either via
__dict__ through setattr) if the property has a __set__ method. The
standard property has this method and there is no way that I can find to
defeat it. So, here is what I use:

class ConstProperty(object):
"""
Provides a property that keeps its return value. The function will
only be called on the first access. After that the same value can
be used over and over again with no function call penalty. If the
cached value needs to be cleared, simply del the attribute.

>>> class MyClass(object):

... def __init__(self, x):
... self.x = x
... @ConstProperty
... def y(self):
... print "HERE"
... return self.x ** 2
...
>>> obj = MyClass(5)
>>> obj.y

HERE
25
>>> obj.y

25
"""

def __init__(self, fn):
self.fn = fn

def __get__(self, target, cls=None):
if target is None:
return self.fn # Helps pydoc
else:
obj = self.fn(target)
setattr(target, self.fn.__name__, obj)
return obj

This is a little different than what you originally posted, but
hopefully it is close enough to be helpful.

Cheers!
Rich

 
Reply With Quote
 
Bruno Desthuilliers
Guest
Posts: n/a
 
      11-05-2007
Rich Harkins a écrit :
> (E-Mail Removed) wrote:
>> Hello everyone,
>>
>> I'm trying to do seemingly trivial thing with descriptors: have
>> another attribute updated on dot access in object defined using
>> descriptors.

>
> [snip]
>
>> A setter function should have updated self.l just like it updated
>> self.s:
>>
>> def __set__(self, obj, val):
>> self.s=val
>> self.l=len(val)
>> print "setting value:", self.s, "length:", self.l
>>
>> Yet it didn't happen.
>>

> [snip]
>
> I noticed that Python will block all attribute overrides (either via
> __dict__ through setattr) if the property has a __set__ method.


It doesn't "block", it controls access to... Of course, if the __set__
method is a no-op, then nothing will happen.

> The
> standard property has this method and there is no way that I can find to
> defeat it.


"defeat" ? Why don't you just pass the appropriate fset function to
property ?

> So, here is what I use:
>
> class ConstProperty(object):
> """
> Provides a property that keeps its return value. The function will
> only be called on the first access. After that the same value can
> be used over and over again with no function call penalty. If the
> cached value needs to be cleared, simply del the attribute.
>
> >>> class MyClass(object):

> ... def __init__(self, x):
> ... self.x = x
> ... @ConstProperty
> ... def y(self):
> ... print "HERE"
> ... return self.x ** 2
> ...
> >>> obj = MyClass(5)
> >>> obj.y

> HERE
> 25
> >>> obj.y

> 25
> """
>
> def __init__(self, fn):
> self.fn = fn
>
> def __get__(self, target, cls=None):
> if target is None:
> return self.fn # Helps pydoc
> else:
> obj = self.fn(target)
> setattr(target, self.fn.__name__, obj)
> return obj




>>> m = MyClass(5)
>>> m.__dict__

{'x': 5}
>>> m.y

HERE
25
>>> m.__dict__

{'y': 25, 'x': 5}
>>> m.x = 42
>>> m.y

25
>>> m.__dict__

{'y': 25, 'x': 42}
>>>



I'm sorry, but this looks like a very complicated way to do a simple thing:

class MySimpleClass(object):
def __init__(self, x):
self.x = x
self.y = x ** 2


 
Reply With Quote
 
Rich Harkins
Guest
Posts: n/a
 
      11-05-2007
Bruno Desthuilliers wrote:
[snip]
> I'm sorry, but this looks like a very complicated way to do a simple thing:
>
> class MySimpleClass(object):
> def __init__(self, x):
> self.x = x
> self.y = x ** 2
>
>


Sure, for the absurdly simplified case I posed as an example.

Here's another:

class Path(tuple):
@ConstProperty
def pathstr(self):
print "DEBUG: Generating string"
return '/'.join(self)

def __add__(self, other):
if isinstance(other, tuple):
return Path(tuple.__add__(self, other))
else:
return Path(tuple.__add__(self, (other,)))

>>> ROOT = Path(())
>>> path = ROOT + 'x' + 'y' + 'z'
>>> path.pathstr

DEBUG: Generating string
/x/y/z
>>> path.pathstr

/x/y/z

Basically, you can use ConstProperty above for items you don't want to
calculate automatically, but only when someone actually WANTS it. After
it is applied, then the penalties for function call of the property and
the computation are wiped out once the second access is requested.

Now, in the original example, len() might be considered too little for
this use and should be just generated in the constructor "for free".
OTOH, that assumes that __len__ hasn't been overridden to do something
more complicated and time consuming. If the antecedent object is
static, and the derivative consequent is also static, then ConstProperty
works very well and shouldn't cost more on the first access than any
other built-in property function.

BTW, another use is to avoid creating lots of unnecessary objects for
free unless they are accessed. Another quickie example:

class Node(object):
hasChildList = False
hasAttributesDict = False

@ConstProperty
def children(self):
self.hasChildList = True
return []

@ConstProperty
def attributes(self):
self.hasAttributesDict = True
return {}

The extra class/object attributes can be used to test for whether the
associated objects were created. When used in a large tree, not
creating a lot of extra lists and dictionaries can save a lot of memory
and CPU as the children and attributes are not created or explored
unless they were manipulated.

Rich
 
Reply With Quote
 
Rich Harkins
Guest
Posts: n/a
 
      11-05-2007
Bruno Desthuilliers wrote:
> Rich Harkins a écrit :
>> (E-Mail Removed) wrote:
>>> Hello everyone,
>>>
>>> I'm trying to do seemingly trivial thing with descriptors: have
>>> another attribute updated on dot access in object defined using
>>> descriptors.

>> [snip]
>>
>>> A setter function should have updated self.l just like it updated
>>> self.s:
>>>
>>> def __set__(self, obj, val):
>>> self.s=val
>>> self.l=len(val)
>>> print "setting value:", self.s, "length:", self.l
>>>
>>> Yet it didn't happen.
>>>

>> [snip]
>>
>> I noticed that Python will block all attribute overrides (either via
>> __dict__ through setattr) if the property has a __set__ method.

>
> It doesn't "block", it controls access to... Of course, if the __set__
> method is a no-op, then nothing will happen.
>
>> The
>> standard property has this method and there is no way that I can find to
>> defeat it.

>
> "defeat" ? Why don't you just pass the appropriate fset function to
> property ?
>
>> So, here is what I use:
>>
>> class ConstProperty(object):
>> """
>> Provides a property that keeps its return value. The function will
>> only be called on the first access. After that the same value can
>> be used over and over again with no function call penalty. If the
>> cached value needs to be cleared, simply del the attribute.
>>
>> >>> class MyClass(object):

>> ... def __init__(self, x):
>> ... self.x = x
>> ... @ConstProperty
>> ... def y(self):
>> ... print "HERE"
>> ... return self.x ** 2
>> ...
>> >>> obj = MyClass(5)
>> >>> obj.y

>> HERE
>> 25
>> >>> obj.y

>> 25
>> """
>>
>> def __init__(self, fn):
>> self.fn = fn
>>
>> def __get__(self, target, cls=None):
>> if target is None:
>> return self.fn # Helps pydoc
>> else:
>> obj = self.fn(target)
>> setattr(target, self.fn.__name__, obj)
>> return obj

>
>
>
> >>> m = MyClass(5)
> >>> m.__dict__

> {'x': 5}
> >>> m.y

> HERE
> 25
> >>> m.__dict__

> {'y': 25, 'x': 5}
> >>> m.x = 42
> >>> m.y

> 25
> >>> m.__dict__

> {'y': 25, 'x': 42}
> >>>

>
>
> I'm sorry, but this looks like a very complicated way to do a simple thing:
>
> class MySimpleClass(object):
> def __init__(self, x):
> self.x = x
> self.y = x ** 2
>
>


 
Reply With Quote
 
Bruno Desthuilliers
Guest
Posts: n/a
 
      11-05-2007
Rich Harkins a écrit :
> Bruno Desthuilliers wrote:
> [snip]
>
>>I'm sorry, but this looks like a very complicated way to do a simple thing:
>>
>>class MySimpleClass(object):
>> def __init__(self, x):
>> self.x = x
>> self.y = x ** 2
>>
>>

>
>
> Sure, for the absurdly simplified case I posed as an example.
>
> Here's another:
>
> class Path(tuple):


Ok, for an immutable type, it might eventually work.

> @ConstProperty
> def pathstr(self):
> print "DEBUG: Generating string"
> return '/'.join(self)


import os.path
help(os.path)

> def __add__(self, other):
> if isinstance(other, tuple):
> return Path(tuple.__add__(self, other))
> else:
> return Path(tuple.__add__(self, (other,)))
>
>
>>>>ROOT = Path(())
>>>>path = ROOT + 'x' + 'y' + 'z'
>>>>path.pathstr

>
> DEBUG: Generating string
> /x/y/z
>
>>>>path.pathstr

>
> /x/y/z


>>> p = Path(('home', 'bruno'))
>>> p += ['toto', 'tata']
>>> p.pathstr

DEBUG: Generating string
Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "/usr/tmp/python-8690chu", line 31, in __get__
File "/usr/tmp/python-8690chu", line 40, in pathstr
TypeError: sequence item 2: expected string, list found
>>>



> Basically, you can use ConstProperty above for items you don't want to
> calculate automatically, but only when someone actually WANTS it.


Which is easy to do with properties too.

> After
> it is applied, then the penalties for function call of the property and
> the computation are wiped out once the second access is requested.


Agreed. But I wouldn't use such a scheme for mutable types - which are
still the common case.

> Now, in the original example, len() might be considered too little for
> this use and should be just generated in the constructor "for free".
> OTOH, that assumes that __len__ hasn't been overridden to do something
> more complicated and time consuming. If the antecedent object is
> static, and the derivative consequent is also static,


You mean 'immutable', I assume...

> then ConstProperty
> works very well and shouldn't cost more on the first access than any
> other built-in property function.
>
> BTW, another use is to avoid creating lots of unnecessary objects for
> free unless they are accessed. Another quickie example:
>
> class Node(object):
> hasChildList = False
> hasAttributesDict = False
>
> @ConstProperty
> def children(self):
> self.hasChildList = True
> return []
>
> @ConstProperty
> def attributes(self):
> self.hasAttributesDict = True
> return {}


Hmm... Perhaps not such a bad idea after all !-)
 
Reply With Quote
 
Rich Harkins
Guest
Posts: n/a
 
      11-05-2007
Bruno Desthuilliers wrote:
> Which is easy to do with properties too.


True enough. It's the caching of the return value that's the value add
of course.

>
>> After
>> it is applied, then the penalties for function call of the property and
>> the computation are wiped out once the second access is requested.

>
> Agreed. But I wouldn't use such a scheme for mutable types - which are
> still the common case.
>


In many cases, yeah. Though I use a lot of immutable stuff in some of
my pet projects and such. ConstProperty is definitely not meant as a
replacement for property, only when something constant can be derived
from something else constant, especially when the derivation is expensive.

>> Now, in the original example, len() might be considered too little for
>> this use and should be just generated in the constructor "for free".
>> OTOH, that assumes that __len__ hasn't been overridden to do something
>> more complicated and time consuming. If the antecedent object is
>> static, and the derivative consequent is also static,

>
> You mean 'immutable', I assume...


Yeah, that's probably the better term.

[snip]

Again, I've used it quite a bit for various things and it's worked well
for the sort of thing the OP was requesting. Of course, your mileage
may vary.

Cheers!
Rich

PS: Sorry about the weird reposts. Thunderbird chaos.

 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
what happens to Popen()'s parent-side file descriptors? Roger Davis Python 12 10-16-2010 02:05 AM
prefix increment operator and side-effects subramanian100in@yahoo.com, India C++ 2 12-25-2009 12:10 PM
macros and side effects copx C Programming 19 05-05-2007 12:00 PM
volatile and "needed side effects" dingoatemydonut@aol.com C Programming 17 10-12-2005 04:34 PM



Advertisments