Velocity Reviews

Velocity Reviews (http://www.velocityreviews.com/forums/index.php)
-   Python (http://www.velocityreviews.com/forums/f43-python.html)
-   -   Debugging difficulty in python with __getattr__, decorated propertiesand AttributeError. (http://www.velocityreviews.com/forums/t960314-debugging-difficulty-in-python-with-__getattr__-decorated-propertiesand-attributeerror.html)

Mr. Joe 05-02-2013 11:34 PM

Debugging difficulty in python with __getattr__, decorated propertiesand AttributeError.
 
Is there any way to raise the original exception that made the call to
__getattr__? I seem to stumble upon a problem where multi-layered attribute
failure gets obscured due to use of __getattr__. Here's a dummy code to
demonstrate my problems:
"""
import traceback


class BackupAlphabet(object):
pass


class Alphabet(object):
@property
def a(self):
return backupalphabet.a


def __getattr__(self, name):
if name == "b":
return "banana"

raise AttributeError(
"'{0} object has no attribute '{1}'"
.format(self.__class__.__name__, name))


alphabet = Alphabet()
backupalphabet = BackupAlphabet()

print(alphabet.a)
print(alphabet.b)
"""

Running the above code produces this:
"""
Traceback (most recent call last):
File "example.py", line 26, in <module>
print(alphabet.a)
File "example.py", line 20, in __getattr__
.format(self.__class__.__name__, name))
AttributeError: 'Alphabet object has no attribute 'a'
"""

While it's easy enough to identify the problem here, the traceback is
rather unhelpful in complex situations. Any comments?

Regards,
TB


Steven D'Aprano 05-03-2013 12:20 AM

Re: Debugging difficulty in python with __getattr__, decoratedpropertiesand AttributeError.
 
On Fri, 03 May 2013 05:34:40 +0600, Mr. Joe wrote:

> Is there any way to raise the original exception that made the call to
> __getattr__?


No. There is some discussion on the Python-Dev mailing list about adding
better error reporting to AttributeError, but that may not go anywhere,
and even if it does, it won't help you until you have dropped all support
for versions below Python 3.4 or 3.5.



> class BackupAlphabet(object):
> pass
>
>
> class Alphabet(object):
> @property
> def a(self):
> return backupalphabet.a


This raises AttributeError. Since __getattr__ uses AttributeError to
decide whether or not the name exists, the behaviour shown is correct:
Alphabet.a does not exist, as far as the Python semantics of attribute
access are concerned.

Either write better code *wink* that doesn't raise AttributeError from
inside properties, or wrap them with something like this:

class Alphabet(object):
@property
def a(self):
try:
return backupalphabet.a
except AttributeError:
raise MyCustomError

where MyCustomError does NOT inherit from AttributeError.

If you're doing this a lot, you can create a decorator to do the wrapping.



--
Steven

Mr. Joe 05-03-2013 07:52 AM

Re: Debugging difficulty in python with __getattr__, decoratedproperties and AttributeError.
 
Thanks for clearing up. Developers of python should address this issue, in
my opinion. 3.4/3.5 maybe, but better late than never.

Recently, I've been beaten back for using some exotic features of python.
One is this[ Took me hours to get to the bottom ]. The other one is
'property' decorator. I was using it extensively until I needed to make a
child class. Then I came to know that 'property' does not play well
with polymorphic code. :( I resorted to some lambda hacks learned from
stackoverflow.com to solve the problem. I know that it's the correct way
for decorators to work, but still, it would be nice to have a language
level solution.


dieter 05-04-2013 05:56 AM

Re: Debugging difficulty in python with __getattr__,decorated properties and AttributeError.
 
"Mr. Joe" <titanix88@gmail.com> writes:

> ...
> Then I came to know that 'property' does not play well
> with polymorphic code. :(


Can you elaborate?

I like "polymorphic code" and decorators (such a "property")
never met a problem with the two working nicely together.

> I resorted to some lambda hacks learned from
> stackoverflow.com to solve the problem. I know that it's the correct way
> for decorators to work, but still, it would be nice to have a language
> level solution.


There are two approaches: change the language or learn how the language works.

I am very happy that the Python developers try hard to retain backward
compatible and, consequently, are very reluctant towards language changes.
I would hate should the decorator behavior change in an incompatible
way.

On the other hand, decorators are very easy to understand: they are
just syntactic sugar:

@<dec_expression>
def f(...): ...

is equivalent to:

def f(...): ...
f = <dec_expression>(f)

The "property" decorator uses an additional concept: "descriptor"s.
A "descriptor" allows you to customize attribute access.

These two concepts graped, the "property" decorator should no longer
cause surprises -- even in "polymorphic code".


Steven D'Aprano 05-04-2013 06:29 AM

Re: Debugging difficulty in python with __getattr__, decoratedproperties and AttributeError.
 
On Fri, 03 May 2013 13:52:23 +0600, Mr. Joe wrote:

> Thanks for clearing up. Developers of python should address this issue,
> in my opinion. 3.4/3.5 maybe, but better late than never.
>
> Recently, I've been beaten back for using some exotic features of
> python.


What do you consider "exotic"? Neither properties nor __getattr__ are
exotic, although of course like all features of a language they have
their own quirks.


> One is this[ Took me hours to get to the bottom ].


Well, of course hindsight is 20:20, and I don't know how complicated your
actual code is, but "hours" seems a bit poor. I could believe half an
hour. Maybe an hour. But of course everything is easy to the guy who
doesn't have to do it.


> The other one
> is 'property' decorator. I was using it extensively until I needed to
> make a child class. Then I came to know that 'property' does not play
> well with polymorphic code. :(


I don't understand what you are trying to say. Properties can be as
polymorphic as any other Python function.


> I resorted to some lambda hacks learned
> from stackoverflow.com to solve the problem.


That's not a good sign. Any sentence containing "hacks" and
"stackoverflow" is a warning that you're probably doing something wrong.



--
Steven

Mr. Joe 05-14-2013 03:09 PM

Re: Debugging difficulty in python with __getattr__, decoratedproperties and AttributeError.
 
Sorry for digging this old topic back. I see that my "'property' does not
play well with polymorphic code" comment generated some controversy. So
here's something in my defense:

Here's the link to stackoveflow topic I am talking about:

http://stackoverflow.com/questions/2...nd-inheritance

The solution that fits my taste:
http://stackoverflow.com/a/14349742

A related blogpost:

http://requires-thinking.blogspot.co...s-are-non.html

Yes, I like decorators and descriptors. I also like the ability to create a
"virtual property" in a python class by binding a bunch of methods as
setter/getter. But I find the implementation of this "virtual property
feature" a bit awkward sometimes - every time I need to override a
getter/setter in a child class, I need to decorate them again. Some of you
may like this "explicitness", but I don't.

To Steven D'Aprano: Seriously, what's all the bashing in your last reply
about? You dissected my "thank-you reply" more strictly than the python
interpreter checking for syntax errors. Not in a mood for fight, but I find
your opinions about "bug finding time", "hacks" and "stackoverflow" quite
silly.


dieter 05-15-2013 06:15 AM

Re: Debugging difficulty in python with __getattr__,decorated properties and AttributeError.
 
"Mr. Joe" <titanix88@gmail.com> writes:
> ...
> Sorry for digging this old topic back. I see that my "'property' does not
> play well with polymorphic code" comment generated some controversy. So
> here's something in my defense:


I did not intend to "attack" you.

> ...
> Yes, I like decorators and descriptors. I also like the ability to create a
> "virtual property" in a python class by binding a bunch of methods as
> setter/getter. But I find the implementation of this "virtual property
> feature" a bit awkward sometimes - every time I need to override a
> getter/setter in a child class, I need to decorate them again. Some of you
> may like this "explicitness", but I don't.


True, I have been hit by this, too - not with "property" but with
other decorators (those for caching).
But, after reflection, I came to the conclusion that I should be
happy with this feature:

If Python would automatically redecorate overridden methods in a derived
class, I would have no control over the process. What if I need
the undecorated method or a differently decorated method (an
uncached or differently cached method, in my case)?

Your getter/setter use case can quite easily be solved
with a class "DelayedMethodAccessor":

class DelayedMethodAccess(object):
"""
def __init__(self, name): self.__name = name
def __call__(self, inst, *args, **kw):
return getattr(inst, self.__name)(*args, **kw)

You can then use:

prop = property(DelayedMethodAccess("<getter_name>"), ...)

Or define a new decorator "property_by_name" and then
use

prop = property_by_name("<getter_name>", ...)

Of course, this requires that the property name and the getter/setter names
differ, but this should be naturally enough.


Python's current behavior is very natural once you know that
decorators are just syntactic sugar and

@<decorator_expression>
def <f>...

simply means:

def <f>...
f = <decorator_expressen>(f)

True, this is no perfect fit for all use cases - but
such a fit (if it existed at all) would have to be so complex
that only experts could understand it.


Mr. Joe 05-15-2013 04:38 PM

Re: Debugging difficulty in python with __getattr__, decoratedproperties and AttributeError.
 
On Wed, May 15, 2013 at 12:15 PM, dieter <dieter@handshake.de> wrote:
>
> If Python would automatically redecorate overridden methods in a derived
> class, I would have no control over the process. What if I need
> the undecorated method or a differently decorated method (an
> uncached or differently cached method, in my case)?
>


On a second thought, I am convinced by your argument.

> Your getter/setter use case can quite easily be solved
> with a class "DelayedMethodAccessor":
>
> class DelayedMethodAccess(object):
> """
> def __init__(self, name): self.__name = name
> def __call__(self, inst, *args, **kw):
> return getattr(inst, self.__name)(*args, **kw)
>
> You can then use:
>
> prop = property(DelayedMethodAccess("<getter_name>"), ...)
>
> Or define a new decorator "property_by_name" and then
> use
>
> prop = property_by_name("<getter_name>", ...)
>
> Of course, this requires that the property name and the getter/setter

names
> differ, but this should be naturally enough.
>
>
> Python's current behavior is very natural once you know that
> decorators are just syntactic sugar and
>
> @<decorator_expression>
> def <f>...
>
> simply means:
>
> def <f>...
> f = <decorator_expressen>(f)
>
> True, this is no perfect fit for all use cases - but
> such a fit (if it existed at all) would have to be so complex
> that only experts could understand it.


Thanks for these really nice patterns. They fits my problem very well.



All times are GMT. The time now is 02:20 AM.

Powered by vBulletin®. Copyright ©2000 - 2014, vBulletin Solutions, Inc.
SEO by vBSEO ©2010, Crawlability, Inc.