Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > Best practise implementation for equal by value objects

Reply
Thread Tools

Best practise implementation for equal by value objects

 
 
Slaunger
Guest
Posts: n/a
 
      08-06-2008
Hi,

I am new here and relatively new to Python, so be gentle:

Is there a recommended generic implementation of __repr__ for objects
equal by value to assure that eval(repr(x)) == x independet of which
module the call is made from?

Example:

class Age:

def __init__(self, an_age):
self.age = an_age

def __eq__(self, obj):
self.age == obj.age

def __repr__(self):
return self.__class__.__name__ + \
"(%r)" % self.age

age_ten = Age(10)
print repr(age_ten)
print eval(repr(age_ten))
print eval(repr(age_ten)).age

Running this gives

Age(10)
Age(10)
10

Exactly as I want to.

The problem arises when the Age class is iomported into another module
in another package as then there is a package prefix and the above
implementation of __repr__ does not work.

I have then experimented with doing somthing like

def __repr__(self):
return self.__module__ + '.' + self.__class__.__name__ +
"(%r)" % self.age

This seems to work when called from the outside, but not from the
inside of the module. That is, if I rerun the script above the the
module name prefixed to the representation I get the following error

Traceback (most recent call last):
File "valuetest.py", line 15, in <module>
print eval(repr(age_ten))
__main__.Age(10)
File "<string>", line 1, in <module>
NameError: name '__main__' is not defined

This is pretty annoying.

My question is: Is there a robust generic type of implementation of
__repr__ which I can use instead?

This is something I plan to reuse for many different Value classes, so
I would like to get it robust.

Thanks,
Slaunger
 
Reply With Quote
 
 
 
 
Terry Reedy
Guest
Posts: n/a
 
      08-06-2008


Slaunger wrote:
> Hi,
>
> I am new here and relatively new to Python, so be gentle:
>
> Is there a recommended generic implementation of __repr__ for objects
> equal by value to assure that eval(repr(x)) == x independet of which
> module the call is made from?


The CPython implementation gives up on that goal and simply prints
<modname.classname object at address> for at least two reasons .

1. In general, it require fairly sophisticated analysis of __init__ to
decide what representation of what attributes to include and decide if
the goal is even possible. If an attribute is an instance of a user
class, then *its* __init__ needs to be analyzed. If an attribute is a
module, class, or function, there is no generic evaluable representation.

2. Whether eval(repr(x)) even works (returns an answer) depends on
whether the name bindings in the globals and locals passed to eval
(which by default are the globals and locals of the context of the eval
call) match the names used in the repr. You discovered that to a first
approximation, this depends on whether the call to repr comes from
within or without the module containing the class definition. But the
situation is far worse. Consider 'import somemod as m'. Even if you
were able to introspect the call and determine that it did not come from
somemod**, prepending 'somemod.' to the repr *still* would not work.
Or, the call to repr could come from one context, the result saved and
passed to another context with different name bindings, and the eval
call made there. So an repr that can be eval'ed in any context is hopeless.

If this is a practical rather than theoretical question, then use your
first repr version that uses the classes definition name and only eval
the result in a context that has that name bound to the class object.

from mymod import Age
#or
import mymod
Age = mymod.Age

#in either case
eval(repr(Age(10))) == Age(10)

> class Age:
>
> def __init__(self, an_age):
> self.age = an_age
>
> def __eq__(self, obj):
> self.age == obj.age
>
> def __repr__(self):
> return self.__class__.__name__ + \
> "(%r)" % self.age

**
While such introspection is not part of the language, I believe one
could do it in CPython, but I forgot the details. There have been
threads like 'How do I determine the caller function' with answers to
that question, and I presume the module of the caller is available also.

Terry Jan Reedy

 
Reply With Quote
 
 
 
 
John Krukoff
Guest
Posts: n/a
 
      08-06-2008

On Wed, 2008-08-06 at 05:50 -0700, Slaunger wrote:
> Hi,
>
> I am new here and relatively new to Python, so be gentle:
>
> Is there a recommended generic implementation of __repr__ for objects
> equal by value to assure that eval(repr(x)) == x independet of which
> module the call is made from?
>
> Example:
>
> class Age:
>
> def __init__(self, an_age):
> self.age = an_age
>
> def __eq__(self, obj):
> self.age == obj.age
>
> def __repr__(self):
> return self.__class__.__name__ + \
> "(%r)" % self.age
>
> age_ten = Age(10)
> print repr(age_ten)
> print eval(repr(age_ten))
> print eval(repr(age_ten)).age
>
> Running this gives
>
> Age(10)
> Age(10)
> 10
>
> Exactly as I want to.
>
> The problem arises when the Age class is iomported into another module
> in another package as then there is a package prefix and the above
> implementation of __repr__ does not work.
>
> I have then experimented with doing somthing like
>
> def __repr__(self):
> return self.__module__ + '.' + self.__class__.__name__ +
> "(%r)" % self.age
>
> This seems to work when called from the outside, but not from the
> inside of the module. That is, if I rerun the script above the the
> module name prefixed to the representation I get the following error
>
> Traceback (most recent call last):
> File "valuetest.py", line 15, in <module>
> print eval(repr(age_ten))
> __main__.Age(10)
> File "<string>", line 1, in <module>
> NameError: name '__main__' is not defined
>
> This is pretty annoying.
>
> My question is: Is there a robust generic type of implementation of
> __repr__ which I can use instead?
>
> This is something I plan to reuse for many different Value classes, so
> I would like to get it robust.
>
> Thanks,
> Slaunger
> --
> http://mail.python.org/mailman/listinfo/python-list


Are you really sure this is what you want to do, and that a less tricky
serialization format such as that provided by the pickle module wouldn't
work for you?

--
John Krukoff <(E-Mail Removed)>
Land Title Guarantee Company

 
Reply With Quote
 
Steven D'Aprano
Guest
Posts: n/a
 
      08-07-2008
On Wed, 06 Aug 2008 05:50:35 -0700, Slaunger wrote:

> Hi,
>
> I am new here and relatively new to Python, so be gentle:
>
> Is there a recommended generic implementation of __repr__ for objects
> equal by value to assure that eval(repr(x)) == x independet of which
> module the call is made from?


In general, no.

....
> My question is: Is there a robust generic type of implementation of
> __repr__ which I can use instead?
>
> This is something I plan to reuse for many different Value classes, so I
> would like to get it robust.


I doubt you could get it that robust, nor is it intended to be.

eval(repr(obj)) giving obj is meant as a guideline, not an invariant --
there are many things that can break it. For example, here's a module
with a simple class:


# Parrot module
class Parrot(object):
def __repr__(self):
return "parrot.Parrot()"
def __eq__(self, other):
# all parrots are equal
return isinstance(other, Parrot)


Now let's use it:

>>> import parrot
>>> p = parrot.Parrot()
>>> s = repr(p)
>>> assert eval(s) == p
>>> del parrot
>>> assert eval(s) == p

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<string>", line 1, in <module>
NameError: name 'parrot' is not defined


If you look at classes in the standard library, they often have reprs
like this:

>>> repr(timeit.Timer())

'<timeit.Timer instance at 0xb7f14bcc>'

Certainly you can't expect to successfully eval that!

I believe the recommendation for eval(repr(obj)) to give obj again is
meant as a convenience for the interactive interpreter, and even there
only for simple types like int or list. If you can do it, great, but if
it doesn't work, so be it. You're not supposed to rely on it, and it's
not meant as a general way to serialize classes.


--
Steven
 
Reply With Quote
 
Slaunger
Guest
Posts: n/a
 
      08-07-2008
On 6 Aug., 21:36, Terry Reedy <(E-Mail Removed)> wrote:
> Slaunger wrote:
> > Hi,

>
> > I am new here and relatively new to Python, so be gentle:

>
> > Is there a recommended generic implementation of __repr__ for objects
> > equal by value to assure that eval(repr(x)) == x independet of which
> > module the call is made from?

>
> The CPython implementation gives up on that goal and simply prints
> <modname.classname object at address> for at least two reasons .
>
> 1. In general, it require fairly sophisticated analysis of __init__ to
> decide what representation of what attributes to include and decide if
> the goal is even possible. *If an attribute is an instance of a user
> class, then *its* __init__ needs to be analyzed. *If an attribute is a
> module, class, or function, there is no generic evaluable representation.


OK, the situation is more complicated than that then. In the case here
though,
the attributes would always be sinmple bulit-in types, where
eval(repr(x))==x
or, where the attribute is a user-defined equal-by-value class, that I
have
control over.

The classes I am making as struct type classes with some added
functionlity for
human readable string representation, packing into a stream or
unpacking from a stream
using a "private" class Struct.

I come from a Java and JUnit world, where, if I am used to always
overriding the default reference based implementations of the
equals(), toString(),
and hashCode() methods for "equals-by-value" objects such that they
work well
and efficient in, e.g., hash maps.

With my swich-over to Python, I looked for equivalent features and
stumbled over the
eval(repr(x))==x recommendation. It is not that I actually (yet) need
the repr implementations,
but mostly because I find the condition very useful in PyUnit to check
in a test that I have remembered
to initialize all instance fields in __init__ and that I have
remembered to include all relevant
attributes in the __eq__ implementation.

Whereas this worked fine in a unit test module dedicated to only test
the specific module, the test failed
when called from other test package modules, wrapping the unit tests
from several unit test modules.

>
> 2. Whether eval(repr(x)) even works (returns an answer) depends on
> whether the name bindings in the globals and locals passed to eval
> (which by default are the globals and locals of the context of the eval
> call) match the names used in the repr. *You discovered that to a first
> approximation, this depends on whether the call to repr comes from
> within or without the module containing the class definition. *But the
> situation is far worse. *Consider 'import somemod as m'. *Even if you
> were able to introspect the call and determine that it did not come from
> somemod**, prepending 'somemod.' to the repr *still* would not work.
> Or, the call to repr could come from one context, the result saved and
> passed to another context with different name bindings, and the eval
> call made there. *So an repr that can be eval'ed in any context is hopeless.
>

Ok, nasty stuff

> If this is a practical rather than theoretical question, then use your
> first repr version that uses the classes definition name and only eval
> the result in a context that has that name bound to the class object.
>
> from mymod import Age
> #or
> import mymod
> Age = mymod.Age
>
> #in either case
> eval(repr(Age(10))) == Age(10)
>
> > class Age:

>
> > * * def __init__(self, an_age):
> > * * * * self.age = an_age

>
> > * * def __eq__(self, obj):
> > * * * * self.age == obj.age

>
> > * * def __repr__(self):
> > * * * * return self.__class__.__name__ + \
> > * * * * * * * *"(%r)" % self.age

>

Yes, it is most from a practicl point of view, altough I was surprised
that I could not find more material on it in the Python documentation
or mailing groups, and I moight just do what you suggest in the unit
test modules to at least make it robust in that context.

Hmm... a bit of a dissapointment for me that this cannot be done
cleaner
> **
> While such introspection is not part of the language, I believe one
> could do it in CPython, but I forgot the details. *There have been
> threads like 'How do I determine the caller function' with answers to
> that question, and I presume the module of the caller is available also.

OK, I think CPython, for the moment, is too much new stuff to dig into
right now.
Just grasping some of all the possibilities in the API, and how to do
things the right way
is giving me enough challenges for now...

>
> Terry Jan Reedy


Again, thank you for your thorough answer,

Slaunger
 
Reply With Quote
 
Slaunger
Guest
Posts: n/a
 
      08-07-2008
On 6 Aug., 21:46, John Krukoff <(E-Mail Removed)> wrote:
> On Wed, 2008-08-06 at 05:50 -0700, Slaunger wrote:
> > Hi,

>
> > I am new here and relatively new to Python, so be gentle:

>
> > Is there a recommended generic implementation of __repr__ for objects
> > equal by value to assure that eval(repr(x)) == x independet of which
> > module the call is made from?

>
> > Example:

>
> > class Age:

>
> > * * def __init__(self, an_age):
> > * * * * self.age = an_age

>
> > * * def __eq__(self, obj):
> > * * * * self.age == obj.age

>
> > * * def __repr__(self):
> > * * * * return self.__class__.__name__ + \
> > * * * * * * * *"(%r)" % self.age

>
> > age_ten = Age(10)
> > print repr(age_ten)
> > print eval(repr(age_ten))
> > print eval(repr(age_ten)).age

>
> > Running this gives

>
> > Age(10)
> > Age(10)
> > 10

>
> > Exactly as I want to.

>
> > The problem arises when the Age class is iomported into another module
> > in another package as then there is a package prefix and the above
> > implementation of __repr__ does not work.

>
> > I have then experimented with doing somthing like

>
> > * * def __repr__(self):
> > * * * * return self.__module__ + '.' + self.__class__.__name__ +
> > "(%r)" % self.age

>
> > This seems to work when called from the outside, but not from the
> > inside of the module. That is, if I rerun the script above the the
> > module name prefixed to the representation I get the following error

>
> > Traceback (most recent call last):
> > * File "valuetest.py", line 15, in <module>
> > * * print eval(repr(age_ten))
> > __main__.Age(10)
> > * File "<string>", line 1, in <module>
> > NameError: name '__main__' is not defined

>
> > This is pretty annoying.

>
> > My question is: Is there a robust generic type of implementation of
> > __repr__ which I can use instead?

>
> > This is something I plan to reuse for many different Value classes, so
> > I would like to get it robust.

>
> > Thanks,
> > Slaunger
> > --
> >http://mail.python.org/mailman/listinfo/python-list

>
> Are you really sure this is what you want to do, and that a less tricky
> serialization format such as that provided by the pickle module wouldn't
> work for you?


Well, it is not so much yet for serialization (although i have not yet
fully understood the implications), it is more because
I think the eval(repr(x))==x is a nice unit test to make sure my
constructor and equals method is implemented correctly (that I have
rememebered all attributes in their implementations).

As mentioned above, I may go for a more pragmatic approach, where i
only use repr if it "standard" imported

Cheers,
Slaunger

>
> --
> John Krukoff <(E-Mail Removed)>
> Land Title Guarantee Company- Skjul tekst i anførselstegn -
>
> - Vis tekst i anførselstegn -


 
Reply With Quote
 
Slaunger
Guest
Posts: n/a
 
      08-07-2008
On 7 Aug., 04:34, Steven D'Aprano <st...@REMOVE-THIS-
cybersource.com.au> wrote:
> On Wed, 06 Aug 2008 05:50:35 -0700, Slaunger wrote:
> > Hi,

>
> > I am new here and relatively new to Python, so be gentle:

>
> > Is there a recommended generic implementation of __repr__ for objects
> > equal by value to assure that eval(repr(x)) == x independet of which
> > module the call is made from?

>
> In general, no.
>
> ...
>

OK.

> > My question is: Is there a robust generic type of implementation of
> > __repr__ which I can use instead?

>
> > This is something I plan to reuse for many different Value classes, so I
> > would like to get it robust.

>
> I doubt you could get it that robust, nor is it intended to be.
>
> eval(repr(obj)) giving obj is meant as a guideline, not an invariant --
> there are many things that can break it. For example, here's a module
> with a simple class:


OK, I had not fully understood the implications of 'not' implementing
__repr__
such that eval(repr(x)) == x, so I just tried to make it work to make
sure
life would be easy for me and my object as I went further into the
Python jungle

As mentioned above, i also find the eval(repr(x))==x condition
convenient from
a unit test point of view.

>
> # Parrot module
> class Parrot(object):
> * * def __repr__(self):
> * * * * return "parrot.Parrot()"
> * * def __eq__(self, other):
> * * * * # all parrots are equal
> * * * * return isinstance(other, Parrot)
>
> Now let's use it:
>
> >>> import parrot
> >>> p = parrot.Parrot()
> >>> s = repr(p)
> >>> assert eval(s) == p
> >>> del parrot
> >>> assert eval(s) == p

>
> Traceback (most recent call last):
> * File "<stdin>", line 1, in <module>
> * File "<string>", line 1, in <module>
> NameError: name 'parrot' is not defined
>


OK, I see, but this isn't exactly eval(repr(x))==x but
s = repr(x)
eval(s) == x

so, of course, is s is deleted in between it won't work.

In my implementation I only expect this should work as a one-liner.

> If you look at classes in the standard library, they often have reprs
> like this:
>
> >>> repr(timeit.Timer())

>
> '<timeit.Timer instance at 0xb7f14bcc>'
>


Yes, I noticed that. But the example here is also an object, which is
equal by reference, not value. And for these
it does not make so much sense to evaluate the representation.

> Certainly you can't expect to successfully eval that!
>
> I believe the recommendation for eval(repr(obj)) to give obj again is
> meant as a convenience for the interactive interpreter, and even there
> only for simple types like int or list. If you can do it, great, but if
> it doesn't work, so be it. You're not supposed to rely on it, and it's
> not meant as a general way to serialize classes.
>
> --
> Steven


OK, I will put less emphasis on it in the future.

Thank you for taking your time to answer.

Slaunger
 
Reply With Quote
 
Terry Reedy
Guest
Posts: n/a
 
      08-07-2008


Slaunger wrote:
> On 6 Aug., 21:36, Terry Reedy <(E-Mail Removed)> wrote:


> OK, the situation is more complicated than that then. In the case here
> though,
> the attributes would always be sinmple bulit-in types, where
> eval(repr(x))==x
> or, where the attribute is a user-defined equal-by-value class, that I
> have
> control over.


I think most would agree that a more accurate and informative
representation is better than a general representation like Pythons
default. For instance,
>>> a=range(2,10,2) # 3.0
>>> a

range(2, 10, 2)

is nicer than <class 'range' object at ######>.

So when the initializers for instances are all 'nice' (as for range), go
for it (as in 'Age(10)'). And test it as you are by eval'ing the rep.
Just accept that the eval will only work in contexts with the class name
bound to the class. For built-in like range, it always is, by default
-- unless masked by another assignment!

Terry Jan Reedy


 
Reply With Quote
 
Paul Rubin
Guest
Posts: n/a
 
      08-07-2008
Terry Reedy <(E-Mail Removed)> writes:
> So when the initializers for instances are all 'nice' (as for range),
> go for it (as in 'Age(10)'). And test it as you are by eval'ing the
> rep. Just accept that the eval will only work in contexts with the
> class name bound to the class. For built-in like range, it always is,
> by default -- unless masked by another assignment!


Eval is extremely dangerous. Think of data from untrusted sources,
then ask yourself how well you really know where ALL your data came
from. It's preferable to avoid using it that way. There have been a
few "safe eval" recipes posted here and at ASPN. It would be good if
one of them made it into the standard library. Note that pickle
(which would otherwise be an obious choice for this) has the same
problems, though not as severely as flat-out evalling something.
 
Reply With Quote
 
Slaunger
Guest
Posts: n/a
 
      08-08-2008
On 7 Aug., 21:25, Paul Rubin <http://(E-Mail Removed)> wrote:
> Terry Reedy <(E-Mail Removed)> writes:
> > So when the initializers for instances are all 'nice' (as for range),
> > go for it (as in 'Age(10)'). *And test it as you are by eval'ing the
> > rep. Just accept that the eval will only work in contexts with the
> > class name bound to the class. *For built-in like range, it always is,
> > by default -- unless masked by another assignment!

>
> Eval is extremely dangerous. *Think of data from untrusted sources,
> then ask yourself how well you really know where ALL your data came
> from. *It's preferable to avoid using it that way. *There have been a
> few "safe eval" recipes posted here and at ASPN. *It would be good if
> one of them made it into the standard library. *Note that pickle
> (which would otherwise be an obious choice for this) has the same
> problems, though not as severely as flat-out evalling something.


Thank you for pointing out the dangers of eval. I think you are right
to
caution about it. In my particular case it is a closed-loop system, so
no
danger there, but that certainly could have been an issue.

That caution should perhaps be mentioned in
http://docs.python.org/lib/built-in-funcs.html
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
setting getElementById().value equal to variable value ll Javascript 2 08-22-2008 03:54 PM
Internet Sharing: Equal upload speeds but un-equal download speeds =?Utf-8?B?TkpU?= Wireless Networking 3 09-15-2007 06:22 AM
Architecture best practise Rich ASP .Net 5 07-11-2006 02:51 PM
Best tool/methodology/practise for Java code protection ? exquisitus Java 0 02-20-2005 06:08 PM
spacing top & bottom - best practise VisionSet HTML 0 08-19-2003 11:01 AM



Advertisments