Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > __eq__() inconvenience when subclassing set

Reply
Thread Tools

__eq__() inconvenience when subclassing set

 
 
Jess Austin
Guest
Posts: n/a
 
      10-29-2009
I'm subclassing set, and redefining __eq__(). I'd appreciate any
relevant advice.

>>> class mySet(set):

.... def __eq__(self, other):
.... print "called mySet.__eq__()!"
.... if isinstance(other, (set, frozenset)):
.... return True
.... return set.__eq__(self, other)
....

I stipulate that this is a weird thing to do, but this is a toy class
to avoid the lengthy definition of the class I actually want to
write. Now I want the builtin set and frozenset types to use the new
__eq__() with mySet symmetrically.

>>> mySet() == set([1])

called mySet.__eq__()!
True
>>> mySet() == frozenset([1])

called mySet.__eq__()!
True
>>> set([1]) == mySet()

called mySet.__eq__()!
True
>>> frozenset([1]) == mySet()

False

frozenset doesn't use mySet.__eq__() because mySet is not a subclass
of frozenset as it is for set. I've tried a number of techniques to
mitigate this issue. If I multiple-inherit from both set and
frozenset, I get the instance lay-out conflict error. I have similar
problems setting mySet.__bases__ directly, and hacking mro() in a
metaclass. So far nothing has worked. If it matters, I'm using 2.6,
but I can change versions if it will help.

Should I give up on this, or is there something else I can try? Keep
in mind, I must redefine __eq__(), and I'd like to be able to compare
instances of the class to both set and frozenset instances.

cheers,
Jess
 
Reply With Quote
 
 
 
 
Mick Krippendorf
Guest
Posts: n/a
 
      10-29-2009
Jess Austin schrieb:
> >>> frozenset([1]) == mySet()

> False
>
> frozenset doesn't use mySet.__eq__() because mySet is not a subclass
> of frozenset as it is for set.


You could just overwrite set and frozenset:

class eqmixin(object):
def __eq__(self, other):
print "called %s.__eq__()" % self.__class__
if isinstance(other, (set, frozenset)):
return True
return super(eqmixin, self).__eq__(other)

class set(eqmixin, set):
pass
class frozenset(eqmixin, frozenset):
pass
class MySet(set):
pass


Regards,
Mick.
 
Reply With Quote
 
 
 
 
Jess Austin
Guest
Posts: n/a
 
      10-29-2009
On Oct 28, 10:07*pm, Mick Krippendorf <(E-Mail Removed)> wrote:
> You could just overwrite set and frozenset:
>
> class eqmixin(object):
> * * def __eq__(self, other):
> * * * * print "called %s.__eq__()" % self.__class__
> * * * * if isinstance(other, (set, frozenset)):
> * * * * * * return True
> * * * * return super(eqmixin, self).__eq__(other)
>
> class frozenset(eqmixin, frozenset):
> * * pass


That's nice, but it means that everyone who imports my class will have
to import the monkeypatch of frozenset, as well. I'm not sure I want
that. More ruby than python, ne?

thanks,
Jess
 
Reply With Quote
 
Mick Krippendorf
Guest
Posts: n/a
 
      10-29-2009
Jess Austin wrote:
> That's nice, but it means that everyone who imports my class will have
> to import the monkeypatch of frozenset, as well. I'm not sure I want
> that. More ruby than python, ne?


I thought it was only a toy class?

Mick.
 
Reply With Quote
 
Jess Austin
Guest
Posts: n/a
 
      10-29-2009
On Oct 29, 3:54*pm, Mick Krippendorf <(E-Mail Removed)> wrote:
> Jess Austin wrote:
> > That's nice, but it means that everyone who imports my class will have
> > to import the monkeypatch of frozenset, as well. *I'm not sure I want
> > that. *More ruby than python, ne?

>
> I thought it was only a toy class?


Well, I posted a toy, but it's a stand-in for something else more
complicated. Trying to conserve bytes, you know.
 
Reply With Quote
 
Gabriel Genellina
Guest
Posts: n/a
 
      10-30-2009
En Wed, 28 Oct 2009 23:12:53 -0300, Jess Austin <(E-Mail Removed)>
escribió:

>>>> class mySet(set):

> ... def __eq__(self, other):
> ... print "called mySet.__eq__()!"
> ... if isinstance(other, (set, frozenset)):
> ... return True
> ... return set.__eq__(self, other)
> ...
>
> Now I want the builtin set and frozenset types to use the new
> __eq__() with mySet symmetrically.
>
>>>> mySet() == set([1])

> called mySet.__eq__()!
> True
>>>> mySet() == frozenset([1])

> called mySet.__eq__()!
> True
>>>> set([1]) == mySet()

> called mySet.__eq__()!
> True
>>>> frozenset([1]) == mySet()

> False
>
> frozenset doesn't use mySet.__eq__() because mySet is not a subclass
> of frozenset as it is for set. [...failed attempts to inherit from both
> set and frozenset...]
> I must redefine __eq__(), and I'd like to be able to compare
> instances of the class to both set and frozenset instances.


We know the last test fails because the == logic fails to recognize mySet
(on the right side) as a "more specialized" object than frozenset (on the
left side), because set and frozenset don't have a common base type
(although they share a lot of implementation)

I think the only way would require modifying tp_richcompare of
set/frozenset objects, so it is aware of subclasses on the right side.
Currently, frozenset() == mySet() effectively ignores the fact that mySet
is a subclass of set.

--
Gabriel Genellina

 
Reply With Quote
 
Jess Austin
Guest
Posts: n/a
 
      10-30-2009
On Oct 29, 10:41*pm, "Gabriel Genellina" <(E-Mail Removed)>
wrote:
> We know the last test fails because the == logic fails to recognize mySet *
> (on the right side) as a "more specialized" object than frozenset (on the *
> left side), because set and frozenset don't have a common base type *
> (although they share a lot of implementation)
>
> I think the only way would require modifying tp_richcompare of *
> set/frozenset objects, so it is aware of subclasses on the right side. *
> Currently, frozenset() == mySet() effectively ignores the fact that mySet *
> is a subclass of set.


I don't think even that would work. By the time set_richcompare() is
called (incidentally, it's used for both set and frozenset), it's too
late. That function is not responsible for calling the subclass's
method. It does call PyAnySet_Check(), but only to short-circuit
equality and inequality for non-set objects. I believe that something
higher-level in the interpreter decides to call the right-side type's
method because it's a subclass of the left-side type, but I'm not
familiar enough with the code to know where that happens. It may be
best not to sully such generalized code with a special case for
this.

I may do some experiments with bytes, str, and unicode, since that
seems to be an analogous case. There is a basestring type, but at
this point I don't know that it really helps with anything.

cheers,
Jess
 
Reply With Quote
 
Gabriel Genellina
Guest
Posts: n/a
 
      11-01-2009
En Fri, 30 Oct 2009 17:55:27 -0300, Jess Austin <(E-Mail Removed)>
escribió:

> On Oct 29, 10:41 pm, "Gabriel Genellina" <(E-Mail Removed)>
> wrote:
>> We know the last test fails because the == logic fails to recognize
>> mySet (on the right side) as a "more specialized" object than frozenset
>> (on the left side), because set and frozenset don't have a common base
>> type (although they share a lot of implementation)
>>
>> I think the only way would require modifying tp_richcompare of
>> set/frozenset objects, so it is aware of subclasses on the right side.
>> Currently, frozenset() == mySet() effectively ignores the fact that
>> mySet is a subclass of set.

>
> I don't think even that would work. By the time set_richcompare() is
> called (incidentally, it's used for both set and frozenset), it's too
> late. That function is not responsible for calling the subclass's
> method. It does call PyAnySet_Check(), but only to short-circuit
> equality and inequality for non-set objects. I believe that something
> higher-level in the interpreter decides to call the right-side type's
> method because it's a subclass of the left-side type, but I'm not
> familiar enough with the code to know where that happens. It may be
> best not to sully such generalized code with a special case for
> this.
>
> I may do some experiments with bytes, str, and unicode, since that
> seems to be an analogous case. There is a basestring type, but at
> this point I don't know that it really helps with anything.


Looks like in 3.1 this can be done with bytes+str and viceversa, even if
bytes and str don't have a common ancestor (other than object; basestring
doesn't exist in 3.x):

p3> Base = bytes
p3> Other = str
p3>
p3> class Derived(Base):
.... def __eq__(self, other):
.... print('Derived.__eq__')
.... return True
....
p3> Derived()==Base()
Derived.__eq__
True
p3> Base()==Derived()
Derived.__eq__
True
p3> Derived()==Other()
Derived.__eq__
True
p3> Other()==Derived()
Derived.__eq__ # !!!
True
p3> Base.mro()
[<class 'bytes'>, <class 'object'>]
p3> Other.mro()
[<class 'str'>, <class 'object'>]

The same example with set+frozenset (the one you're actually interested
in) doesn't work, unfortunately.
After further analysis, this works for bytes and str because both types
refuse to guess and compare to each other; they return NotImplemented when
the right-side operand is not of the same type. And this gives that other
operand the chance of being called.

set and frozenset, on the other hand, are promiscuous: their
tp_richcompare slot happily accepts any set of any kind, derived or not,
and compares their contents. I think it should be a bit more strict: if
the right hand side is not of the same type, and its tp_richcompare slot
is not the default one, it should return NotImplemented. This way the
other type has a chance to be called.

--
Gabriel Genellina

 
Reply With Quote
 
Jess Austin
Guest
Posts: n/a
 
      11-03-2009
On Nov 1, 1:13*am, "Gabriel Genellina" <(E-Mail Removed)> wrote:
> Looks like in 3.1 this can be done with bytes+str and viceversa, even if *
> bytes and str don't have a common ancestor (other than object; basestring *
> doesn't exist in 3.x):
>
> p3> Base = bytes
> p3> Other = str
> p3>
> p3> class Derived(Base):
> ... * def __eq__(self, other):
> ... * * print('Derived.__eq__')
> ... * * return True
> ...
> p3> Derived()==Base()
> Derived.__eq__
> True
> p3> Base()==Derived()
> Derived.__eq__
> True
> p3> Derived()==Other()
> Derived.__eq__
> True
> p3> Other()==Derived()
> Derived.__eq__ * * * * * *# !!!
> True
> p3> Base.mro()
> [<class 'bytes'>, <class 'object'>]
> p3> Other.mro()
> [<class 'str'>, <class 'object'>]
>
> The same example with set+frozenset (the one you're actually interested *
> in) doesn't work, unfortunately.
> After further analysis, this works for bytes and str because both types *
> refuse to guess and compare to each other; they return NotImplemented when *
> the right-side operand is not of the same type. And this gives that other *
> operand the chance of being called.
>
> set and frozenset, on the other hand, are promiscuous: their *
> tp_richcompare slot happily accepts any set of any kind, derived or not, *
> and compares their contents. I think it should be a bit more strict: if *
> the right hand side is not of the same type, and its tp_richcompare slot *
> is not the default one, it should return NotImplemented. This way the *
> other type has a chance to be called.


Thanks for this, Gabriel! There seems to be a difference between the
two cases, however:

>>> str() == bytes()

False
>>> set() == frozenset()

True

I doubt that either of these invariants is amenable to modification,
even for purposes of "consistency". I'm not sure how to resolve this,
but you've definitely helped me here. Perhaps the test in
set_richcompare can return NotImplemented in particular cases but not
in others? I'll think about this; let me know if you come up with
anything more.

thanks,
Jess
 
Reply With Quote
 
Gabriel Genellina
Guest
Posts: n/a
 
      11-03-2009
En Mon, 02 Nov 2009 22:05:42 -0300, Jess Austin <(E-Mail Removed)>
escribió:
> On Nov 1, 1:13 am, "Gabriel Genellina" <(E-Mail Removed)> wrote:


>> Looks like in 3.1 this can be done with bytes+str and viceversa, even
>> if bytes and str don't have a common ancestor (other than object;
>> basestring doesn't exist in 3.x):
>> The same example with set+frozenset (the one you're actually interested
>> in) doesn't work, unfortunately.


> Thanks for this, Gabriel! There seems to be a difference between the
> two cases, however:
>
>>>> str() == bytes()

> False
>>>> set() == frozenset()

> True
>
> I doubt that either of these invariants is amenable to modification,
> even for purposes of "consistency". I'm not sure how to resolve this,
> but you've definitely helped me here. Perhaps the test in
> set_richcompare can return NotImplemented in particular cases but not
> in others? I'll think about this; let me know if you come up with
> anything more.


I think it should return NotImplemented only when the right-hand side
operand has overriden tp_richcompare. That way, set()==frozenset() would
still be True. Only when one inherits from set/frozenset AND overrides
__eq__, set_richcompare should step aside and let the more specific __eq__
be called (by just returning NotImplemented).

What is your goal when overriding __eq__ for your new set class? It may
help building a case for this change; a concrete use case is much better
than an abstract request.

--
Gabriel Genellina

 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Eclipse editor autoscroll inconvenience Warren Tang Java 0 10-24-2011 03:23 PM
inconvenience latch VenturaPOR VHDL 1 07-02-2008 06:02 PM
Windows inconvenience with ".avi" file heskum Software 1 06-10-2008 02:04 AM
Inconvenience due to abstract class does not have a constructor www Java 2 10-22-2007 04:12 PM
Subclassing UserControl makes the Visual Studio designer does not work S Guiboud ASP .Net 1 07-18-2003 03:55 PM



Advertisments