Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > Is __mul__ sufficient for operator '*'?

Reply
Thread Tools

Is __mul__ sufficient for operator '*'?

 
 
Muhammad Alkarouri
Guest
Posts: n/a
 
      10-20-2009
Hi everyone,

I was having a go at a simple implementation of Maybe in Python when I
stumbled on a case where x.__mul__(y) is defined while x*y is not.

The class defining x is:

class Maybe(object):
def __init__(self, obj):
self.o = obj
def __repr__(self):
return 'Maybe(%s)' % object.__getattribute__(self, "o")
def __getattribute__(self, name):
try:
o = object.__getattribute__(self, "o")
r = getattr(o,name)
if callable(r):
f = lambda *x:Maybe(r(*x))
return f
else:
return Maybe(r)
except:
return Maybe(None)

The code exercising this class is:

>>> x=Maybe(9)
>>> x.__mul__(7)

Maybe(63)
>>> x*7


Traceback (most recent call last):
File "<pyshell#83>", line 1, in <module>
x*7
TypeError: unsupported operand type(s) for *: 'Maybe' and 'int'

The farthest I can go in this is that I presume that __mul__ (as
called by operator *) is supposed to be a bound method while I am
returning a lambda function. Is this correct? And How can I make the
implementation support such operators?

Cheers,

Muhammad Alkarouri
 
Reply With Quote
 
 
 
 
Gary Herron
Guest
Posts: n/a
 
      10-20-2009
Muhammad Alkarouri wrote:
> Hi everyone,
>
> I was having a go at a simple implementation of Maybe in Python when I
> stumbled on a case where x.__mul__(y) is defined while x*y is not.
>
> The class defining x is:
>
> class Maybe(object):
> def __init__(self, obj):
> self.o = obj
> def __repr__(self):
> return 'Maybe(%s)' % object.__getattribute__(self, "o")
> def __getattribute__(self, name):
> try:
> o = object.__getattribute__(self, "o")
> r = getattr(o,name)
> if callable(r):
> f = lambda *x:Maybe(r(*x))
> return f
> else:
> return Maybe(r)
> except:
> return Maybe(None)
>
> The code exercising this class is:
>
>
>>>> x=Maybe(9)
>>>> x.__mul__(7)
>>>>

> Maybe(63)
>
>>>> x*7
>>>>

>
> Traceback (most recent call last):
> File "<pyshell#83>", line 1, in <module>
> x*7
> TypeError: unsupported operand type(s) for *: 'Maybe' and 'int'
>


The product 7*x will execute __mul__, but to catch x*7, you need to
define __rmul__ (the 'r' stands for reverse or some such).

However, in fact, you have *not* defined __mul__ in your class. Your
__getattribute__ is catching __mul__ as an undefined reference, and
doing something with it -- not sure what though.

As proof, continue testing: x.__mul__ and x.__rmul__ both return
values (lambdas defined within __getattribute__) and neither x*7 or 7*x
work.

If you want both x* and 7*x to be defined, try
def __mul__(self,r):
...
def __rmul__(self,r):
....

Or the operation is commutative, perhaps you can get away with reusing
__mul__ for both.
def __mul__(self,r):
...
__rmul__ = __mull__


Gary Herron


> The farthest I can go in this is that I presume that __mul__ (as
> called by operator *) is supposed to be a bound method while I am
> returning a lambda function. Is this correct? And How can I make the
> implementation support such operators?
>
> Cheers,
>
> Muhammad Alkarouri
>


 
Reply With Quote
 
 
 
 
Mick Krippendorf
Guest
Posts: n/a
 
      10-20-2009
Muhammad Alkarouri schrieb:
> Traceback (most recent call last):
> File "<pyshell#83>", line 1, in <module>
> x*7
> TypeError: unsupported operand type(s) for *: 'Maybe' and 'int'
>
> The farthest I can go in this is that I presume that __mul__ (as
> called by operator *) is supposed to be a bound method while I am
> returning a lambda function. Is this correct? And How can I make the
> implementation support such operators?


It does not so much depend on the function being bound, but on the fact
that Python expects Maybe.__mul__ to be present already when it tries to
execute '*'. You could always add this, though:

def __mul__(self, other):
return Maybe.__getattribute__(self, "__mul__")(other)

but of course you'd have to do it for every operator you want to use on
your Maybe objects. Your use of __getattribute__ OTH suggests you're
trying to avoid exactly that. I'd rather go for a monadic implementation
with unit, bind and lift operations, e.g:

----8<--------8<--------8<--------8<--------8<----

class Monad(object):
# unit:
def __init__(self, value):
self.value = value
# bind:
def __rshift__(self, function):
return function(self.__class__)(self.value)
def __str__(self):
return "%s(%s)" % (self.__class__.__name__, self.value)

def lift(f):
def lift_unit(m):
def lift_bind(x):
return m(f(x))
return lift_bind
return lift_unit

class idem(Monad):
def __call__(self, value):
return value

class Nothing(object):
def __rshift__(self, function):
return Nothing
def __str__(self):
return "Nothing"
Nothing = Nothing()

class Maybe(Monad):
def __new__(cls, value=Nothing):
if value is Nothing:
return Nothing
return super(Monad, cls).__new__(cls)

if __name__ == "__main__":

x = Maybe(9)
print x >> lift(lambda v: v * 7)
print x >> lift(lambda v: v * 7) >> idem

y = Maybe(Nothing)
print y >> lift(lambda v: v * 7)
print y >> lift(lambda v: v * 7) >> idem

----8<--------8<--------8<--------8<--------8<----

While I can see how this monadic stuff is usefull in Haskell et al.,
I'm still not sure how to apply it to Python. And for the impenetrable
mathematical language in which Monads are usually presented, well...


HTH,
Mick.
 
Reply With Quote
 
Gabriel Genellina
Guest
Posts: n/a
 
      10-20-2009
En Mon, 19 Oct 2009 21:31:44 -0300, Muhammad Alkarouri
<(E-Mail Removed)> escribió:

> I was having a go at a simple implementation of Maybe in Python when I
> stumbled on a case where x.__mul__(y) is defined while x*y is not.


__special__ methods are searched in the type, not in the instance
directly. x*y looks for type(x).__mul__ (among other things)


--
Gabriel Genellina

 
Reply With Quote
 
Mick Krippendorf
Guest
Posts: n/a
 
      10-20-2009
Gabriel Genellina schrieb:
> __special__ methods are searched in the type, not in the instance
> directly. x*y looks for type(x).__mul__ (among other things)


So I thought too, but:

class meta(type):
def __mul__(*args):
return 123

class boo(object):
__metaclass__ = meta

print boo.__mul__

b = boo()
print b * 7

also explodes. Or am I misinterpreting the word "type" here?


Mick.
 
Reply With Quote
 
Gabriel Genellina
Guest
Posts: n/a
 
      10-20-2009
En Tue, 20 Oct 2009 00:59:12 -0300, Mick Krippendorf <(E-Mail Removed)>
escribió:
> Gabriel Genellina schrieb:


>> __special__ methods are searched in the type, not in the instance
>> directly. x*y looks for type(x).__mul__ (among other things)

>
> So I thought too, but:
>
> class meta(type):
> def __mul__(*args):
> return 123
>
> class boo(object):
> __metaclass__ = meta
>
> print boo.__mul__
>
> b = boo()
> print b * 7
>
> also explodes. Or am I misinterpreting the word "type" here?


This is by design; see
http://docs.python.org/reference/dat...-style-classes

Methods defined on the meta-type aren't considered methods of the type;
otherwise, every object would have the methods defined in `type` itself,
because this is the metatype of every other object.

When I said "x*y looks for type(x).__mul__" the search for '__mul__' is
done in type(x) itself, and all its base types along the MRO - NOT on the
metatype, and not using getattr. There is no way to express this exact
search in Python code (that I know of), but it's more or less like
searching for '__mul__' in dir(type(x)).

In particular, __mul__ is stored in two slots (a slot is a field in a
structure holding function pointers: nb_multiply in the tp_as_number
structure, *and* sq_repeat in tp_as_sequence); defining or assigning to
__mul__ "magically" updates those internal pointers. When executing x*y,
__mul__ is retrieved directly from those pointers -- it is *not* searched
by name.

In short, you have to define the __mul__ method on the type itself or any
of its bases.

--
Gabriel Genellina

 
Reply With Quote
 
Mick Krippendorf
Guest
Posts: n/a
 
      10-20-2009
Gabriel Genellina schrieb:
> http://docs.python.org/reference/dat...-style-classes

Ok. That explains a lot. And your explanation tells the rest. Thank you.

> In short, you have to define the __mul__ method on the type itself or
> any of its bases.

I found this, where Raymond Hettinger shows how this could be changed:
http://mail.python.org/pipermail/pyt...er/031439.html

Mick.
 
Reply With Quote
 
Mick Krippendorf
Guest
Posts: n/a
 
      10-26-2009
Muhammad Alkarouri schrieb:
> I was having a go at a simple implementation of Maybe in Python when I
> stumbled on a case where x.__mul__(y) is defined while x*y is not.
>
> class Maybe(object):
> def __init__(self, obj):
> self.o = obj
> def __repr__(self):
> return 'Maybe(%s)' % object.__getattribute__(self, "o")
> def __getattribute__(self, name):
> try:
> o = object.__getattribute__(self, "o")
> r = getattr(o,name)
> if callable(r):
> f = lambda *x:Maybe(r(*x))
> return f
> else:
> return Maybe(r)
> except:
> return Maybe(None)
>
>>>> x=Maybe(9)
>>>> x.__mul__(7)

> Maybe(63)
>>>> x*7

>
> Traceback (most recent call last):
> File "<pyshell#83>", line 1, in <module>
> x*7
> TypeError: unsupported operand type(s) for *: 'Maybe' and 'int'


Here's how I'd do it. It will not win a beauty contest any time soon,
but at least it's a workaround:

----8<--------8<--------8<--------8<--------8<--------8<--------8<----

def meta(lift, lifted, not_lifted=[]):
not_lifted = list(not_lifted) + object.__dict__.keys()
class MetaMaybe(type):
def __new__(meta, mcls, bases, dct):
dct.update(
(name, lift(name))
for name in set(lifted) - set(not_lifted)
)
return type(mcls, bases, dct)
return MetaMaybe

class Nothing(object):
__metaclass__ = meta(lambda name: lambda self, *a, **k: self, (
"__add__", "__sub__", "__mul__", "__div__", "__truediv__",
"__floordiv__", "__divmod__", "__radd__", "__rsub__",
"__rmul__", "__rdiv__", "__rtruediv__", "__rfloordiv__",
"__rdivmod__", "__rshift__", "__lshift__", "__call__",
# and so on, for every special method that Nothing knows
))
def __new__(cls, value=None):
try: # singleton
return cls.value
except AttributeError:
cls.value = super(Nothing, cls).__new__(cls)
return cls.value
def __str__(self):
return "Nothing"
__repr__ = __str__

Nothing = Nothing()

def just(vcls):
def lifter(name):
attr = getattr(vcls, name)
def lifted(self, *ms):
try:
return self.lift(attr)(self, *ms)
except:
return Nothing
return lifted
class Just(object):
__metaclass__ = meta(lifter, vcls.__dict__.keys())
def __new__(cls, value):
if value in (Nothing, NotImplemented):
return Nothing
return super(Just, cls).__new__(cls)
def __init__(self, value):
self.value = value
def __str__(self):
return "Just(%s)" % self.value
@classmethod
def lift(c, f):
return lambda *ms:c(f(*(m.value for m in ms)))
return Just

from collections import defaultdict

class TypeDict(defaultdict):
def __missing__(self, key):
if self.default_factory is None:
raise KeyError(key)
return self.default_factory(key)

class Maybe(object):
typemap = TypeDict(just)
def __new__(cls, value):
return Maybe.typemap[value.__class__](value)

def foo(x, y):
return x * 2 + y * 3

if __name__ == "__main__":

print Maybe(Nothing)
print Maybe(1) / Maybe(0)
print Maybe(10.) * Maybe(5) / Maybe(2) ** Maybe(3)
print Maybe(foo)(Maybe(6), Maybe(10))
print Maybe("hello").upper()
print Maybe("hello").startswith(Maybe("h"))
print getattr(Maybe("hello"), "startswith")(Maybe("h"))
print Maybe(foo)(Maybe("hello! "), Maybe("what? "))
print Maybe(foo)(Maybe("hello! "), Nothing)

----8<--------8<--------8<--------8<--------8<--------8<--------8<----

I haven't tested it very thoroughly, so it's quite possible there are
lots of bugs in it, but it is only intended as a demo.

As Gabriel Genellina pointed out, the search for special methods is done
in the type, so we have to put our own versions there, during type
creation in the metaclass' __new__ method. The above code does this for
all methods of the wrapped object's type, not just special ones, and
"lifts" them to expect Maybe objects instead of "normal" objects. They
also wrap their return values into Maybe objects.

Maybe is an algebraic data type. The call Maybe(some_value) returns
either Nothing, if some_value happens to be Nothing, or else an object
of type Just that wraps some_value. More precisely, there is not one
type Just, but as many as types of Just-wrapped objects (Just<int>,
Just<string>, ...). Just is therefore a kind of parameterized type. It's
similar to inheriting from a C++ template parameter type:

template <class T>
class MyType : public T {...}

but not quite, since in C++ the inherited member functions' signatures
are unchanged, whereas in the above code "inherited" methods are changed
to expect and return Maybe objects.

Some things don't work, though, e.g. slicing. But this could be
implemented via specialized lifting functions for the __XXXitem__
methods. One thing that does work though, is that ordinary functions can
be wrapped as Maybe objects which then "do the same thing" on other
Maybe objects that the normal functions do on normal objects, like in
the foo-example. So it's quite close to a monadic version, in that it
"lifts" objects and functions from one type space into another one, the
Maybe space. But compared to a real Monads it's much more pythonic, IMO.


HTH,
Mick.
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
__mul__ vs __rmul__ (and others) priority for different classes dmitrey Python 1 12-11-2009 08:24 PM
Using '__mul__' within a class Gerard Flanagan Python 12 09-25-2005 02:28 PM
Using __mul__ Thomas Philips Python 2 07-03-2004 03:28 PM
newbie - PIX 501 sufficient Kevin Laro Cisco 6 05-25-2004 05:47 PM
system lacked sufficient buffer space ali Java 0 08-18-2003 07:58 AM



Advertisments