Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > what is lambda used for in real code?

Reply
Thread Tools

what is lambda used for in real code?

 
 
Steven Bethard
Guest
Posts: n/a
 
      12-31-2004
I thought it might be useful to put the recent lambda threads into
perspective a bit. I was wondering what lambda gets used for in "real"
code, so I grepped my Python Lib directory. Here are some of the ones I
looked, classified by how I would rewrite them (if I could):


* Rewritable as def statements (<name> = lambda <args>: <expr> usage)
These are lambdas used when a lambda wasn't needed -- an anonymous
function was created with lambda and then immediately bound to a name.
Since this is essentially what def does, using lambdas here is (IMHO) silly.

pickletools.py: getpos = lambda: None
def getpos(): return None
tarfile.py: normpath = lambda path:
os.path.normpath(path).replace(os.sep, "/")
def normpath(path): os.path.normpath(path).replace(os.sep, "/")
urllib2.py: H = lambda x: md5.new(x).hexdigest()
def H(x): md5.new(x).hexdigest()
urllib2.py: H = lambda x: sha.new(x).hexdigest()
def H(x): sha.new(x).hexdigest()


* Rewritable with existing functions
Mainly these are examples of code that can benefit from using the
functions available in the operator module, especially
operator.itemgetter and operator.attrgetter (available in 2.4)

cgi.py: return map(lambda v: v.value, value)
return map(operator.attrgetter('value'), value)
CGIHTTPServer.py: nobody = 1 + max(map(lambda x: x[2], pwd.getpwall()))
nobody = 1 + max(map(operator.itemgetter(2), pwd.getpwall()))
SimpleXMLRPCServer.py: server.register_function(lambda x,y: x+y, 'add')
server.register_function(operator.add, 'add')
SimpleXMLRPCServer.py: server.register_function(lambda x,y: x+y, 'add')
server.register_function(operator.add, 'add')
sre_constants.py: items.sort(key=lambda a: a[1])
items.sort(key=operator.itemgetter(1))
tarfile.py: return map(lambda m: m.name, self.infolist())
return map(operator.attrgetter('name'), self.infolist())


* Rewritable with list comprehensions/generator expressions
Lambdas in map or filter expressions can often be replaced by an
appropriate list comprehension or generator expression (in Python 2.3/2.4)

cgi.py: plist = map(lambda x: x.strip(), line.split(';'))
plist = [x.strip() for x in line.split(';')
cgi.py: return map(lambda v: v.value, value)
return [v.value for v in value]
CGIHTTPServer.py: nobody = 1 + max(map(lambda x: x[2], pwd.getpwall()))
nobody = 1 + max(x[2] for x in pwd.getpwall())
glob.py: names=filter(lambda x: x[0]!='.',names)
names=[x for x in names if x[0] != '.']
hmac.py: return "".join(map(lambda x, y: chr(ord(x) ^ ord(y)),
s1, s2))
return "".join(chr(ord(x) ^ ord(y)) for x, y in zip(s1, s2))
imaplib.py: l = map(lambda x:'%s: "%s"' % (x[0], x[1][0] and
'" "'.join(x[1]) or ''), l)
l = ['%s: "%s"' % (x[0], x[1][0] and '" "'.join(x[1]) or '')
for x in l]
inspect.py: suffixes = map(lambda (suffix, mode, mtype):
(-len(suffix), suffix, mode, mtype),
imp.get_suffixes())
suffixes = [(-len(suffix), suffix, mode, mtype)
for suffix, mode, mtype in imp.get_suffixes()
inspect.py: return join(map(lambda o, c=convert, j=join:
strseq(o, c, j), object))
return join([strseq(o, convert, join) for o in object])
mailcap.py: entries = filter(lambda e,key=key: key in e, entries)
entries = [e for e in entries if key in e]
poplib.py: digest = ''.join(map(lambda x:'%02x'%ord(x), digest))
digest = ''.join('%02x' % ord(x) for x in digest)
pstats.py: if line and not filter(lambda x,a=abbrevs:
x not in a,line.split()):
if line and not [x for x in line.split() if x not in abbrevs]:
tabnanny.py: firsts = map(lambda tup: str(tup[0]), w)
firsts = [str(tup[0]) for tup in w]
tarfile.py: return map(lambda m: m.name, self.infolist())
return [m.name for m in self.infolist()]
tarfile.py: return filter(lambda m: m.type in REGULAR_TYPES,
self.tarfile.getmembers())
return [m for m in self.tarfile.getmembers()
if m.type in REGULAR_TYPES]
urllib2.py: return map(lambda x: x.strip(), list)
return [x.strip() for x in list]
webbrowser.py: _tryorder = filter(lambda x: x.lower() in _browsers
or x.find("%s") > -1, _tryorder
_tryorder = [x for x in _tryorder
if x.lower() in _browsers or x.find("%s") > -1]


* Functions I don't know how to rewrite
Some functions I looked at, I couldn't figure out a way to rewrite them
without introducing a new name or adding new statements. (Warning: I
have trouble following code that uses 'reduce', so I only glossed over
lambdas in reduce calls.)

calendar.py: _months.insert(0, lambda x: "")
cgitb.py: inspect.formatargvalues(args, varargs, varkw, locals,
formatvalue=lambda value: '=' + pydoc.html.repr(value))
cgitb.py: inspect.formatargvalues(args, varargs, varkw, locals,
formatvalue=lambda value: '=' + pydoc.text.repr(value))
csv.py: quotechar = reduce(lambda a, b, quotes = quotes:
(quotes[a] > quotes[b]) and a or b, quotes.keys())
csv.py: delim = reduce(lambda a, b, delims = delims:
(delims[a] > delims[b]) and a or b, delims.keys())
difflib.py: matches = reduce(lambda sum, triple: sum + triple[-1],
self.get_matching_blocks(), 0)
gettext.py: return eval('lambda n: int(%s)' % plural)
gettext.py: self.plural = lambda n: int(n != 1)
inspect.py: classes.sort(key=lambda c: (c.__module__, c.__name__))
inspect.py: def formatargspec(args, varargs=None, varkw=None,
...
formatvarargs=lambda name: '*' + name,
formatvarkw=lambda name: '**' + name,
formatvalue=lambda value: '=' + repr(value),
inspect.py: def formatargvalues(args, varargs, varkw, locals,
...
formatvarargs=lambda name: '*' + name,
formatvarkw=lambda name: '**' + name,
formatvalue=lambda value: '=' + repr(value),
pyclbr.py: objs.sort(lambda a, b: cmp(getattr(a, 'lineno', 0),
getattr(b, 'lineno', 0)))
SimpleHTTPServer.py: list.sort(key=lambda a: a.lower())
subprocess.py: p = Popen(["id"], preexec_fn=lambda: os.setuid(100))
symtable.py: self.__params = self.__idents_matching(lambda x:
x & DEF_PARAM)
symtable.py: self.__locals = self.__idents_matching(lambda x:
x & DEF_BOUND)
symtable.py: self.__globals = self.__idents_matching(lambda x:
x & glob)
urllib2.py:setattr(self, '%s_open' % type,
lambda r, proxy=url, type=type, meth=self.proxy_open:
meth(r, proxy, type))
xdrlib.py: unpacktest = [
(up.unpack_uint, (), lambda x: x == 9),
(up.unpack_bool, (), lambda x: not x),
(up.unpack_bool, (), lambda x: x),
(up.unpack_uhyper, (), lambda x: x == 45L),
(up.unpack_float, (), lambda x: 1.89 < x < 1.91),
(up.unpack_double, (), lambda x: 1.89 < x < 1.91),
(up.unpack_string, (), lambda x: x == 'hello world'),
(up.unpack_list, (up.unpack_uint,), lambda x: x == range(5)),
(up.unpack_array, (up.unpack_string,),
lambda x: x == ['what', 'is', 'hapnin', 'doctor']),
]



Of the functions that I don't know how to rewrite, I think there are a
few interesting cases:

(1) lambda x: ""
This is the kind of parameter adaptation that I think Jeff Shannon was
talking about in another lambda thread. Using the ignoreargs function I
suggested there[1], you could rewrite this as:
ignoreargs(str, 1)


(2) lambda a: a.lower()
My first thought here was to use str.lower instead of the lambda, but of
course that doesn't work if 'a' is a unicode object:

py> str.lower(u'a')
Traceback (most recent call last):
File "<interactive input>", line 1, in ?
TypeError: descriptor 'lower' requires a 'str' object but received a
'unicode'

It's too bad I can't do something like:
basestring.lower


(3) self.plural = lambda n: int(n != 1)
Note that this is *almost* writable with def syntax. If only we could do:
def self.plural(n):
int(n != 1)


(4) objs.sort(lambda a, b: cmp(getattr(a, 'lineno', 0),
getattr(b, 'lineno', 0)))
My first intuition here was to try something like:
objs.sort(key=operator.attrgetter('lineno'))
but this doesn't work because then we don't get the default value of 0
if the attribute doesn't exist. I wonder if operator.attrgetter should
take an optional "default" parameter like getattr does:
Help on built-in function getattr in module __builtin__:

getattr(...)
getattr(object, name[, default]) -> value


(5) lambda x: x & DEF_PARAM
This could probably be written as:
functional.partial(operator.and_, DEF_PARAM)
if PEP 309[2] was accepted, thought I'm not claiming that's any clearer...



So, those are my thoughts on how lambdas are "really" used. If others
out there have real-life code that uses lambdas in interesting ways,
feel free to share them here!

Steve

[1]http://mail.python.org/pipermail/python-list/2004-December/257982.html
[2]http://python.fyxm.net/peps/pep-0309.html
 
Reply With Quote
 
 
 
 
Alex Martelli
Guest
Posts: n/a
 
      12-31-2004
Steven Bethard <(E-Mail Removed)> wrote:

> (2) lambda a: a.lower()
> My first thought here was to use str.lower instead of the lambda, but of
> course that doesn't work if 'a' is a unicode object:


Right, but string.lower works (after an 'import string'). More
generally, maybe it would be nice to have a way to say "call a method on
x" without x's type being checked, just like attrgetter says "fetch an
attribute on x" -- say s/thing like:

def methodcaller(method_name, *a, **k):
def callit(x):
return getattr(x, method_name)(*a, **k)
callit.__name__ = method_name
return callit


> (3) self.plural = lambda n: int(n != 1)
> Note that this is *almost* writable with def syntax. If only we could do:
> def self.plural(n):
> int(n != 1)


Not sure about the context, but maybe we could use, at class-level:
@staticmethod
def plural(n):
return int(n != 1)


> (4) objs.sort(lambda a, b: cmp(getattr(a, 'lineno', 0),
> getattr(b, 'lineno', 0)))
> My first intuition here was to try something like:
> objs.sort(key=operator.attrgetter('lineno'))
> but this doesn't work because then we don't get the default value of 0
> if the attribute doesn't exist. I wonder if operator.attrgetter should
> take an optional "default" parameter like getattr does:


The optional default parameter sounds like a good idea to me.


Even though a good number of lambda uses may be avoidable or removable
by such means, I think there's just slightly too much variety -- in some
cases, a def with a name will have to be best (just as it would even
today if, say, an if/else had to be part of the logic -- simulations of
ternary operators being rarely clearest and most solid).


Alex
 
Reply With Quote
 
 
 
 
Diez B. Roggisch
Guest
Posts: n/a
 
      12-31-2004
> So, those are my thoughts on how lambdas are "really" used. If others
> out there have real-life code that uses lambdas in interesting ways,
> feel free to share them here!


I use them in conjunction with metaclasses and properties:

def _s_item(self, item):
""" saw::active """
self.__item = item
self.set_state()
self.indexWidget.setText("%i" % item.index)
created = item.created
dt = QDateTime(QDate(created.year, created.month, created.day),
QTime(created.hour, created.minute,created.second))
self.createdWidget.setDateTime(dt)
self.set_text()
self.set_list_items(self.history, item.history)
self.set_list_items(self.trainlog, item.train_log)
self.set_classification_result()

self.adjust_header_sizes()

def _g_item(self):
return self.__item

# the lambda is needed for late-binding so that metaclass-wrapping will
# be in effect.
item = property(_g_item, lambda self, v: self._s_item(v))


The doc string of _s_item contains a token the metaclass is aware of and
creates a wrapper around _s_item. That works nice on methods, but I found
that properties got bound to their functions _before_ the metaclass kicks
in, so the property wasn't called in the wrapped version, resulting in
errors. So I introduced the lambda that makes the method call "lazy". Of
course I could have introduced a

def _s_item_unwrapped(self, v):
self._s_item(v)

and used that in the property - but as there are lambdas, I use them

And the second def here is not more explanatory, as one has to graps the
internal details of python properties to understand why that extra hoop is
introduced in the first place.
--
Regards,

Diez B. Roggisch
 
Reply With Quote
 
Steven Bethard
Guest
Posts: n/a
 
      12-31-2004
Alex Martelli wrote:
> Steven Bethard <(E-Mail Removed)> wrote:
>
>>(2) lambda a: a.lower()
>>My first thought here was to use str.lower instead of the lambda, but of
>>course that doesn't work if 'a' is a unicode object:

>
>
> Right, but string.lower works (after an 'import string'). More
> generally, maybe it would be nice to have a way to say "call a method on
> x" without x's type being checked, just like attrgetter says "fetch an
> attribute on x" -- say s/thing like:
>
> def methodcaller(method_name, *a, **k):
> def callit(x):
> return getattr(x, method_name)(*a, **k)
> callit.__name__ = method_name
> return callit


Yeah, that's exactly the kind of thing I was looking for. Very nice!

>>(3) self.plural = lambda n: int(n != 1)
>>Note that this is *almost* writable with def syntax. If only we could do:
>> def self.plural(n):
>> int(n != 1)

>
>
> Not sure about the context, but maybe we could use, at class-level:
> @staticmethod
> def plural(n):
> return int(n != 1)


The context was within the _parse method of GNUTranslations. Basically,
this method uses the fp passed in and a bunch of conditionals to
determine how to define the plural method. So I don't think it can be
done at the class level. Also, doesn't the assignment:
self.plural = lambda n: int(n != 1)
make this more like (at class level):
def plural(self, n):
return int(n != 1)
that is, isn't this an instance method, not a staticmethod?

py> class C(object):
.... def __init__(self):
.... self.plural = lambda n: int(n != 1)
....
py> c = C()
py> c.__class__.plural(1)
Traceback (most recent call last):
File "<interactive input>", line 1, in ?
AttributeError: type object 'C' has no attribute 'plural'
py> c.plural(1)
0

> Even though a good number of lambda uses may be avoidable or removable
> by such means, I think there's just slightly too much variety -- in some
> cases, a def with a name will have to be best


Yup, that was my feeling. I was only able to rewrite as an expression
about 50% of the lambdas that I found. However, I (personally) don't
have much of a problem with adding a def in most of the other cases.
The only ones that make me a little nervous are examples like:

inspect.py: def formatargspec(args, varargs=None, varkw=None,
...
formatvarargs=lambda name: '*' + name,
formatvarkw=lambda name: '**' + name,
formatvalue=lambda value: '=' + repr(value),

where the lambdas are declaring functions as keyword arguments in a def.
I'm not sure how much I like adding to the module multiple function
defs that are really intended to be accessed only within formatargspec.
Still, were lambda to go away in Python 3000, it certainly wouldn't be
the end of the world.

Steve
 
Reply With Quote
 
Alex Martelli
Guest
Posts: n/a
 
      12-31-2004
Steven Bethard <(E-Mail Removed)> wrote:

> >>(3) self.plural = lambda n: int(n != 1)
> >>Note that this is *almost* writable with def syntax. If only we could do:
> >> def self.plural(n):
> >> int(n != 1)

> >
> > Not sure about the context, but maybe we could use, at class-level:
> > @staticmethod
> > def plural(n):
> > return int(n != 1)

>
> The context was within the _parse method of GNUTranslations. Basically,
> this method uses the fp passed in and a bunch of conditionals to
> determine how to define the plural method. So I don't think it can be


Ah, OK -- I see, then you're probably quite right here!

> done at the class level. Also, doesn't the assignment:
> self.plural = lambda n: int(n != 1)
> make this more like (at class level):
> def plural(self, n):
> return int(n != 1)
> that is, isn't this an instance method, not a staticmethod?


Apart from the different possible definitions (which are of course
crucial), I don't see that. Given the fact that, if you define plural
as an instancemethod, you're not using 'self' anyway, what usage would
break with a staticmethod? "Doesn't use 'self'" smells more like a
staticmethod to me, even if you always call it on an instance.

> py> class C(object):
> ... def __init__(self):
> ... self.plural = lambda n: int(n != 1)
> ...
> py> c = C()
> py> c.__class__.plural(1)
> Traceback (most recent call last):
> File "<interactive input>", line 1, in ?
> AttributeError: type object 'C' has no attribute 'plural'
> py> c.plural(1)
> 0


This shows that staticmethod has slightly wider applicability, yes, but
I don't see this as a problem. IOW, I see no real use cases where it's
important that hasattr(C, 'plural') is false while hasattr(C(),
'plural') is true [I could of course be missing something!].


Alex

 
Reply With Quote
 
Steven Bethard
Guest
Posts: n/a
 
      12-31-2004
Alex Martelli wrote:
> Steven Bethard <(E-Mail Removed)> wrote:
>>
>>py> class C(object):
>>... def __init__(self):
>>... self.plural = lambda n: int(n != 1)
>>...
>>py> c = C()
>>py> c.__class__.plural(1)
>>Traceback (most recent call last):
>> File "<interactive input>", line 1, in ?
>>AttributeError: type object 'C' has no attribute 'plural'
>>py> c.plural(1)
>>0

>
>
> This shows that staticmethod has slightly wider applicability, yes, but
> I don't see this as a problem. IOW, I see no real use cases where it's
> important that hasattr(C, 'plural') is false while hasattr(C(),
> 'plural') is true [I could of course be missing something!].


True, true. I guess I was just wrapped up in reproducing the class
behavior. Making it available as a staticmethod of the class would of
course only add functionality, not remove any.

Steve
 
Reply With Quote
 
Adam DePrince
Guest
Posts: n/a
 
      12-31-2004
On Fri, 2004-12-31 at 01:53, Steven Bethard wrote:
> I thought it might be useful to put the recent lambda threads into
> perspective a bit. I was wondering what lambda gets used for in "real"
> code, so I grepped my Python Lib directory. Here are some of the ones I
> looked, classified by how I would rewrite them (if I could):
>

<snipping wonderful verbosity>

> So, those are my thoughts on how lambdas are "really" used. If others
> out there have real-life code that uses lambdas in interesting ways,
> feel free to share them here!
>


Lets not forget the "real reason" for lambda ... the elegance of
orthogonality. Why treat functions differently than any other object?

We can operate on every other class without having to involve the
namespace, why should functions be any different? Wouldn't it to awful
if we had to write:

x = 3 * y ** 2 + 4 * y + 5

as

a = 3
e = 2
b = 4
c = 5
x = a * y ** e + b * y + c

Everybody understand that sometimes a number just isn't important enough
to assign to the name space. And we all can understand that this
applies to other data types, for example:

print "The answer is", x

Unless internationalization was a concern, few people would write:

THE_ANSWER_IS = "The answer is"
print THE_ANSWER_IS, x

But when we consider functions, we suddenly change. Understandably we
have a certain bias towards functions. When programming, the most
commonly constructed object is the function. We likely spend more time
crafting function objects than any other object. Our reflex to
economize on the programming process focuses on the reduction in
function code creation time, hence the a focus on reuseabiity and a
plethora of ways to categorize our code to achieve this end.

The notion that we would use a function exactly once is almost blasphemy
to such a perspective. But it is true ... there are times when a
programmer will want to construct and use a function in exactly one
place for one singular purpose. In my own code, this occurs most often
when the function is used as a parameters to another function.

Examples of this are the cmp parameters to [].sort. The function I
provide to cmp is only barely more important to preserve for posterity
than the value I might provide to the same functions "reverse"
parameter.

In sort, we must preserve the ability to create an anonymous function
simply because we can do so for every other object type, and functions
are not special enough to permit this special case.

Adam DePrince


 
Reply With Quote
 
Steven Bethard
Guest
Posts: n/a
 
      12-31-2004
Adam DePrince wrote:
> Lets not forget the "real reason" for lambda ... the elegance of
> orthogonality. Why treat functions differently than any other object?
>
> We can operate on every other class without having to involve the
> namespace, why should functions be any different?


Yup. I think in most of the examples that I didn't know how to rewrite,
this was basically the issue. On the other hand, I do think that
lambdas get overused, as indicated by the number of examples I *was*
able to rewrite.[1]

Still, I have to admit that in some cases (especially those involving
reduce), I wish the coder had named the function -- it would have given
me a little bit more documentation as to what the code was trying to do.

On the other hand, in other cases, like when a function is a keyword
argument to another function (e.g. inspect.py's "def formatargspec..."
example) using a def statement and naming the function would be redundant.

Steve

[1] Note that this isn't entirely fair to the examples, some of which
were written before list comprehensions, generator expressions and
itemgetter/attrgetter.
 
Reply With Quote
 
Hans Nowak
Guest
Posts: n/a
 
      12-31-2004
Adam DePrince wrote:

> In sort, we must preserve the ability to create an anonymous function
> simply because we can do so for every other object type, and functions
> are not special enough to permit this special case.


Your reasoning makes sense... lambda enables you to create a function as
part of an expression, just like other types can be part of an
expression. However, by that same reasoning, maybe classes aren't
special enough either to warrant a special case. Where's the keyword to
create an anonymous class?

--
Hans Nowak
http://zephyrfalcon.org/

 
Reply With Quote
 
Steven Bethard
Guest
Posts: n/a
 
      12-31-2004
Hans Nowak wrote:
> Adam DePrince wrote:
>
>> In sort, we must preserve the ability to create an anonymous function
>> simply because we can do so for every other object type, and functions
>> are not special enough to permit this special case.

>
>
> Your reasoning makes sense... lambda enables you to create a function as
> part of an expression, just like other types can be part of an
> expression. However, by that same reasoning, maybe classes aren't
> special enough either to warrant a special case. Where's the keyword to
> create an anonymous class?


Well, no keyword, but you can use the type function:

py> d = dict(c=type('C', (object,), dict(spam=42)),
.... d=type('D', (dict,), dict(badger=True)))
py> d['c'].spam
42
py> d['c']()
<__main__.C object at 0x063F2DD0>


Steve
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Type of lambda function returning a lambda function... Haochen Xie C++ 4 03-17-2013 11:23 PM
Why is that in JDK8: value used in lambda expression shuld beeffectively final? jeti789@web.de Java 27 01-07-2013 10:08 PM
lambda vs non-lambda proc Steve Dogers Ruby 1 03-30-2009 10:11 PM
Re: Lambda as declarative idiom (was RE: what is lambda used for inreal code?) Roman Suzi Python 13 01-07-2005 09:33 PM
product of real and (integer)(after converted to real one) value - vhdl found fatal error senthil VHDL 5 01-24-2004 04:37 AM



Advertisments