Velocity Reviews > List comprehension + lambdas - strange behaviour

# List comprehension + lambdas - strange behaviour

Artur Siekielski
Guest
Posts: n/a

 05-06-2010
Hello.
I found this strange behaviour of lambdas, closures and list
comprehensions:

>>> funs = [lambda: x for x in range(5)]
>>> [f() for f in funs]

[4, 4, 4, 4, 4]

Of course I was expecting the list [0, 1, 2, 3, 4] as the result. The
'x' was bound to the final value of 'range(5)' expression for ALL
defined functions. Can you explain this? Is this only counterintuitive
example or an error in CPython?

Regards,
Artur

Raymond Hettinger
Guest
Posts: n/a

 05-06-2010
On May 6, 9:34*pm, Artur Siekielski <(E-Mail Removed)>
wrote:
> Hello.
> I found this strange behaviour of lambdas, closures and list
> comprehensions:
>
> >>> funs = [lambda: x for x in range(5)]
> >>> [f() for f in funs]

>
> [4, 4, 4, 4, 4]
>
> Of course I was expecting the list [0, 1, 2, 3, 4] as the result. The
> 'x' was bound to the final value of 'range(5)' expression for ALL
> defined functions. Can you explain this? Is this only counterintuitive
> example or an error in CPython?

Try binding the value of x for each of the inner functions:

>>> funs = [lambda x=x: x for x in range(5)]
>>> [f() for f in funs]

[0, 1, 2, 3, 4]

Otherwise, the 'x' is just a global value and the lambdas look it up
at when the function is invoked. Really, not surprising at all:

>>> x = 10
>>> def f():

.... return x
....
>>> x = 20
>>> f()

20

Raymond

Emile van Sebille
Guest
Posts: n/a

 05-06-2010
On 5/6/2010 12:34 PM Artur Siekielski said...
> Hello.
> I found this strange behaviour of lambdas, closures and list
> comprehensions:
>
>>>> funs = [lambda: x for x in range(5)]

funs is now a list of lambda functions that return 'x' (whatever it
currently is from whereever it's accessible when invoked)

>>> [f() for f,x in zip(funs,range(5))]

[0, 1, 2, 3, 4]
>>> del x
>>> [f() for f in funs]

Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "<stdin>", line 1, in <lambda>
NameError: global name 'x' is not defined
>>>

Emile

Benjamin Peterson
Guest
Posts: n/a

 05-06-2010
Artur Siekielski <artur.siekielski <at> gmail.com> writes:
>
> Of course I was expecting the list [0, 1, 2, 3, 4] as the result. The
> 'x' was bound to the final value of 'range(5)' expression for ALL
> defined functions. Can you explain this? Is this only counterintuitive
> example or an error in CPython?

The former. Closures are rebound in a loop.

Terry Reedy
Guest
Posts: n/a

 05-07-2010
On 5/6/2010 3:34 PM, Artur Siekielski wrote:
> Hello.
> I found this strange behaviour of lambdas, closures and list
> comprehensions:
>
>>>> funs = [lambda: x for x in range(5)]
>>>> [f() for f in funs]

> [4, 4, 4, 4, 4]

You succumbed to lambda hypnosis, a common malady .
The above will not work in 3.x, which does not leak comprehension
iteration variables. It is equivalent to

funs = [lambda: x for y in range(5)]
del y # only for 2.x. y is already gone in 3.x
x = 4
[f() for f in funs]

Now, I am sure, you would expect what you got.

and nearly equivalent to

def f(): return x
x=8
funs = [f for x in range(5)]
[f() for f in funs]

# [8,8,8,8,8] in 3.x

Ditto

Terry Jan Reedy

Neil Cerutti
Guest
Posts: n/a

 05-07-2010
On 2010-05-07, Terry Reedy <(E-Mail Removed)> wrote:
> On 5/6/2010 3:34 PM, Artur Siekielski wrote:
>> Hello.
>> I found this strange behaviour of lambdas, closures and list
>> comprehensions:
>>
>>>>> funs = [lambda: x for x in range(5)]
>>>>> [f() for f in funs]

>> [4, 4, 4, 4, 4]

>
> You succumbed to lambda hypnosis, a common malady . The
> above will not work in 3.x, which does not leak comprehension
> iteration variables.

It functions the same in 3.1.

Python 3.1.1 (r311:74483, Aug 17 2009, 17:02:12) [MSC v.1500 32 bit (Intel)] on
win32
>>> funs = [lambda: x for x in range(5)]
>>> [f() for f in funs]

[4, 4, 4, 4, 4]

--
Neil Cerutti
*** Your child was bitten by a Bat-Lizard. ***

Terry Reedy
Guest
Posts: n/a

 05-07-2010
On 5/7/2010 8:31 AM, Neil Cerutti wrote:
> On 2010-05-07, Terry Reedy<(E-Mail Removed)> wrote:
>> On 5/6/2010 3:34 PM, Artur Siekielski wrote:
>>> Hello.
>>> I found this strange behaviour of lambdas, closures and list
>>> comprehensions:
>>>
>>>>>> funs = [lambda: x for x in range(5)]
>>>>>> [f() for f in funs]
>>> [4, 4, 4, 4, 4]

>>
>> You succumbed to lambda hypnosis, a common malady . The
>> above will not work in 3.x, which does not leak comprehension
>> iteration variables.

>
> It functions the same in 3.1.
>
> Python 3.1.1 (r311:74483, Aug 17 2009, 17:02:12) [MSC v.1500 32 bit (Intel)] on
> win32
>>>> funs = [lambda: x for x in range(5)]
>>>> [f() for f in funs]

> [4, 4, 4, 4, 4]

Ok.

>>> x

Traceback (most recent call last):
File "<pyshell#1>", line 1, in <module>
x
NameError: name 'x' is not defined
#only in 3.x

But because the list comp is implemented in 3.x as an anonymous
function, which is then called and discarded (an implementation that I
believe is not guaranteed by the language ref), the lambda expression
defines a nested function which captures the (final) value of x.

>>> funs[0].__closure__[0].cell_contents

4

So it works (runs without exception), but somewhat accidentally and for
a different reason than in 2.x, where 'x' is 4 at the global level.

Terry Jan Reedy