Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > Misunderstanding about closures

Reply
Thread Tools

Misunderstanding about closures

 
 
Alexander May
Guest
Posts: n/a
 
      06-07-2004
When I define a function in the body of a loop, why doesn't the function
"close" on the loop vairable? See example below.

Thanks,
Alex

C:\Documents and Settings\Alexander May>python
Python 2.3.3 (#51, Dec 18 2003, 20:22:39) [MSC v.1200 32 bit (Intel)] on
win32
Type "help", "copyright", "credits" or "license" for more information.
>>> l=[]
>>> for x in xrange(10):

.... def f():
.... return x
.... l.append(f)
....
>>> l

[<function f at 0x008E66F0>, <function f at 0x008E6AB0>, <function f at
0x008EC130>, <function f at 0x008EC170>, <functi
on f at 0x008EC1B0>, <function f at 0x008EC1F0>, <function f at 0x008EC230>,
<function f at 0x008EC270>, <function f at
0x008EC2B0>, <function f at 0x008EC2F0>]
>>> for f in l:

.... f()
....
9
9
9
9
9
9
9
9
9
9

On the other hand, the following works as I expected.

>>> l=[]
>>> def makef(x):

.... def f():
.... return x
.... return f
....
>>> for x in xrange(10):

.... l.append(makef(x))
....
>>> l

[<function f at 0x008E6AB0>, <function f at 0x008EC330>, <function f at
0x008EC1F0>, <function f at 0x008EC230>, <functi
on f at 0x008EC270>, <function f at 0x008EC2B0>, <function f at 0x008EC0F0>,
<function f at 0x008EC370>, <function f at
0x008EC3B0>, <function f at 0x008EC3F0>]
>>> for f in l:

.... f()
....
0
1
2
3
4
5
6
7
8
9
>>>



 
Reply With Quote
 
 
 
 
Michael Geary
Guest
Posts: n/a
 
      06-07-2004
Alexander May wrote:
> When I define a function in the body of a loop, why doesn't the
> function "close" on the loop vairable? See example below.


Hi Alex,

Your message title is correct: you're misunderstanding how closures work.


A closure doesn't save the current value that's bound to a name, it saves a
reference to the name itself. If you bind a different value to the name
later, the closure will see that new value.

In your first example, there is a single local name x that each instance of
the f closure refers to, so they all see the same value.

In the second example, each time you call the makef function, you create a
new local name x that belongs to that instance of makef. So when you create
the closures inside makef, each one sees its own value that is unrelated to
the others.

BTW, you'll see the same effect in JavaScript or any other language that
supports closures.

-Mike

> C:\Documents and Settings\Alexander May>python
> Python 2.3.3 (#51, Dec 18 2003, 20:22:39) [MSC v.1200 32 bit (Intel)] on
> win32
> Type "help", "copyright", "credits" or "license" for more information.
> >>> l=[]
> >>> for x in xrange(10):

> ... def f():
> ... return x
> ... l.append(f)
> ...
> >>> l

> [<function f at 0x008E66F0>, <function f at 0x008E6AB0>, <function f at
> 0x008EC130>, <function f at 0x008EC170>, <functi
> on f at 0x008EC1B0>, <function f at 0x008EC1F0>, <function f at

0x008EC230>,
> <function f at 0x008EC270>, <function f at
> 0x008EC2B0>, <function f at 0x008EC2F0>]
> >>> for f in l:

> ... f()
> ...
> 9
> 9
> 9
> 9
> 9
> 9
> 9
> 9
> 9
> 9
>
> On the other hand, the following works as I expected.
>
> >>> l=[]
> >>> def makef(x):

> ... def f():
> ... return x
> ... return f
> ...
> >>> for x in xrange(10):

> ... l.append(makef(x))
> ...
> >>> l

> [<function f at 0x008E6AB0>, <function f at 0x008EC330>, <function f at
> 0x008EC1F0>, <function f at 0x008EC230>, <functi
> on f at 0x008EC270>, <function f at 0x008EC2B0>, <function f at

0x008EC0F0>,
> <function f at 0x008EC370>, <function f at
> 0x008EC3B0>, <function f at 0x008EC3F0>]
> >>> for f in l:

> ... f()
> ...
> 0
> 1
> 2
> 3
> 4
> 5
> 6
> 7
> 8
> 9
> >>>

>
>



 
Reply With Quote
 
 
 
 
fishboy
Guest
Posts: n/a
 
      06-07-2004
On Mon, 07 Jun 2004 03:27:22 GMT, "Alexander May"
<(E-Mail Removed)> wrote:

>When I define a function in the body of a loop, why doesn't the function
>"close" on the loop vairable? See example below.
>

Hmmm, I'm having a hard time phrasing the answer to this in a form
besides, "Because the function doesn't 'close' on a loop variable"

Hrmmm
......
Ok, how about this

x = 'a'
for x in range(10):
foo(x)
bar(x)

What value do you expect 'x' to be in bar(x)?

Because it sounds like you were expecting it to be bar('a') instead of
bar(9).

Hrmm, I just remembered. That's a C thing, isn't it? The loop
variable being only defined inside the loop.

God that makes to happy. Forgetting stuff like that. Maybe in
another 20 years I'll forget everything I ever knew about static typed
languages and pass into hacker Nirvana.

><{{{*>




 
Reply With Quote
 
Alexander Schmolck
Guest
Posts: n/a
 
      06-07-2004
"Michael Geary" <(E-Mail Removed)> writes:

> Alexander May wrote:
>> When I define a function in the body of a loop, why doesn't the
>> function "close" on the loop vairable? See example below.

>
> Hi Alex,
>
> Your message title is correct: you're misunderstanding how closures work.
>


Not necessarily. See below.

[snipped]

> BTW, you'll see the same effect in JavaScript or any other language that
> supports closures.


Not quite. In fact in the language that more or less started it all, scheme,
the standard iteration construct 'do' does indeed introduce a *new binding* on
each iteration.


;; Note: this is is a literate translation and *not* idiomatic scheme -- no
;; schemer would write it like that
(define l '()) ; an empty list

(do ((x 0 (+ 1 x))) ; start x with 0, then add 1 at each step
((= x 10)) ; stop when x is 10
(set! l (cons (lambda () x) l))) ; add a new function closing over x to
; the *front* of l

(set! l (reverse l)) ; since we added to the front we
; have to reverse l

((list-ref l 3)) ; get list element 3 and execute it
> 3 ;ANSWER



Common Lisp OTOH doesn't -- like python:

;; This is similarly non-idiomatic (and won't work in emacs lisp, which hasn't
;; got lexical scope)
(defvar l '()) ; an empty list

(do ((x 0 (+ 1 x))) ; start x with 0, then add 1 at each step
((= x 10)) ; when x is 10 return the reversed list
(setf l (cons (lambda () x) l))) ; add a new function closing over x to
; the *front* of l

(setq l (reverse l)) ; since we added to the front we
; have to reverse l

(funcall (nth 3 l)) ; get list element 3 and execute it
> 10





 
Reply With Quote
 
Hung Jung Lu
Guest
Posts: n/a
 
      06-07-2004
"Michael Geary" <(E-Mail Removed)> wrote:
>
> Your message title is correct: you're misunderstanding how closures work.
>


So seems to be your case.

> In your first example, there is a single local name x that each instance of
> the f closure refers to, so they all see the same value.


There is a name, period. This name is neither local or global. This
name is looked up first in the locals() dictionary, then in the
globals() dictionary. In this particular example, because the locals()
dictionary is empty, this name is actually pulled from globals().

> In the second example, each time you call the makef function, you create a
> new local name x that belongs to that instance of makef. So when you create
> the closures inside makef, each one sees its own value that is unrelated to
> the others.


And your explanation is supposed to enlighten a beginner?

A more complete explanation. The key is in the argument list. Because
'x' appears in the argument list of makef(x), this name is inserted
into the locals() dictionary of makef()'s scope. That is, the name 'x'
inside makef()'s scope is pulled from the locals() dictionary of that
scope. Now, due to the magic of nested scope (which was not always the
case in older versions of Python), this 'x' is also inserted into the
locals() dictionary of the nested f() function, and this 'x' is bound
to the value at that moment, because during the constructions of f(),
it is found that 'x' is used in expressions AND it exists in the
containing scope's locals() dictionary at the moment of construction.
In particular, it will not work correctly if you replace the statement
"return x" with "return eval('x')". Everything becomes more clear when
you insert statements to print out the locals() and globals()
dictionaries.

regards,

Hung Jung
 
Reply With Quote
 
Michael Geary
Guest
Posts: n/a
 
      06-07-2004
> > Alexander May wrote:
> >> When I define a function in the body of a loop, why doesn't the
> >> function "close" on the loop vairable? See example below.


> Michael Geary wrote:
> > Your message title is correct: you're misunderstanding how
> > closures work.
> >
> > BTW, you'll see the same effect in JavaScript or any other
> > language that supports closures.


Alexander Schmolck wrote:
> Not quite. In fact in the language that more or less started it all,
> scheme, the standard iteration construct 'do' does indeed introduce
> a *new binding* on each iteration.


Yeah, there are two separate issues here. It wasn't clear which one Alex M.
was misunderstanding, and I made an assumption about it (always a bad
idea!).

One issue, which I assumed was the problem, has nothing to do with loops,
but with the very nature of closures: What does a closure save, the current
value that a name or variable is bound to, or a reference to that variable?

The other issue is what you're talking about: Does a loop introduce new
bindings on each iteration or not?

To illustrate, here's a variation on Alex's example without the loop:

g = []

def outer():
x = 1
def f():
return x
g.append( f )
x = 2
g.append( f )

outer()
print g[0](), g[1]()

This prints:

2 2

If a closure saved the current value of a variable, it would print:

1 2

Now I am guessing that if you translated this code into any language that
supports closures, including Scheme, you would get the "2 2" result, is that
right? After all, this is pretty much the definition of a closure, that it
saves a reference, not the current value.

If the point of confusion was whether a loop creates new bindings or not,
then my reply was irrelevant--but maybe this discussion will help someone
else understand closures better.

Alex M., now you know the rest of the story...

-Mike


 
Reply With Quote
 
Michael Geary
Guest
Posts: n/a
 
      06-07-2004
Hung Jung Lu wrote:
> And your explanation is supposed to enlighten a beginner?
>
> A more complete explanation. The key is in the argument list.
> Because 'x' appears in the argument list of makef(x), this name
> is inserted into the locals() dictionary of makef()'s scope. That
> is, the name 'x' inside makef()'s scope is pulled from the locals()
> dictionary of that scope. Now, due to the magic of nested scope
> (which was not always the case in older versions of Python),
> this 'x' is also inserted into the locals() dictionary of the nested
> f() function, and this 'x' is bound to the value at that moment,
> because during the constructions of f(), it is found that 'x' is
> used in expressions AND it exists in the containing scope's
> locals() dictionary at the moment of construction. In particular,
> it will not work correctly if you replace the statement "return x"
> with "return eval('x')". Everything becomes more clear when
> you insert statements to print out the locals() and globals()
> dictionaries.


Thank you for the more complete and accurate explanation, Hung Jung. I was
thinking in two languages at once, JavaScript and Python, and I started to
write in terms of how closures work in JavaScript. Then I thought I'd better
make it more relevant to Python but didn't do a very good job switching
over.

-Mike


 
Reply With Quote
 
Alexander Schmolck
Guest
Posts: n/a
 
      06-07-2004
"Michael Geary" <(E-Mail Removed)> writes:
> g = []
> def outer():
> x = 1
> def f():
> return x
> g.append( f )
> x = 2
> g.append( f )
>
> outer()
> print g[0](), g[1]()
>
> This prints:
>
> 2 2


> Now I am guessing that if you translated this code into any language that
> supports closures, including Scheme, you would get the "2 2" result, is that
> right?


Yep -- here's the scheme version.

(define g '())
(define (outer)
(let* ((x 1)
(f (lambda () x)))
(set! g (cons f g))
(set! x 2)
(set! g (cons f g))))
(outer)
(list ((car g)) ((cadr g)))
=> (2 2)

'as
 
Reply With Quote
 
Michele Simionato
Guest
Posts: n/a
 
      06-08-2004
Alexander Schmolck <(E-Mail Removed)> wrote in message news:<(E-Mail Removed)>...
> Not quite. In fact in the language that more or less started it all, scheme,
> the standard iteration construct 'do' does indeed introduce a *new binding* on
> each iteration.

<snip>
> Common Lisp OTOH doesn't -- like python:
>

<snip>

A thoughtful discussion of Python/Scheme/Lisp closures and for loops
was made by
Jacek Generowicz in this thread:

http://groups.google.it/groups?hl=it....lang.python.*
 
Reply With Quote
 
Alexander May
Guest
Posts: n/a
 
      06-08-2004
Thanks everybody. I now have a complete understanding of all the issues.
Alex

"Michele Simionato" <(E-Mail Removed)> wrote in message
news:(E-Mail Removed) om...
> Alexander Schmolck <(E-Mail Removed)> wrote in message

news:<(E-Mail Removed)>...
> > Not quite. In fact in the language that more or less started it all,

scheme,
> > the standard iteration construct 'do' does indeed introduce a *new

binding* on
> > each iteration.

> <snip>
> > Common Lisp OTOH doesn't -- like python:
> >

> <snip>
>
> A thoughtful discussion of Python/Scheme/Lisp closures and for loops
> was made by
> Jacek Generowicz in this thread:
>
>

http://groups.google.it/groups?hl=it...3D%26ie%3DUTF-
8%26oe%3DUTF-8%26q%3Dsimionato%2Blambda%26btnG%3DCerca%26meta%3 Dgroup%253Dco
mp.lang.python.*


 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
vec - a misunderstanding Mark Hobley Perl 1 03-18-2006 07:32 PM
RE: Misunderstanding about closures Robert Brewer Python 1 06-07-2004 04:27 PM
errors in book or my misunderstanding? David K MCSE 4 12-12-2003 03:22 PM
Re: Class property misunderstanding Kevin Spencer ASP .Net 1 06-26-2003 05:28 PM



Advertisments