Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > Reasoning behind nested scope

Reply
Thread Tools

Reasoning behind nested scope

 
 
Andy Baker
Guest
Posts: n/a
 
      08-03-2004
Hi there,

I'm learning Python at the moment and trying to grok the thinking behind
it's scoping and nesting rules.

I was googling for nested functions and found this Guido quote:
(http://www.python.org/search/hyperma...1993/0343.html)

"This is because nested function definitions don't have access to the
local variables of the surrounding block -- only to the globals of the
containing module. This is done so that lookup of globals doesn't
have to walk a chain of dictionaries -- as in C, there are just two
nested scopes: locals and globals (and beyond this, built-ins).
Therefore, nested functions have only a limited use. This was a
deliberate decision, based upon experience with languages allowing
arbitraries nesting such as Pascal and both Algols -- code with too
many nested scopes is about as readable as code with too many GOTOs.
And, of course, in Python, the "proper" way is to use an
object-oriented programming style"

This sounds reasonable to me although nested scope always struck me as more
natural and intuitive ('Don't be surprising')

I was wondering how the balance changed in favour of nested scope? What
changed people's minds about 'readability' vs other factors? Are nested
scopes still regarded as leading to spagetti code etc?

(On a side note is there any way to call a nested function from outside the
parent? I was kind of expecting nested functions to be addressable through
dot notation like methods are but I can see why that wouldn't be quite
right. This might be a better question for the tutor list...)

Andy Baker

 
Reply With Quote
 
 
 
 
Mel Wilson
Guest
Posts: n/a
 
      08-03-2004
In article <(E-Mail Removed)>,
"Andy Baker" <(E-Mail Removed)> wrote:
>"This is because nested function definitions don't have access to the
>local variables of the surrounding block -- only to the globals of the
>containing module. This is done so that lookup of globals doesn't
>have to walk a chain of dictionaries -- as in C, there are just two
>nested scopes: locals and globals (and beyond this, built-ins).
>Therefore, nested functions have only a limited use. This was a
>deliberate decision, based upon experience with languages allowing
>arbitraries nesting such as Pascal and both Algols -- code with too
>many nested scopes is about as readable as code with too many GOTOs.
>And, of course, in Python, the "proper" way is to use an
>object-oriented programming style"
>
>This sounds reasonable to me although nested scope always struck me as more
>natural and intuitive ('Don't be surprising')
>
>I was wondering how the balance changed in favour of nested scope? What
>changed people's minds about 'readability' vs other factors? Are nested
>scopes still regarded as leading to spagetti code etc?


I guess a consensus built up that the power/safety
tradeoff had given too much emphasis to safety.

My own poster child for nested scopes is in some old post
on c.l.p . I wanted to build a lot of similar labelled text
controls in wxPython, following a naming convention. I
could have written a function, but pre-nested-scope, any
change I made to the information used in building the
controls would have meant changing the function parameter
list, and all the function calls. I came up with a template
string which got parameter substituted via the '%' operator,
and then `exec`ed so as code it had full access to all the
info that was in the contemporary scope.

Post nested scope, any info the function needed from the
surrounding scope could simply be read from there. The code
was much less unusual.

I believe (without proof) that any language that won't
let me shoot myself in the foot will also prevent me doing
anything useful. Debate is heating up now about write
access to enclosing scopes. We'll see how strong my faith
really is. People will be able to write function nests
instead of classes.

>(On a side note is there any way to call a nested function from outside the
>parent? I was kind of expecting nested functions to be addressable through
>dot notation like methods are but I can see why that wouldn't be quite
>right. This might be a better question for the tutor list...)


Yup.

def f1 (a):
if a:
x = 2
else:
x = 3

def f2 (y):
return y % x

return f2

import os
b = f1 (os.name.startswith ('p'))
for c in range (10):
print b (c)



Regards. Mel.
 
Reply With Quote
 
 
 
 
Larry Bates
Guest
Posts: n/a
 
      08-03-2004
Ever since my first Fortran class some 30+ years ago
I've always liked nested scopes. If you want something
to be global, say so in your code. In Fortran we used
common blocks, in Python you use a global directive.
Making everything global can lead to many name clashes
or you must have lots of unique variable names which
lengthens code and makes reuse more difficult.

Judicious use of OOP and globals seems to be the best
approach. The more I learn/use OOP the less I seem
to require globals, but there are some times they
seem a preferable solution.

I don't believe there is any way to call nested
functions from outside their parent. If you need
something like that, make the function a class and
make the nested function a method of the class.

HTH,
Larry Bates
Syscon, Inc.


"Andy Baker" <(E-Mail Removed)> wrote in message
news:(E-Mail Removed)...
> Hi there,
>
> I'm learning Python at the moment and trying to grok the thinking behind
> it's scoping and nesting rules.
>
> I was googling for nested functions and found this Guido quote:
> (http://www.python.org/search/hyperma...1993/0343.html)
>
> "This is because nested function definitions don't have access to the
> local variables of the surrounding block -- only to the globals of the
> containing module. This is done so that lookup of globals doesn't
> have to walk a chain of dictionaries -- as in C, there are just two
> nested scopes: locals and globals (and beyond this, built-ins).
> Therefore, nested functions have only a limited use. This was a
> deliberate decision, based upon experience with languages allowing
> arbitraries nesting such as Pascal and both Algols -- code with too
> many nested scopes is about as readable as code with too many GOTOs.
> And, of course, in Python, the "proper" way is to use an
> object-oriented programming style"
>
> This sounds reasonable to me although nested scope always struck me as

more
> natural and intuitive ('Don't be surprising')
>
> I was wondering how the balance changed in favour of nested scope? What
> changed people's minds about 'readability' vs other factors? Are nested
> scopes still regarded as leading to spagetti code etc?
>
> (On a side note is there any way to call a nested function from outside

the
> parent? I was kind of expecting nested functions to be addressable through
> dot notation like methods are but I can see why that wouldn't be quite
> right. This might be a better question for the tutor list...)
>
> Andy Baker
>



 
Reply With Quote
 
Peter Otten
Guest
Posts: n/a
 
      08-03-2004
Andy Baker wrote:

> (On a side note is there any way to call a nested function from outside
> the parent? I was kind of expecting nested functions to be addressable
> through dot notation like methods are but I can see why that wouldn't be
> quite right. This might be a better question for the tutor list...)


When you are nesting functions, you don't get one function sitting inside
another function. Instead, the function creation code of the "inner"
function is executed every time to the effect that you get a new inner
function every time you call the outer one:

>>> def make():

.... def f(): return "shoobidoo"
.... return f
....
>>> f1 = make()
>>> f2 = make()
>>> f1(), f2()

('shoobidoo', 'shoobidoo')
>>> f1 is f2

False

You wouldn't do that in cases like the above, when you get nothing in return
for the extra overhead, but somtimes nesting is useful - because of the
change in the scoping rules you now get readonly-closures:

>>> def make(s):

.... def f(): return s
.... return f
....
>>> f1 = make("now")
>>> f2 = make("what")
>>> f1(), f2()

('now', 'what')

Peter

 
Reply With Quote
 
Nigel Rowe
Guest
Posts: n/a
 
      08-04-2004
Peter Otten wrote:

> Andy Baker wrote:
>
>> (On a side note is there any way to call a nested function from outside
>> the parent? I was kind of expecting nested functions to be addressable
>> through dot notation like methods are but I can see why that wouldn't be
>> quite right. This might be a better question for the tutor list...)

>
> When you are nesting functions, you don't get one function sitting inside
> another function. Instead, the function creation code of the "inner"
> function is executed every time to the effect that you get a new inner
> function every time you call the outer one:
>
>>>> def make():

> ... def f(): return "shoobidoo"
> ... return f
> ...
>>>> f1 = make()
>>>> f2 = make()
>>>> f1(), f2()

> ('shoobidoo', 'shoobidoo')
>>>> f1 is f2

> False
>
> You wouldn't do that in cases like the above, when you get nothing in
> return for the extra overhead, but somtimes nesting is useful - because of
> the change in the scoping rules you now get readonly-closures:
>
>>>> def make(s):

> ... def f(): return s
> ... return f
> ...
>>>> f1 = make("now")
>>>> f2 = make("what")
>>>> f1(), f2()

> ('now', 'what')
>
> Peter


What work is actually done when the
'nested function creation code of the "inner" function'
is executed?

Given a quick test:-
<code>
def outer():
def inner():
pass
return inner

b1 = outer()
b2 = outer()

attrs=[a for a in dir(b1) if not a.startswith('_')]
for a, a1, a2 in zip(attrs,
[getattr(b1,a) for a in attrs],
[getattr(b2,a) for a in attrs]):
print a, a1 is a2

</code>
<result>
func_closure True
func_code True
func_defaults True
func_dict True
func_doc True
func_globals True
func_name True
</result>

it appears that all the components of the inner function are the same, which
just leaves the binding of the code object to 'inner'.

Am I missing something, or is the overhead no worse than, say, foo=self.foo,
where self.foo is a method?


--
Nigel Rowe
A pox upon the spammers that make me write my address like..
rho (snail) swiftdsl (stop) com (stop) au
 
Reply With Quote
 
Peter Otten
Guest
Posts: n/a
 
      08-04-2004
Nigel Rowe wrote:

> Peter Otten wrote:
>
>> Andy Baker wrote:
>>
>>> (On a side note is there any way to call a nested function from outside
>>> the parent? I was kind of expecting nested functions to be addressable
>>> through dot notation like methods are but I can see why that wouldn't be
>>> quite right. This might be a better question for the tutor list...)

>>
>> When you are nesting functions, you don't get one function sitting inside
>> another function. Instead, the function creation code of the "inner"
>> function is executed every time to the effect that you get a new inner
>> function every time you call the outer one:
>>
>>>>> def make():

>> ... def f(): return "shoobidoo"
>> ... return f
>> ...
>>>>> f1 = make()
>>>>> f2 = make()
>>>>> f1(), f2()

>> ('shoobidoo', 'shoobidoo')
>>>>> f1 is f2

>> False
>>
>> You wouldn't do that in cases like the above, when you get nothing in
>> return for the extra overhead, but somtimes nesting is useful - because
>> of the change in the scoping rules you now get readonly-closures:
>>
>>>>> def make(s):

>> ... def f(): return s
>> ... return f
>> ...
>>>>> f1 = make("now")
>>>>> f2 = make("what")
>>>>> f1(), f2()

>> ('now', 'what')
>>
>> Peter

>
> What work is actually done when the
> 'nested function creation code of the "inner" function'
> is executed?
>
> Given a quick test:-
> <code>
> def outer():
> def inner():
> pass
> return inner
>
> b1 = outer()
> b2 = outer()
>
> attrs=[a for a in dir(b1) if not a.startswith('_')]
> for a, a1, a2 in zip(attrs,
> [getattr(b1,a) for a in attrs],
> [getattr(b2,a) for a in attrs]):
> print a, a1 is a2
>
> </code>
> <result>
> func_closure True
> func_code True
> func_defaults True
> func_dict True
> func_doc True
> func_globals True
> func_name True
> </result>
>
> it appears that all the components of the inner function are the same,
> which just leaves the binding of the code object to 'inner'.


Not the binding, a new function object is created every time outer is run,
i. e. (faked, but I did it for real above with f1 and f2)

>>> b1 is b2

False

> Am I missing something, or is the overhead no worse than, say,
> foo=self.foo, where self.foo is a method?


If it were pure Python it would rather be something like

foo = Function(func_closure=..., func_code=..., ...)

Another perspective is to look at the byte code:

>>> import dis
>>> def outer():

.... def inner(): pass
.... return inner
....
>>> dis.dis(outer)

2 0 LOAD_CONST 1 (<code object inner at
0x4028eee0, file "<stdin>", line 2>)
3 MAKE_FUNCTION 0
6 STORE_FAST 0 (inner)

3 9 LOAD_FAST 0 (inner)
12 RETURN_VALUE
13 LOAD_CONST 0 (None)
16 RETURN_VALUE
>>> def inner():

.... pass
....
>>> def outer2():

.... return inner
....
>>> dis.dis(outer2)

2 0 LOAD_GLOBAL 0 (inner)
3 RETURN_VALUE
4 LOAD_CONST 0 (None)
7 RETURN_VALUE

In the example you get three extra instructions, LOAD_CONST, MAKE_FUNCTION
and STORE_FAST (and earn the slight benefit that inner is now a local
instead of a global). A constant code object is used, meaning compilation
takes place at most once when the module is loaded. There is a dedicated
op-code, so function creation should be faster than "normal" creation of a
class instance.

Now this is all nice and dandy, but how do the two contenders perform?

$ timeit.py -s"def inner(): pass" -s"def outer(): return inner" "outer()"
1000000 loops, best of 3: 0.469 usec per loop
$ timeit.py -s"def outer():" -s" def inner(): pass" -s" return inner"
"outer()"
1000000 loops, best of 3: 1.12 usec per loop

i. e. nesting the two functions roughly doubles execution time.
However, creation of an inner function often will only take a small fraction
of the total time spent in the outer function - in the end it's just a
matter of style.

I use inner functions only when they depend on the local context because I
think it nicely discriminates closures from helpers. I tend to omit
implicit parameters even for helper funtions, therefore nesting them would
gain me nothing.

Peter

 
Reply With Quote
 
Nigel Rowe
Guest
Posts: n/a
 
      08-05-2004
Peter Otten wrote:

> Nigel Rowe wrote:
>
>> Peter Otten wrote:
>>
>>> Andy Baker wrote:
>>>

>


<snip>

>> What work is actually done when the
>> 'nested function creation code of the "inner" function'
>> is executed?


< snip >

>
> In the example you get three extra instructions, LOAD_CONST, MAKE_FUNCTION
> and STORE_FAST (and earn the slight benefit that inner is now a local
> instead of a global). A constant code object is used, meaning compilation
> takes place at most once when the module is loaded. There is a dedicated
> op-code, so function creation should be faster than "normal" creation of a
> class instance.
>
> Now this is all nice and dandy, but how do the two contenders perform?
>
> $ timeit.py -s"def inner(): pass" -s"def outer(): return inner" "outer()"
> 1000000 loops, best of 3: 0.469 usec per loop
> $ timeit.py -s"def outer():" -s" def inner(): pass" -s" return inner"
> "outer()"
> 1000000 loops, best of 3: 1.12 usec per loop
>
> i. e. nesting the two functions roughly doubles execution time.
> However, creation of an inner function often will only take a small
> fraction of the total time spent in the outer function - in the end it's
> just a matter of style.
>
> I use inner functions only when they depend on the local context because I
> think it nicely discriminates closures from helpers. I tend to omit
> implicit parameters even for helper funtions, therefore nesting them would
> gain me nothing.
>
> Peter


Thanks Peter, I didn't know about timeit.py (it's now on my $PATH).

I don't think I'm going to worry about the overhead (except in deeply nested
loops). From some quick testing it looks like the overhead is about the
same as 1.5 times
a=b[10:20] # where b="the quick brown jox jumps over the lazy dog"
and less than 1/2 of
a=math.sin(0)

So if the nested code is easier to read, in it goes.
--
Nigel Rowe
A pox upon the spammers that make me write my address like..
rho (snail) swiftdsl (stop) com (stop) au
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Conceptual question: Reasoning behind private subnetting scheme? John H. Cisco 3 03-10-2010 04:14 PM
Michael Reichmann reasoning for AA filters? W Digital Photography 29 10-22-2007 09:10 AM
what's wrong with my popen reasoning? Rick Spencer Python 1 02-05-2006 10:56 PM
What is the reasoning behind Windows "installing" programs and the way Mac does? Marc Computer Support 8 05-06-2005 09:00 PM
Reasoning for load-external-dtd Xerces default setting? blu4899 XML 4 10-31-2003 08:53 PM



Advertisments