Converting functions
 

Hi all,

I'm trying to convert functions - pass a few functions to a converting
function, which change their behaviour and return the changed
functions:

>>> def cfuncs(*funcs):
n = []
for f in funcs:
def ff(*args, **key):
print 'Start!', f.func_name
res = f(*args, **key)
print 'End', f.func_name
return res
n.append(ff)
return n

then I try it using two functions:

>>> def f1():
print 'hello'


>>> def f2(x):
return 2 * x

Eventually:
>>> newfuncs = cfuncs(f1, f2)

I would expect newfuncs to hold changed versions of f1 and f2, but
what is actually contained in newfuncs is twice the changed version of
f2.

That is:

>>> newfuncs[1](100)
Start! f2
End f2
200

which is what I expected,

but:

>>> newfuncs[0]()
Start! f2

Traceback (most recent call last):
File "<pyshell#267>", line 1, in <module>
newfuncs[0]()
File "<pyshell#261>", line 6, in ff
res = f(*args, **key)
TypeError: f2() takes exactly 1 argument (0 given)

which is not.

I'll appreciate your help in pointing out the mistake in defining
cfuncs and how to fix it.
Thank you very much!

Re: Converting functions
 

iu2 wrote:

> I'm trying to convert functions - pass a few functions to a converting
> function, which change their behaviour and return the changed
> functions:
>
> >>> def cfuncs(*funcs):
> n = []
> for f in funcs:
> def ff(*args, **key):
> print 'Start!', f.func_name
> res = f(*args, **key)
> print 'End', f.func_name
> return res
> n.append(ff)
> return n
>
> then I try it using two functions:
>
> >>> def f1():
> print 'hello'
>
>
> >>> def f2(x):
> return 2 * x
>
> Eventually:
> >>> newfuncs = cfuncs(f1, f2)
>
> I would expect newfuncs to hold changed versions of f1 and f2, but
> what is actually contained in newfuncs is twice the changed version of
> f2.

That is because the inner ff() references f which is a local variable of
cfuncs(). By the time you invoke your newly created functions cfuncs() and
thus the 'for f in funcs' loop has finished and the value of f is that of
the last item in the funcs tuple. You can avoid the problem with another
indirection

def make_ff(f):
def ff(*args, **key):
print 'Start!', f.func_name
res = f(*args, **key)
print 'End', f.func_name
return res
return ff

def cfuncs(*funcs):
return [make_ff(f) for f in funcs]

Peter

Re: Converting functions
 

On Jan 24, 9:51*am, Peter Otten <__pete...@web.de> wrote:
> iu2 wrote:
> > I'm trying to convert functions - pass a few functions to a converting
> > function, which change their behaviour and return the changed
> > functions:
>
> > >>> def cfuncs(*funcs):
> > * * * * n = []
> > * * * * for f in funcs:
> > * * * * * * * * def ff(*args, **key):
> > * * * * * * * * * * * * print 'Start!', f.func_name
> > * * * * * * * * * * * * res = f(*args, **key)
> > * * * * * * * * * * * * print 'End', f.func_name
> > * * * * * * * * * * * * return res
> > * * * * * * * * n.append(ff)
> > * * * * return n
>
> > then I try it using two functions:
>
> > >>> def f1():
> > * * * * print 'hello'
>
> > >>> def f2(x):
> > * * * * return 2 * x
>
> > Eventually:
> > >>> newfuncs = cfuncs(f1, f2)
>
> > I would expect newfuncs to hold changed versions of f1 and f2, but
> > what is actually contained in newfuncs is twice the changed version of
> > f2.
>
> That is because the inner ff() references f which is a local variable of
> cfuncs(). By the time you invoke your newly created functions cfuncs() and
> thus the 'for f in funcs' loop has finished and the value of f is that of
> the last item in the funcs tuple. You can avoid the problem with another
> indirection
>
> def make_ff(f):
> * * def ff(*args, **key):
> * * * * print 'Start!', f.func_name
> * * * * res = f(*args, **key)
> * * * * print 'End', f.func_name
> * * * * return res
> * * return ff * * * *
>
> def cfuncs(*funcs):
> * * return [make_ff(f) for f in funcs]
>
> Peter- Hide quoted text -
>
> - Show quoted text -

Thanks!

I thought a function definition creates a closure around all used
vars.
As I understand now only variables that are passed as function
arguments can participate in a closure.

Re: Converting functions
 

iu2 wrote:

> I thought a function definition creates a closure around all used
> vars.
> As I understand now only variables that are passed as function
> arguments can participate in a closure.

No, it's just that all closures see the value of a variable at the time when
the closure is run, not when it's defined.

I don't know how to express it more clearly, so here's another example:

>>> def f():
.... def g(): return a * a
.... def h(): return a + a
.... a = 5
.... return g, h
....
>>> g, h = f()
>>> g(), h()
(25, 10)

As you can see the local variable is not yet set when g() and h() are
defined; but only the value by the time they are invoked matters.

Here's a more involved generator version where the value may change between
invocations:

>>> def f(items):
.... def g(): return a * a
.... def h(): return a + a
.... yield g, h
.... for a in items:
.... yield a
....
>>> ff = f([2,3,4])
>>> g, h = next(ff)
>>> g()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 2, in g
NameError: free variable 'a' referenced before assignment in enclosing scope
>>> next(ff)
2
>>> g(), h()
(4, 4)
>>> next(ff)
3
>>> g(), h()
(9, 6)

I think this behaviour is also called "late binding".

Peter

Re: Converting functions
 

Peter Otten <__peter__@web.de> writes:

> I don't know how to express it more clearly, so here's another example:
>
>>>> def f():
> ... def g(): return a * a
> ... def h(): return a + a
> ... a = 5
> ... return g, h
> ...
>>>> g, h = f()
>>>> g(), h()
> (25, 10)

IMHO this whole confusion just shows that mingling assignment and
binding makes understanding scope harder. In Python, the first
assignment inside a function body also creates a binding (unless told
not to do so by global) the scope of which is the _whole_ of the
function body. A variable reference refers to the lexically innermost
surrounding binding of said variable. Now, while it might seem that some
magic happens in the example on the return of the function, this is in
fact not so, since the assignment "a = 5" actually creates a binding for
/a/ that is visible from the body of /g/, because the lexical scope of
the binding is the whole body of /f/, so that the capture of the
variable happens inside of the def expression (as one would expect) and
not on return as you seem to imply.

Slightly OT -- the above also explains why closed over variables are
read only in Python. An assignment inside a closure would implicitly
create a binding, so that all (even previous) references to that
variable would refer to this new binding.

> I think this behaviour is also called "late binding".

"Retroactive implicit scope" would be closer.

--
A change in perspective is worth 80 IQ points. --- Alan Kay

Re: Converting functions
 

Edmunds Cers wrote:

> Peter Otten <__peter__@web.de> writes:
>
>> I don't know how to express it more clearly, so here's another example:
>>
>>>>> def f():
>> ... def g(): return a * a
>> ... def h(): return a + a
>> ... a = 5
>> ... return g, h
>> ...
>>>>> g, h = f()
>>>>> g(), h()
>> (25, 10)
>
> IMHO this whole confusion just shows that mingling assignment and
> binding makes understanding scope harder. In Python, the first
> assignment inside a function body also creates a binding (unless told
> not to do so by global) the scope of which is the _whole_ of the
> function body. A variable reference refers to the lexically innermost
> surrounding binding of said variable. Now, while it might seem that some
> magic happens in the example on the return of the function, this is in
> fact not so, since the assignment "a = 5" actually creates a binding for
> /a/ that is visible from the body of /g/, because the lexical scope of
> the binding is the whole body of /f/, so that the capture of the
> variable happens inside of the def expression (as one would expect) and
> not on return as you seem to imply.
>
> Slightly OT -- the above also explains why closed over variables are
> read only in Python. An assignment inside a closure would implicitly
> create a binding, so that all (even previous) references to that
> variable would refer to this new binding.

Well, in Python 3 they no longer are, courtesy of the nonlocal statement:

>>> def f():
.... def set(x):
.... nonlocal a
.... a = x
.... def get():
.... return a
.... return get, set
.... a = 42
....
>>> get, set = f()
>>> get()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 6, in get
NameError: free variable 'a' referenced before assignment in enclosing scope
>>> set(42)
>>> get()
42

That closed-over variables are read-only by default is just to avoid the
ambiguity about the scope they are supposed to live in, just like global
variables that are to be changed from within a function.

Peter