«

For a project I'm working on I need a way to retrieve the source
code of dynamically generated Python functions. (These functions
are implemented dynamically in order to simulate "partial application"
in Python.[1]) The ultimate goal is to preserve a textual record
of transformations performed on data, along with all the data (both
pre- and post- transformation) itself.

These transformation functions could be dynamically generated as
closures, but I suspect that this would make it more difficult to
extract source code that could serve as a reasonably self-contained
description of the transformation (because this source code would
refer to variables defined outside of the function). An alternative
would be to generate the *source code* for the functions dynamically,
from a template having slots (within the function's definition)
that gets filled in with actual parameter values, and pass this
source code to exec.

In any case, the problem remains of how to extract the
dynamically-generated function's source code.

One possibility would be to define a Transformation wrapper class
whose __init__ takes the dynamically-generated source code (a
string) as argument, keeps it around as an instance attribute for
future reference, and exec's it to define its __call__ method.

Is this overkill/reinventing the wheel? IOW, does Python already
have a built-in way to achieve this same effect?

(Also, the wrapper scheme above is somewhat loose: it's relatively
easy for the source code instance attribute (as described) to fall
out of sync with the function that actually executes when __call__
runs. Maybe a tighter connection between the obtained source code
and the code that actually executes when __call__ runs is possible.)

I'm aware of the inspect module, but from reading its source code
I gather that it is designed for inspecting source code that is
explicitly written in files, and would not be too good at inspecting
functions that are generated dynamically (i.e. not from source code
explicitly given in a source file--I hope that made sense).

Your comments and suggestions would be much appreciated. Many
thanks in advance!

G

[1] For example, given a base function spam that has the signature
(typeT, typeT, typeT, int, int, int) and three specific integer
values X, Y, and Z, dynamically generate a new function spamXYZ
with signature (typeT, typeT, typeT) such that spamXYZ(A, B, C) is
identical to spam(A, B, C, X, Y, Z), for all possible values of A,
B, C.

On 11/5/2010 9:55 AM gb345 said...

> In any case, the problem remains of how to extract the
> dynamically-generated function's source code.
>

Are you looking for the source code of the dynamically created wrapper
function (effectively the piece that calls the original function) or of
the wrapped function? (the piece that ultimately gets called?)

I'm pretty sure you can't get source from the executable function
(python 2.x anyway), but with your naming scheme, it may be possible to
trace into the wrapped function's source.

Emile

gb345 wrote:

> For a project I'm working on I need a way to retrieve the source
> code of dynamically generated Python functions. (These functions
> are implemented dynamically in order to simulate "partial application"
> in Python.[1]) The ultimate goal is to preserve a textual record
> of transformations performed on data, along with all the data (both
> pre- and post- transformation) itself.

Are you aware of functools.partial?

>>> from functools import partial
>>> def f(a, b, c, x, y, z):
.... return a*x + b*y*y + c*z*z*z
....
>>> fstar = partial(f, x=1, y=2, z=3)
>>> fstar(1, 0, 0)
1
>>> fstar(0, 1, 0)
4
>>> fstar(0, 0, 1)
27
>>> fstar.args, fstar.keywords
((), {'y': 2, 'x': 1, 'z': 3})
>>> fstar.func
<function f at 0x7fc4868e6848>

In <ib1fai$8g3$02$> Peter Otten <__peter__@web.de> writes:

>gb345 wrote:

>> For a project I'm working on I need a way to retrieve the source
>> code of dynamically generated Python functions. (These functions
>> are implemented dynamically in order to simulate "partial application"
>> in Python.[1])

>Are you aware of functools.partial?

>>>> from functools import partial
>>>> def f(a, b, c, x, y, z):
>... return a*x + b*y*y + c*z*z*z
>...
>>>> fstar = partial(f, x=1, y=2, z=3)
>>>> fstar(1, 0, 0)
>1
>>>> fstar(0, 1, 0)
>4
>>>> fstar(0, 0, 1)
>27
>>>> fstar.args, fstar.keywords
>((), {'y': 2, 'x': 1, 'z': 3})
>>>> fstar.func
><function f at 0x7fc4868e6848>

I was not aware of functools.partial. This makes the problem a
lot easier. Thanks!

G

On Nov 5, 5:55*pm, gb345 <gb...@invalid.com> wrote:
> For a project I'm working on I need a way to retrieve the source
> code of dynamically generated Python functions. *(These functions
> are implemented dynamically in order to simulate "partial application"
> in Python.[1]) *The ultimate goal is to preserve a textual record
> of transformations performed on data, along with all the data (both
> pre- and post- transformation) itself.
>
> These transformation functions could be dynamically generated as
> closures, but I suspect that this would make it more difficult to
> extract source code that could serve as a reasonably self-contained
> description of the transformation (because this source code would
> refer to variables defined outside of the function). *An alternative
> would be to generate the *source code* for the functions dynamically,
> from a template having slots (within the function's definition)
> that gets filled in with actual parameter values, and pass this
> source code to exec.
>
> In any case, the problem remains of how to extract the
> dynamically-generated function's source code.
>
> One possibility would be to define a Transformation wrapper class
> whose __init__ takes the dynamically-generated source code (a
> string) as argument, keeps it around as an instance attribute for
> future reference, and exec's it to define its __call__ method.
>
> Is this overkill/reinventing the wheel? *IOW, does Python already
> have a built-in way to achieve this same effect?
>
> (Also, the wrapper scheme above is somewhat loose: it's relatively
> easy for the source code instance attribute (as described) to fall
> out of sync with the function that actually executes when __call__
> runs. *Maybe a tighter connection between the obtained source code
> and the code that actually executes when __call__ runs is possible.)
>
> I'm aware of the inspect module, but from reading its source code
> I gather that it is designed for inspecting source code that is
> explicitly written in files, and would not be too good at inspecting
> functions that are generated dynamically (i.e. not from source code
> explicitly given in a source file--I hope that made sense).
>
> Your comments and suggestions would be much appreciated. *Many
> thanks in advance!
>
> G
>
> [1] For example, given a base function spam that has the signature
> (typeT, typeT, typeT, int, int, int) and three specific integer
> values X, Y, and Z, dynamically generate a new function spamXYZ
> with signature (typeT, typeT, typeT) such that spamXYZ(A, B, C) is
> identical to spam(A, B, C, X, Y, Z), for all possible values of A,
> B, C.

I wondered a lot about this issue when writing my own decorator
module.
You may look at the source code of the FunctionMaker class which tries
to address the problem by storing the source code of the generated
function into an attribute. Using inspect.getsource does not work
(see http://bugs.python.org/issue1764286). Of course if you
functools.partial
is enough to solve your problem use it and be done with it.