Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > Decorating functions without losing their signatures

Reply
Thread Tools

Decorating functions without losing their signatures

 
 
Rotwang
Guest
Posts: n/a
 
      04-03-2013
Hi all,

Here's a Python problem I've come up against and my crappy solution.
Hopefully someone here can suggest something better. I want to decorate
a bunch of functions with different signatures; for example, I might
want to add some keyword-only arguments to all functions that return
instances of a particular class so that the caller can create instances
with additional attributes. So I do something like this:

import functools

def mydecorator(f):
@functools.wraps(f)
def wrapped(*args, attribute = None, **kwargs):
result = f(*args, **kwargs)
result.attribute = attribute
return result
return wrapped

@mydecorator
def f(x, y = 1, *a, z = 2, **k):
return something

The problem with this is, when I subsequently type 'f(' in IDLE, the
signature prompt that appears is not very useful; it looks like this:

(*args, attribute=None, **kwargs)

whereas I'd like it to look like this:

(x, y=1, *a, z=2, attribute=None, **k)


After thinking about it for a while I've come up with the following
abomination:

import inspect

def sigwrapper(sig):
if not isinstance(sig, inspect.Signature):
sig = inspect.signature(sig)
def wrapper(f):
ps = 'args = []\n\t\t'
ks = 'kwargs = {}\n\t\t'
for p in sig.parameters.values():
if p.kind in (p.POSITIONAL_ONLY, p.POSITIONAL_OR_KEYWORD):
ps = '%sargs.append(%s)\n\t\t' % (ps, p.name)
elif p.kind == p.VAR_POSITIONAL:
ps = '%sargs.extend(%s)\n\t\t' % (ps, p.name)
elif p.kind == p.KEYWORD_ONLY:
ks = '%skwargs[%r] = %s\n\t\t' % (ks, p.name, p.name)
elif p.kind == p.VAR_KEYWORD:
ks = '%skwargs.update(%s)\n\t\t' % (ks, p.name)
loc = {'wrapped': f}
defstring = ('def wrapouter(wrapped = wrapped):'
'\n\tdef wrapinner%s:'
'\n\t\t%s%sreturn wrapped(*args, **kwargs)'
'\n\treturn wrapinner' % (sig, ps, ks))
exec(defstring, f.__globals__, loc)
return loc['wrapouter']()
return wrapper

The function sigwrapper() may be passed an inspect.Signature object sig
(or function, if that function has the right signature) and returns a
decorator that gives any function the signature sig. I can then replace
my original decorator with something like

def mydecorator(f):
sig = inspect.signature(f)
sig = do_something(sig) # add an additional kw-only argument to sig
@functools.wraps(f)
@sigwrapper(sig)
def wrapped(*args, attribute = None, **kwargs):
result = f(*args, **kwargs)
result.attribute = attribute
return result
return wrapped

It seems to work, but I don't like it. Does anyone know of a better way
of doing the same thing?
 
Reply With Quote
 
 
 
 
Steven D'Aprano
Guest
Posts: n/a
 
      04-03-2013
On Wed, 03 Apr 2013 02:05:31 +0100, Rotwang wrote:

> Hi all,
>
> Here's a Python problem I've come up against and my crappy solution.
> Hopefully someone here can suggest something better. I want to decorate
> a bunch of functions with different signatures;

[...]
> After thinking about it for a while I've come up with the following
> abomination:

[...]
> It seems to work, but I don't like it. Does anyone know of a better way
> of doing the same thing?



Wait until Python 3.4 or 3.5 (or Python 4000?) when functools.wraps
automatically preserves the function signature?

Alas, I think this is a hard problem to solve with current Python. You
might like to compare your solution with that of Michele Simionato's
"decorator" module:

http://micheles.googlecode.com/hg/de...mentation.html


See this for some other ideas:

http://numericalrecipes.wordpress.co...re-preserving-
function-decorators/



Good luck!




--
Steven
 
Reply With Quote
 
 
 
 
Jan Riechers
Guest
Posts: n/a
 
      04-03-2013
On 03.04.2013 04:05, Rotwang wrote:
> Hi all,
>
> Here's a Python problem I've come up against and my crappy solution.
> Hopefully someone here can suggest something better. I want to decorate
> a bunch of functions with different signatures; for example, I might
> want to add some keyword-only arguments to all functions that return
> instances of a particular class so that the caller can create instances
> with additional attributes. So I do something like this:

[...]
> It seems to work, but I don't like it. Does anyone know of a better way
> of doing the same thing?


Hi,

I think you might want to check out that Pycon2013 Video about Metaclass
Prgoramming of David Beazley:
http://www.pyvideo.org/video/1716/py...etaprogramming

He explains how to passing attributes, such creating custom classes on
demand and returning there signatures even when wrapped.

I think that was what you wanted to archive?

Regards
Jan
 
Reply With Quote
 
Michele Simionato
Guest
Posts: n/a
 
      04-04-2013
On Wednesday, April 3, 2013 3:05:31 AM UTC+2, Rotwang wrote:
> After thinking about it for a while I've come up with the following
>
> abomination



Alas, there is actually no good way to implement this feature in pure Python without abominations. Internally the decorator module does something similar to what you are doing. However, instead of cooking up yourself your custom solution, it is probably better if you stick to the decorator module which has been used in production for several years and has hundreds of thousands of downloads. I am not claiming that it is bug free, but it is stable,bug reports come very rarely and it works for all versions of Python from 2.5 to 3.3.
 
Reply With Quote
 
Rotwang
Guest
Posts: n/a
 
      04-04-2013
On 03/04/2013 02:05, Rotwang wrote:
> [...]
>
> After thinking about it for a while I've come up with the following
> abomination:
>
> import inspect
>
> def sigwrapper(sig):
> if not isinstance(sig, inspect.Signature):
> sig = inspect.signature(sig)
> def wrapper(f):
> ps = 'args = []\n\t\t'
> ks = 'kwargs = {}\n\t\t'
> for p in sig.parameters.values():
> if p.kind in (p.POSITIONAL_ONLY, p.POSITIONAL_OR_KEYWORD):
> ps = '%sargs.append(%s)\n\t\t' % (ps, p.name)
> elif p.kind == p.VAR_POSITIONAL:
> ps = '%sargs.extend(%s)\n\t\t' % (ps, p.name)
> elif p.kind == p.KEYWORD_ONLY:
> ks = '%skwargs[%r] = %s\n\t\t' % (ks, p.name, p.name)
> elif p.kind == p.VAR_KEYWORD:
> ks = '%skwargs.update(%s)\n\t\t' % (ks, p.name)
> loc = {'wrapped': f}
> defstring = ('def wrapouter(wrapped = wrapped):'
> '\n\tdef wrapinner%s:'
> '\n\t\t%s%sreturn wrapped(*args, **kwargs)'
> '\n\treturn wrapinner' % (sig, ps, ks))
> exec(defstring, f.__globals__, loc)
> return loc['wrapouter']()
> return wrapper


Oops! Earlier I found out the hard way that this fails when the
decorated function has arguments called 'args' or 'kwargs'. Here's a
modified version that fixes said bug, but presumably not the many others
I haven't noticed yet:

def sigwrapper(sig):
if not isinstance(sig, inspect.Signature):
sig = inspect.signature(sig)
n = 0
while True:
pn = 'p_%i' % n
kn = 'k_%i' % n
if pn not in sig.parameters and kn not in sig.parameters:
break
n += 1
ps = '%s = []\n\t\t' % pn
ks = '%s = {}\n\t\t' % kn
for p in sig.parameters.values():
if p.kind in (p.POSITIONAL_ONLY, p.POSITIONAL_OR_KEYWORD):
ps = '%s%s.append(%s)\n\t\t' % (ps, pn, p.name)
elif p.kind == p.VAR_POSITIONAL:
ps = '%s%s.extend(%s)\n\t\t' % (ps, pn, p.name)
elif p.kind == p.KEYWORD_ONLY:
ks = '%s%s[%r] = %s\n\t\t' % (ks, kn, p.name, p.name)
elif p.kind == p.VAR_KEYWORD:
ks = '%s%s.update(%s)\n\t\t' % (ks, kn, p.name)
defstring = ('def wrapouter(wrapped = wrapped):'
'\n\tdef wrapinner%s:'
'\n\t\t%s%sreturn wrapped(*%s, **%s)'
'\n\treturn wrapinner' % (sig, ps, ks, pn, kn))
def wrapper(f):
loc = {'wrapped': f}
exec(defstring, f.__globals__, loc)
return loc['wrapouter']()
return wrapper
 
Reply With Quote
 
Rotwang
Guest
Posts: n/a
 
      04-04-2013
On 03/04/2013 05:15, Steven D'Aprano wrote:
> On Wed, 03 Apr 2013 02:05:31 +0100, Rotwang wrote:
>
>> Hi all,
>>
>> Here's a Python problem I've come up against and my crappy solution.
>> Hopefully someone here can suggest something better. I want to decorate
>> a bunch of functions with different signatures;

> [...]
>> After thinking about it for a while I've come up with the following
>> abomination:

> [...]
>> It seems to work, but I don't like it. Does anyone know of a better way
>> of doing the same thing?

>
>
> Wait until Python 3.4 or 3.5 (or Python 4000?) when functools.wraps
> automatically preserves the function signature?
>
> Alas, I think this is a hard problem to solve with current Python. You
> might like to compare your solution with that of Michele Simionato's
> "decorator" module:
>
> http://micheles.googlecode.com/hg/de...mentation.html
>
>
> See this for some other ideas:
>
> http://numericalrecipes.wordpress.co...re-preserving-
> function-decorators/
>
>
>
> Good luck!


Thanks. It'll take me a while to fully absorb the links, but it looks
like both are similarly based on abusing the exec function.

Thanks to Jan too.
 
Reply With Quote
 
Rotwang
Guest
Posts: n/a
 
      04-04-2013
On 04/04/2013 02:18, Michele Simionato wrote:
> On Wednesday, April 3, 2013 3:05:31 AM UTC+2, Rotwang wrote:
>> After thinking about it for a while I've come up with the following
>>
>> abomination

>
> Alas, there is actually no good way to implement this feature in pure
> Python without abominations. Internally the decorator module does
> something similar to what you are doing. However, instead of cooking up
> yourself your custom solution, it is probably better if you stick to
> the decorator module which has been used in production for several
> years and has hundreds of thousands of downloads. I am not claiming
> that it is bug free, but it is stable, bug reports come very rarely and
> it works for all versions of Python from 2.5 to 3.3.


Thanks, I'll check it out. Looking at the link Steven provided, I didn't
see an easy way to add additional keyword-only arguments to a function's
signature, though (but then I've yet to figure out how the FunctionMaker
class works).

 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
decorating functions with generic signatures (not for the faint of heart) Michele Simionato Python 4 06-05-2012 05:31 AM
Decorating class member functions Andy Terrel Python 12 05-04-2007 05:19 PM
Decorating div boxes with CSS Paul B HTML 10 04-14-2006 04:25 PM
why doen't sun mycrosystem provide signatures of their main dev files? onetitfemme Java 4 10-18-2005 12:52 AM
decorating classes with metaclass Bengt Richter Python 1 03-15-2005 02:36 AM



Advertisments