Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > doctests and decorators

Reply
Thread Tools

doctests and decorators

 
 
Eric Snow
Guest
Posts: n/a
 
      06-16-2009
Apparently there is a known issue with doctests, in which tests in
functions using externally defined decorators are ignored. The
recommended fix is to update the order of checks in the _from_module
method of DocTestFinder in the doctest module. The bug and fix are
discussed at the following URLs (and several places in this group):

http://bugs.python.org/issue1108
http://mail.python.org/pipermail/pyt...er/627866.html

The fix implies that the inpect.getmodule function will find the
module of the function object and not of the decorator. However, in
2.4 the inspect.getmodule function returns the module of the
decorator. I have subsequently tested this in 2.5 and 2.6, and it
also returns the module of the decorator. As such, the fix for
doctests does not work in my tests. Below is the test code that I
used:

<EXAMPLE>

test1.py
++++++++++++++++++++++++++++
def decorator(function):
def new_function(*args, **kwargs):
return function(*args, **kwargs)
return new_function

test2.py
++++++++++++++++++++++++++++
import test1
import inspect

class Test(object):
@test1.decorator
def test2(self): pass

def run_tests():
test = Test()
test.test2()

print("Test is class, test is instance, test2 is method of Test
(has decorator)")
print("test's module: %s" % inspect.getmodule(test))
print("Test's module: %s" % inspect.getmodule(Test))
print("test.test2's module: %s" % inspect.getmodule
(test.test2))
print("Test.test2's module: %s" % inspect.getmodule
(Test.test2))
print("test.test2's func_name: %s" % test.test2.func_name)
print("Test.test2's func_name: %s" % Test.test2.func_name)

if __name__ == "__main__":
run_tests()

</EXAMPLE>

Here is the output that I got in 2.4, 2.5, and 2.6:

Test is class, test is instance, test2 is method of Test (has
decorator)
test's module: <module '__main__' from 'test2.py'>
Test's module: <module '__main__' from 'test2.py'>
test.test2's module: <module 'test1' from '/tmp/test1.py'>
Test.test2's module: <module 'test1' from '/tmp/test1.py'>
test.test2's func_name: new_function
Test.test2's func_name: new_function

If things were working right, then the module for test.test2 would be
the same as the module for test. I must be missing something, as the
referenced discussion suggests a simple conclusion. Any ideas?

-eric
 
Reply With Quote
 
 
 
 
Eric Snow
Guest
Posts: n/a
 
      06-16-2009
On Jun 16, 9:59*am, Eric Snow <(E-Mail Removed)> wrote:
> Apparently there is a known issue with doctests, in which tests in
> functions using externally defined decorators are ignored. *The
> recommended fix is to update the order of checks in the _from_module
> method of DocTestFinder in the doctest module. *The bug and fix are
> discussed at the following URLs (and several places in this group):
>
> http://bugs.python.org/issue1108http...er/627866.html
>
> The fix implies that the inpect.getmodule function will find the
> module of the function object and not of the decorator. *However, in
> 2.4 the inspect.getmodule function returns the module of the
> decorator. *I have subsequently tested this in 2.5 and 2.6, and it
> also returns the module of the decorator. *As such, the fix for
> doctests does not work in my tests. *Below is the test code that I
> used:
>
> <EXAMPLE>
>
> test1.py
> ++++++++++++++++++++++++++++
> def decorator(function):
> * * def new_function(*args, **kwargs):
> * * * * return function(*args, **kwargs)
> * * return new_function
>
> test2.py
> ++++++++++++++++++++++++++++
> import test1
> import inspect
>
> class Test(object):
> * * @test1.decorator
> * * def test2(self): pass
>
> def run_tests():
> * * test = Test()
> * * test.test2()
>
> * * print("Test is class, test is instance, test2 is method of Test
> (has decorator)")
> * * print("test's module: * * * * *%s" % inspect.getmodule(test))
> * * print("Test's module: * * * * *%s" % inspect.getmodule(Test))
> * * print("test.test2's module: * *%s" % inspect.getmodule
> (test.test2))
> * * print("Test.test2's module: * *%s" % inspect.getmodule
> (Test.test2))
> * * print("test.test2's func_name: %s" % test.test2.func_name)
> * * print("Test.test2's func_name: %s" % Test.test2.func_name)
>
> if __name__ == "__main__":
> * * run_tests()
>
> </EXAMPLE>
>
> Here is the output that I got in 2.4, 2.5, and 2.6:
>
> Test is class, test is instance, test2 is method of Test (has
> decorator)
> test's module: * * * * *<module '__main__' from 'test2.py'>
> Test's module: * * * * *<module '__main__' from 'test2.py'>
> test.test2's module: * *<module 'test1' from '/tmp/test1.py'>
> Test.test2's module: * *<module 'test1' from '/tmp/test1.py'>
> test.test2's func_name: new_function
> Test.test2's func_name: new_function
>
> If things were working right, then the module for test.test2 would be
> the same as the module for test. *I must be missing something, as the
> referenced discussion suggests a simple conclusion. *Any ideas?
>
> -eric


One work-around I found is the following change in example:

<EXAMPLE>

test1.py
++++++++++++++++++++++++++++
def decorator(function):
def new_function(*args, **kwargs):
return function(*args, **kwargs)
new_function.__module__ = function.__module__
new_function.__doc__ = function.__doc__
new_function.__name__ = function.__name__
return new_function

</EXAMPLE>

However, this seems pretty lame. The doctest module should be able to
figure out that the docstring belongs is there in the module.

-eric
 
Reply With Quote
 
 
 
 
Eric Snow
Guest
Posts: n/a
 
      06-16-2009
On Jun 16, 10:31*am, Christian Heimes <(E-Mail Removed)> wrote:
> Eric Snow schrieb:
>
>
>
> > Apparently there is a known issue with doctests, in which tests in
> > functions using externally defined decorators are ignored. *The
> > recommended fix is to update the order of checks in the _from_module
> > method of DocTestFinder in the doctest module. *The bug and fix are
> > discussed at the following URLs (and several places in this group):

>
> >http://bugs.python.org/issue1108
> >http://mail.python.org/pipermail/pyt...er/627866.html

>
> > The fix implies that the inpect.getmodule function will find the
> > module of the function object and not of the decorator. *However, in
> > 2.4 the inspect.getmodule function returns the module of the
> > decorator. *I have subsequently tested this in 2.5 and 2.6, and it
> > also returns the module of the decorator. *As such, the fix for
> > doctests does not work in my tests. *Below is the test code that I
> > used:

>
> It's not an issue with the doctest module but with your code. You want
> to use functools.wraps().
>
> http://docs.python.org/library/funct...unctools.wraps
>
> Christian


Unfortunately, I am stuck on 2.4 for now, which does not have the
functools.

-eric
 
Reply With Quote
 
Michele Simionato
Guest
Posts: n/a
 
      06-16-2009
On Jun 16, 6:39*pm, Eric Snow <(E-Mail Removed)> wrote:
> On Jun 16, 10:31*am, Christian Heimes <(E-Mail Removed)> wrote:
>
>
>
> > Eric Snow schrieb:

>
> > > Apparently there is a known issue with doctests, in which tests in
> > > functions using externally defined decorators are ignored. *The
> > > recommended fix is to update the order of checks in the _from_module
> > > method of DocTestFinder in the doctest module. *The bug and fix are
> > > discussed at the following URLs (and several places in this group):

>
> > >http://bugs.python.org/issue1108
> > >http://mail.python.org/pipermail/pyt...er/627866.html

>
> > > The fix implies that the inpect.getmodule function will find the
> > > module of the function object and not of the decorator. *However, in
> > > 2.4 the inspect.getmodule function returns the module of the
> > > decorator. *I have subsequently tested this in 2.5 and 2.6, and it
> > > also returns the module of the decorator. *As such, the fix for
> > > doctests does not work in my tests. *Below is the test code that I
> > > used:

>
> > It's not an issue with the doctest module but with your code. You want
> > to use functools.wraps().

>
> >http://docs.python.org/library/funct...unctools.wraps

>
> > Christian

>
> Unfortunately, I am stuck on 2.4 for now, which does not have the
> functools.
>
> -eric


But you can always use the decorator module: http://pypi.python.org/pypi/decorator
 
Reply With Quote
 
Eric Snow
Guest
Posts: n/a
 
      06-16-2009
On Jun 16, 11:24*am, Michele Simionato <(E-Mail Removed)>
wrote:
> On Jun 16, 6:39*pm, Eric Snow <(E-Mail Removed)> wrote:
>
>
>
> > On Jun 16, 10:31*am, Christian Heimes <(E-Mail Removed)> wrote:

>
> > > Eric Snow schrieb:

>
> > > > Apparently there is a known issue with doctests, in which tests in
> > > > functions using externally defined decorators are ignored. *The
> > > > recommended fix is to update the order of checks in the _from_module
> > > > method of DocTestFinder in the doctest module. *The bug and fix are
> > > > discussed at the following URLs (and several places in this group):

>
> > > >http://bugs.python.org/issue1108
> > > >http://mail.python.org/pipermail/pyt...er/627866.html

>
> > > > The fix implies that the inpect.getmodule function will find the
> > > > module of the function object and not of the decorator. *However, in
> > > > 2.4 the inspect.getmodule function returns the module of the
> > > > decorator. *I have subsequently tested this in 2.5 and 2.6, and it
> > > > also returns the module of the decorator. *As such, the fix for
> > > > doctests does not work in my tests. *Below is the test code that I
> > > > used:

>
> > > It's not an issue with the doctest module but with your code. You want
> > > to use functools.wraps().

>
> > >http://docs.python.org/library/funct...unctools.wraps

>
> > > Christian

>
> > Unfortunately, I am stuck on 2.4 for now, which does not have the
> > functools.

>
> > -eric

>
> But you can always use the decorator module:http://pypi.python.org/pypi/decorator


Thanks to both of you. Very helpful.

So in general should decorators always hide themselves? I am guessing
not, otherwise this would already be part of their behavior. Still,
is it the common case to camouflage the decorator like this? If so, I
would expect it to be the default behavior of decorators.

-eric
 
Reply With Quote
 
Steven D'Aprano
Guest
Posts: n/a
 
      06-17-2009
On Tue, 16 Jun 2009 12:04:32 -0700, Scott David Daniels wrote:

> Eric Snow wrote:
>> In general should decorators always hide themselves? I am guessing
>> not, otherwise this would already be part of their behavior. Still, is
>> it the common case to camouflage the decorator like this? If so, I
>> would expect it to be the default behavior of decorators.

>
> The Python goal is "no magic". So, if you want the stuff wrapped, you
> do it (as the default traceback shows where the code actually goes). It
> would be far more complicated to display the truth if decorators
> defaulted to modifying the builtins, and you had to do magic to remove
> that part of the decoration.



I'm afraid I can't understand what you're saying. What do you consider
"magic"? What's a "default traceback"? What do you mean, "display the
truth"?


> A decorator has _very_ simple semantics,
> while anything that automatically copied attributes would have funny
> semantics indeed for use by funny decorators like:

[...]


functools.wraps() automatically copies attributes:

>>> import functools
>>> def dec(func):

.... @functools.wraps(func)
.... def inner(*args):
.... return func(args) + 1
.... return inner
....
>>> def f(x):

.... return 1
....
>>> f.attr = "Attribute"
>>> f = dec(f)
>>> f(3)

2
>>> f.attr

'Attribute'



--
Steven
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
doctests working both in Python 2.X and Python 3.X Michele Simionato Python 0 04-11-2010 08:29 AM
Doctests and decorated methods don't get along Steven D'Aprano Python 7 02-09-2010 08:40 AM
Running doctests with unittest Thomas Heller Python 1 03-09-2005 09:04 PM
unicode in doctests Bjorn Tillenius Python 0 11-29-2004 06:27 PM
PEP 318 decorators are not Decorators Arien Malec Python 11 08-16-2004 06:38 PM



Advertisments