Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > Using a dict as if it were a module namespace

Reply
Thread Tools

Using a dict as if it were a module namespace

 
 
Steven D'Aprano
Guest
Posts: n/a
 
      01-27-2008
I have a problem which I think could be solved by using a dict as a
namespace, in a similar way that exec and eval do.

When using the timeit module, it is very inconvenient to have to define
functions as strings. A good alternative is to create the function as
normal, and import it:

def myfunc(x, y):
return x+y

timeit.Timer("myfunc(59, 60)", "from __main__ import myfunc").timeit()


Not only is this an easy idiom to follow, but myfunc can live in another
module: just replace __main__ with the module name.

Now, I'm trying to build a suite of tests to use with timeit. I have a
bunch of tests which I've created as dicts:

test_suite= [dict(x=59, y=60), dict(x=-1, y=-2)]

What I *think* I want to do is use the from ... import idiom to grab
arguments from the dicts as if they were modules, but this doesn't work:

expr = "myfunc(x, y)"
for test in test_suite:
setup = "from __main__ import myfunc; from test import x, y"
t = timeit.Timer(expr, setup).timeit()


Even if the Timer could see test, it is not a module and you can't import
from it. Naturally.


Alternatives that I have found:

(1) Import the test and grab the values needed from it:

setup = """from __main__ import myfunc, test
x, y = test['x'], test['y']"""


I don't like this one. It doesn't seem very elegant to me, and it gets
unwieldy as the complexity increases. Every item I need from test has to
be named twice, violating the principle Don't Repeat Yourself. If the
tests change, the setup string has to be explicitly changed also.


(2) Mess with the global namespace:

globals().update(t)
setup = "from __main__ import myfunc"

I don't like this one. It looks hackish, and I worry about conflicts and
side-effects. If it works (and I haven't tested it) it relies on an
implementation detail of timeit.Timer.__init__, namely the line
"exec code in globals(), ns". Worst of all, it pollutes or even mangles
the global namespace of the calling code, not the code being tested.


(3) Explicitly pass a namespace dict to the Timer class, possibly even
getting rid of setup altogether:

test['myfunc'] = myfunc
t = timeit.Timer(expr, '', ns=test).timeit()

This would be the most elegant solution, but at this time it is
completely hypothetical. Timer does not have that functionality.


(4) Dump the test data straight into the setup string:

setup = "from __main__ import myfunc; x = %(x)s; y = %(y)s" % t

Again, unwieldy and against DRY. The additional disadvantage is that
there are many types of test data that can't be converted to and from
strings like that.


What do others think? Have I missed something? What other alternatives
are there?



--
Steven
 
Reply With Quote
 
 
 
 
thebjorn
Guest
Posts: n/a
 
      01-27-2008
On Jan 27, 8:45 am, Steven D'Aprano <st...@REMOVE-THIS-
cybersource.com.au> wrote:
> I have a problem which I think could be solved by using a dict as a
> namespace, in a similar way that exec and eval do.
>
> When using the timeit module, it is very inconvenient to have to define
> functions as strings. A good alternative is to create the function as
> normal, and import it:
>
> def myfunc(x, y):
> return x+y
>
> timeit.Timer("myfunc(59, 60)", "from __main__ import myfunc").timeit()
>
> Not only is this an easy idiom to follow, but myfunc can live in another
> module: just replace __main__ with the module name.
>
> Now, I'm trying to build a suite of tests to use with timeit. I have a
> bunch of tests which I've created as dicts:
>
> test_suite= [dict(x=59, y=60), dict(x=-1, y=-2)]
>
> What I *think* I want to do is use the from ... import idiom to grab
> arguments from the dicts as if they were modules, but this doesn't work:
>
> expr = "myfunc(x, y)"
> for test in test_suite:
> setup = "from __main__ import myfunc; from test import x, y"
> t = timeit.Timer(expr, setup).timeit()
>
> Even if the Timer could see test, it is not a module and you can't import
> from it. Naturally.
>
> Alternatives that I have found:
>
> (1) Import the test and grab the values needed from it:
>
> setup = """from __main__ import myfunc, test
> x, y = test['x'], test['y']"""
>
> I don't like this one. It doesn't seem very elegant to me, and it gets
> unwieldy as the complexity increases. Every item I need from test has to
> be named twice, violating the principle Don't Repeat Yourself. If the
> tests change, the setup string has to be explicitly changed also.
>
> (2) Mess with the global namespace:
>
> globals().update(t)
> setup = "from __main__ import myfunc"
>
> I don't like this one. It looks hackish, and I worry about conflicts and
> side-effects. If it works (and I haven't tested it) it relies on an
> implementation detail of timeit.Timer.__init__, namely the line
> "exec code in globals(), ns". Worst of all, it pollutes or even mangles
> the global namespace of the calling code, not the code being tested.
>
> (3) Explicitly pass a namespace dict to the Timer class, possibly even
> getting rid of setup altogether:
>
> test['myfunc'] = myfunc
> t = timeit.Timer(expr, '', ns=test).timeit()
>
> This would be the most elegant solution, but at this time it is
> completely hypothetical. Timer does not have that functionality.
>
> (4) Dump the test data straight into the setup string:
>
> setup = "from __main__ import myfunc; x = %(x)s; y = %(y)s" % t
>
> Again, unwieldy and against DRY. The additional disadvantage is that
> there are many types of test data that can't be converted to and from
> strings like that.
>
> What do others think? Have I missed something? What other alternatives
> are there?
>
> --
> Steven


You might have lost me, but wouldn't it be easier to do some variation
on this

test_suite = [
'(x=59, y=60)', # either have strings here...
'(x=-1, y=-2)',
]

for test in test_suite:
# ... or convert the dicts to appropriate strings here...
expr = 'myfunc' + test
t = timeit.Timer(expr, 'from __main__ import myfunc').timeit()
...

-- bjorn
 
Reply With Quote
 
 
 
 
Ross Ridge
Guest
Posts: n/a
 
      01-27-2008
Steven D'Aprano writes:
>(1) Import the test and grab the values needed from it:
>
> setup = """from __main__ import myfunc, test
>x, y = test['x'], test['y']"""
>
>
>I don't like this one. It doesn't seem very elegant to me, and it gets
>unwieldy as the complexity increases. Every item I need from test has to
>be named twice, violating the principle Don't Repeat Yourself. If the
>tests change, the setup string has to be explicitly changed also.


I think this is the way to go as it follows the principle of "say what
you mean." You can however simplify it, and repeat yourself less,
by using the extended call syntax:

expr = "myfunc(**test)"
setup = """from __main__ import myfunc, test"""

....
>I don't like this one. It looks hackish, and I worry about conflicts and
>side-effects. If it works (and I haven't tested it) it relies on an
>implementation detail of timeit.Timer.__init__, namely the line
>"exec code in globals(), ns". Worst of all, it pollutes or even mangles
>the global namespace of the calling code, not the code being tested.


It wouldn't work because the timeit module's "globals" are different
from the __main__ module's globals.

Ross Ridge

--
l/ // Ross Ridge -- The Great HTMU
[oo][oo] http://www.velocityreviews.com/forums/(E-Mail Removed)
-()-/()/ http://www.csclub.uwaterloo.ca/~rridge/
db //
 
Reply With Quote
 
Steven Bethard
Guest
Posts: n/a
 
      01-27-2008
Steven D'Aprano wrote:
> I have a problem which I think could be solved by using a dict as a
> namespace, in a similar way that exec and eval do.
>
> When using the timeit module, it is very inconvenient to have to define
> functions as strings. A good alternative is to create the function as
> normal, and import it:
>
> def myfunc(x, y):
> return x+y
>
> timeit.Timer("myfunc(59, 60)", "from __main__ import myfunc").timeit()
>
>
> Not only is this an easy idiom to follow, but myfunc can live in another
> module: just replace __main__ with the module name.
>
> Now, I'm trying to build a suite of tests to use with timeit. I have a
> bunch of tests which I've created as dicts:
>
> test_suite= [dict(x=59, y=60), dict(x=-1, y=-2)]
>
> What I *think* I want to do is use the from ... import idiom to grab
> arguments from the dicts as if they were modules, but this doesn't work:
>
> expr = "myfunc(x, y)"
> for test in test_suite:
> setup = "from __main__ import myfunc; from test import x, y"
> t = timeit.Timer(expr, setup).timeit()

[snip]
> (2) Mess with the global namespace:
>
> globals().update(t)
> setup = "from __main__ import myfunc"


Why not mess with the namespace inside the setup code? E.g.::

>>> test_suite = [dict(x=59, y=60), dict(x=-1, y=-2)]
>>> expr = "myfunc(x, y)"
>>> for test in test_suite:

... setup = textwrap.dedent('''
... from __main__ import myfunc, test
... globals().update(test)''')
... t = timeit.Timer(expr, setup).timeit()
...

That shouldn't pollute your actual namespace::

>>> x

Traceback (most recent call last):
File "<interactive input>", line 1, in <module>
NameError: name 'x' is not defined
>>> y

Traceback (most recent call last):
File "<interactive input>", line 1, in <module>
NameError: name 'y' is not defined

STeVe
 
Reply With Quote
 
Steven D'Aprano
Guest
Posts: n/a
 
      01-28-2008
On Sun, 27 Jan 2008 11:16:00 -0700, Steven Bethard wrote:

> Why not mess with the namespace inside the setup code? E.g.::

[snip]

Ah, thanks, good thinking. It's not perfect, but it should be close
enough for what I'm trying to do.


--
Steven
who still wishes you could explicitly pass a namespace to timeit.Timer...

 
Reply With Quote
 
Steven D'Aprano
Guest
Posts: n/a
 
      01-28-2008
On Sun, 27 Jan 2008 10:04:06 -0500, Ross Ridge wrote:

> I think this is the way to go as it follows the principle of "say what
> you mean." You can however simplify it, and repeat yourself less, by
> using the extended call syntax:
>
> expr = "myfunc(**test)"
> setup = """from __main__ import myfunc, test"""


Ah, also good thinking! I'd have to modify it, because the tests carry
other information which shouldn't be passed to the test function, but
that's a simple modification.

Thanks to all who replied.


--
Steven
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
dict.items() vs dict.iteritems and similar questions Drew Python 19 03-15-2007 09:23 PM
dict.has_key(x) versus 'x in dict' Paul Melis Python 48 12-15-2006 05:55 PM
Inconsistency in dictionary behaviour: dict(dict) not calling __setitem__ Almad Python 8 12-14-2006 07:37 PM
dict!ident as equivalent of dict["ident"] Alexander Kozlovsky Python 5 05-22-2006 08:06 AM
Re: dict->XML->dict? Or, passing small hashes through text? Skip Montanaro Python 0 08-15-2003 03:46 PM



Advertisments