Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > surprising behaviour of global dictionaries

Reply
Thread Tools

surprising behaviour of global dictionaries

 
 
Michele Simionato
Guest
Posts: n/a
 
      10-09-2012
I have the following module implementing a registry of functions with a decorator:

$ cat x.py
registry = {} # global dictionary

def dec(func):
registry[func.__name__] = func
print registry, id(registry)
return func

if __name__ == '__main__':
import xlib
print registry, id(registry)

The library xlib just defines two dummy functions:

$ cat xlib.py
from x import dec

@dec
def f1():
pass

@dec
def f2():
pass

Then I get the following output:

$ python x.py
{'f1': <function f1 at 0x7f7bce0cd668>} 27920352
{'f1': <function f1 at 0x7f7bce0cd668>, 'f2': <function f2 at 0x7f7bce0cd6e0>} 27920352
{} 27395472

This is surprising since I would expect to have a single global dictionary, not two: how comes the registry inside the ``if __name__ == '__main__'`` block is different from the one seen in the library?

This is python 2.7.3 on Ubuntu.
 
Reply With Quote
 
 
 
 
Peter Otten
Guest
Posts: n/a
 
      10-09-2012
Michele Simionato wrote:

> I have the following module implementing a registry of functions with a
> decorator:
>
> $ cat x.py
> registry = {} # global dictionary
>
> def dec(func):
> registry[func.__name__] = func
> print registry, id(registry)
> return func
>
> if __name__ == '__main__':
> import xlib
> print registry, id(registry)
>
> The library xlib just defines two dummy functions:
>
> $ cat xlib.py
> from x import dec
>
> @dec
> def f1():
> pass
>
> @dec
> def f2():
> pass
>
> Then I get the following output:
>
> $ python x.py
> {'f1': <function f1 at 0x7f7bce0cd668>} 27920352
> {'f1': <function f1 at 0x7f7bce0cd668>, 'f2': <function f2 at
> {0x7f7bce0cd6e0>} 27920352 } 27395472
>
> This is surprising since I would expect to have a single global
> dictionary, not two: how comes the registry inside the ``if __name__ ==
> '__main__'`` block is different from the one seen in the library?
>
> This is python 2.7.3 on Ubuntu.


Welcome to python -- this is a trap every newbie falls into

Seriously, you shouldn't use the main script as a library; it is put into
the sys.modules cache under the "__main__" key. Subsequent imports under its
real name will not find that name in the cache and import another instance
of the module, with puzzling effects, like

$ cat x.py
import x
class A: pass
a = A()
assert isinstance(a, x.A)

$ python x.py
Traceback (most recent call last):
File "x.py", line 4, in <module>
assert isinstance(a, x.A)
AssertionError
$


 
Reply With Quote
 
 
 
 
Michele Simionato
Guest
Posts: n/a
 
      10-09-2012
On Tuesday, October 9, 2012 5:24:17 PM UTC+2, Peter Otten wrote:
> Seriously, you shouldn't use the main script as a library; it is put into
>
> the sys.modules cache under the "__main__" key. Subsequent imports under its
>
> real name will not find that name in the cache and import another instance
>
> of the module, with puzzling effects


Actually I usually never use the main script as a library, this is why I never experience this puzzling behavior before. But now it is clear, thanks.
 
Reply With Quote
 
Michele Simionato
Guest
Posts: n/a
 
      10-09-2012
On Tuesday, October 9, 2012 5:24:17 PM UTC+2, Peter Otten wrote:
> Seriously, you shouldn't use the main script as a library; it is put into
>
> the sys.modules cache under the "__main__" key. Subsequent imports under its
>
> real name will not find that name in the cache and import another instance
>
> of the module, with puzzling effects


Actually I usually never use the main script as a library, this is why I never experience this puzzling behavior before. But now it is clear, thanks.
 
Reply With Quote
 
Grant Edwards
Guest
Posts: n/a
 
      10-09-2012
On 2012-10-09, Peter Otten <(E-Mail Removed)> wrote:

> Welcome to python -- this is a trap every newbie falls into
>
> Seriously, you shouldn't use the main script as a library;


There must be something wrong with me. It never even occurred to me
to try to import a file from within that same file. I don't think
I've ever even heard of that before...

--
Grant Edwards grant.b.edwards Yow! The Korean War must
at have been fun.
gmail.com
 
Reply With Quote
 
Dave Angel
Guest
Posts: n/a
 
      10-09-2012
On 10/09/2012 11:36 AM, Michele Simionato wrote:
> On Tuesday, October 9, 2012 5:24:17 PM UTC+2, Peter Otten wrote:
>> Seriously, you shouldn't use the main script as a library; it is put into
>>
>> the sys.modules cache under the "__main__" key. Subsequent imports under its
>>
>> real name will not find that name in the cache and import another instance
>>
>> of the module, with puzzling effects

> Actually I usually never use the main script as a library, this is why I never experience this puzzling behavior before. But now it is clear, thanks.

More generally, you should arrange your imports so that there are no
cycles. If module/script "a" imports "b", directly or indirectly, "b"
should not try to import "a." Move the common code to a third place,
and import it from both "a" and from "b".

The other symptoms you can get are more subtle than this one, but just
as surprising.


--

DaveA

 
Reply With Quote
 
Peter Otten
Guest
Posts: n/a
 
      10-09-2012
Grant Edwards wrote:

> On 2012-10-09, Peter Otten <(E-Mail Removed)> wrote:
>
>> Welcome to python -- this is a trap every newbie falls into
>>
>> Seriously, you shouldn't use the main script as a library;

>
> There must be something wrong with me. It never even occurred to me
> to try to import a file from within that same file.


It is typically done in two steps:

(1) Write module x, use it in module y.
(2) For convenience add "if __name__ == '__main__'" and import module y.

Hilarity ensues.

> I don't think I've ever even heard of that before...


As I was poking fun at Michele who really is an expert and likely knows more
about Python than I do I may have exaggerated a bit

But it does come up, see

"Singleton implementation problems"
http://mail.python.org/pipermail/pyt...ly/470770.html

"Dynamically declared shared constant/variable imported twice problem"
http://mail.python.org/pipermail/pyt...ay/535619.html

There are more, but I don't know a convenient way to find the posts.
Related problems with reload() or paths into a package occur even more
frequently.

 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
List of lists surprising behaviour candide Python 15 06-18-2010 04:47 PM
updating dictionaries from/to dictionaries Brandon Python 12 08-15-2008 12:35 AM
surprising behaviour of os.environ.clear Joe P. Cool Python 7 06-29-2008 02:39 AM
Pickling dictionaries containing dictionaries: failing,recursion-style! lysdexia Python 6 12-02-2007 12:03 AM
Surprising(?) behaviour of the comma operator Frank Schmitt Ruby 3 12-09-2003 09:43 AM



Advertisments