Velocity Reviews

Velocity Reviews (http://www.velocityreviews.com/forums/index.php)
-   Python (http://www.velocityreviews.com/forums/f43-python.html)
-   -   Unexpected python exception (http://www.velocityreviews.com/forums/t704918-unexpected-python-exception.html)

Richard Purdie 11-11-2009 10:59 AM

Unexpected python exception
 
I've been having problems with an unexpected exception from python which
I can summarise with the following testcase:

def A():
import __builtin__
import os

__builtin__.os = os

def B():
os.stat("/")
import os

A()
B()

which results in:

Traceback (most recent call last):
File "./test.py", line 12, in <module>
B()
File "./test.py", line 8, in B
os.stat("/")
UnboundLocalError: local variable 'os' referenced before assignment

If I remove the "import os" from B(), it works as expected.

>From what I've seen, its very unusual to have something operate

"backwards" in scope in python. Can anyone explain why this happens?

Cheers,

Richard


Diez B. Roggisch 11-11-2009 11:21 AM

Re: Unexpected python exception
 
Richard Purdie schrieb:
> I've been having problems with an unexpected exception from python which
> I can summarise with the following testcase:
>
> def A():
> import __builtin__
> import os
>
> __builtin__.os = os
>
> def B():
> os.stat("/")
> import os
>
> A()
> B()
>
> which results in:
>
> Traceback (most recent call last):
> File "./test.py", line 12, in <module>
> B()
> File "./test.py", line 8, in B
> os.stat("/")
> UnboundLocalError: local variable 'os' referenced before assignment
>
> If I remove the "import os" from B(), it works as expected.
>
>>From what I've seen, its very unusual to have something operate

> "backwards" in scope in python. Can anyone explain why this happens?


As the import-statement in a function/method-scope doesn't leak the
imported names into the module scope, python treats them as locals.
Which makes your code equivalent to


x = 1000

def foo():
print x
x = 10

Throws the same error. The remedy is to inform python that a specific
name belongs to global scope, using the "global"-statement.

def foo():
global x
print x
x = 10


Beware though that then of course *assigning* to x is on global level.
This shouldn't be of any difference in your case though, because of the
import-only-once-mechanics of python.

Diez

Andreas L÷scher 11-11-2009 11:25 AM

Re: Unexpected python exception
 
Hi,
unfortunatley I cannot reproduce your error. Which Python Version do you
use?

The expected case in this scenario is that the exception is thrown, as
you import os in A() where it is stored in the local namespace of the
function.

I tested it with Python 2.4, 2.5 and 2.6 and in both cases an exception
is thrown.

Best,
Andreas


Chris Rebert 11-11-2009 12:23 PM

Re: Unexpected python exception
 
On Wed, Nov 11, 2009 at 8:49 AM, Eduardo Lenz <lenz@joinville.udesc.br> wrote:
> Em Qua 11 Nov 2009, ├*s 03:21:55, Diez B. Roggisch escreveu:
>> Richard Purdie schrieb:
>> > I've been having problems with an unexpected exception from python which
>> > I can summarise with the following testcase:
>> >
>> > def A():
>> > ┬* ┬* import __builtin__
>> > ┬* ┬* import os
>> >
>> > ┬* ┬* __builtin__.os = os
>> >
>> > def B():
>> > ┬* ┬* os.stat("/")
>> > ┬* ┬* import os
>> >
>> > A()
>> > B()
>> >
>> > which results in:
>> >
>> > Traceback (most recent call last):
>> > ┬* File "./test.py", line 12, in <module>
>> > ┬* ┬* B()
>> > ┬* File "./test.py", line 8, in B
>> > ┬* ┬* os.stat("/")
>> > UnboundLocalError: local variable 'os' referenced before assignment
>> >
>> > If I remove the "import os" from B(), it works as expected.
>> >
>> >>From what I've seen, its very unusual to have something operate
>> >
>> > "backwards" in scope in python. Can anyone explain why this happens?

>>
>> As the import-statement in a function/method-scope doesn't leak the
>> imported names into the module scope, python treats them as locals.
>> Which makes your code equivalent to
>>
>>
>> x = 1000
>>
>> def foo():
>> ┬* ┬* ┬*print x
>> ┬* ┬* ┬*x = 10
>>
>> Throws the same error. The remedy is to inform python that a specific
>> name belongs to global scope, using the "global"-statement.
>>
>> def foo():
>> ┬* ┬* ┬*global x
>> ┬* ┬* ┬*print x
>> ┬* ┬* ┬*x = 10
>>
>>
>> Beware though that then of course *assigning* to x is on global level.
>> This shouldn't be of any difference in your case though, because of the
>> import-only-once-mechanics of python.
>>
>> Diez
>>

>
> So...it should not work
>
> def A():
> ┬* ┬* import __builtin__
> ┬* ┬* import os
> ┬* ┬* __builtin__.os = os
>
> A()
> os.stat("/")
>
> but it does. ┬*Why ? B() cannot see the import, but the global level can ?


The optimization which results in the behavior in question is only
done on functions scopes, not global scope.

Cheers,
Chris
--
http://blog.rebertia.com

Andreas L÷scher 11-11-2009 12:23 PM

Re: Unexpected python exception
 
Python searches for Variables not only in local or global scoop but also
in __builtins__. If you do something like __builtins__.os = os, than
this variable should be accessible global.

If you then write something like:
def B():
os.stat("/")
import os

Python recognises on compile time, that os is a local variable in B and
allocates memory for it. All reading or writing now goes to the local
variable and not the one in __builtins__. And the local variable has
never been assigned to a value and an Exception is thrown.

Best


Richard Purdie 11-11-2009 12:37 PM

Re: Unexpected python exception
 
On Wed, 2009-11-11 at 12:21 +0100, Diez B. Roggisch wrote:
> As the import-statement in a function/method-scope doesn't leak the
> imported names into the module scope, python treats them as locals.
> Which makes your code equivalent to
>
>
> x = 1000
>
> def foo():
> print x
> x = 10


Aha, thanks. This makes it clear whats happening.

> Throws the same error. The remedy is to inform python that a specific
> name belongs to global scope, using the "global"-statement.
>
> def foo():
> global x
> print x
> x = 10
>
>
> Beware though that then of course *assigning* to x is on global level.
> This shouldn't be of any difference in your case though, because of the
> import-only-once-mechanics of python.


Is there a way to make the "global x" apply to all functions without
adding it to each one?

I suspect this equates to intentionally "leaking the imported names into
the module scope"? :)

What I'm trying to do is to avoid having "import X" statements
everywhere by changing __builtin__. It seems my approach doesn't have
quite the same effect as a true import though.

Cheers,

Richard


Chris Rebert 11-11-2009 01:04 PM

Re: Unexpected python exception
 
On Wed, Nov 11, 2009 at 4:37 AM, Richard Purdie <rpurdie@rpsys.net> wrote:
<snip>
> Is there a way to make the "global x" apply to all functions without
> adding it to each one?


Thankfully, no.

> What I'm trying to do is to avoid having "import X" statements
> everywhere by changing __builtin__. It seems my approach doesn't have
> quite the same effect as a true import though.


And you can't just put all your imports together at the top of the
file because...?
If you're importing the same stuff across multiple modules, I'd say
you have a code smell on your hands. Injecting stuff into the builtin
namespace purely for convenience's sake is Just Evil (tm). At the
least, you can put all the imports in one module and then use `from
all_imports import *`, which is just slightly less evil.

Cheers,
Chris
--
http://blog.rebertia.com

Ralax 11-11-2009 02:11 PM

Re: Unexpected python exception
 
On Nov 11, 6:59*pm, Richard Purdie <rpur...@rpsys.net> wrote:
> I've been having problems with an unexpected exception from python which
> I can summarise with the following testcase:
>
> def A():
> * * import __builtin__
> * * import os
>
> * * __builtin__.os = os
>
> def B():
> * * os.stat("/")
> * * import os
>
> A()
> B()
>
> which results in:
>
> Traceback (most recent call last):
> * File "./test.py", line 12, in <module>
> * * B()
> * File "./test.py", line 8, in B
> * * os.stat("/")
> UnboundLocalError: local variable 'os' referenced before assignment
>
> If I remove the "import os" from B(), it works as expected.
>
> >From what I've seen, its very unusual to have something operate

>
> "backwards" in scope in python. Can anyone explain why this happens?
>
> Cheers,
>
> Richard


One word, Python treat objects in a function or method-scope as
locals.
So os is not defined in B(), is it right?

Richard Purdie 11-11-2009 02:29 PM

Re: Unexpected python exception
 
On Wed, 2009-11-11 at 05:04 -0800, Chris Rebert wrote:
> On Wed, Nov 11, 2009 at 4:37 AM, Richard Purdie <rpurdie@rpsys.net> wrote:
> <snip>
> > Is there a way to make the "global x" apply to all functions without
> > adding it to each one?

>
> Thankfully, no.


Hmm :(.

> > What I'm trying to do is to avoid having "import X" statements
> > everywhere by changing __builtin__. It seems my approach doesn't have
> > quite the same effect as a true import though.

>
> And you can't just put all your imports together at the top of the
> file because...?


The application in question is bitbake, the parsing tool for the
OpenEmbedded and Poky projects.

We're talking about python code fragments spread across about 8,000
files which make up the "metadata" some of which is public and some of
which is not. Rightly or wrongly bitbake does some interesting things
with its execution contexts for these code fragments.

Lets just say its not a traditional use case, we know the way bitbake
does some things is evil but it does other things rather well and we
can't break those.

Cheers,

Richard




Eduardo Lenz 11-11-2009 04:49 PM

Re: Unexpected python exception
 
Em Qua 11 Nov 2009, Ós 03:21:55, Diez B. Roggisch escreveu:
> Richard Purdie schrieb:
> > I've been having problems with an unexpected exception from python which
> > I can summarise with the following testcase:
> >
> > def A():
> > import __builtin__
> > import os
> >
> > __builtin__.os = os
> >
> > def B():
> > os.stat("/")
> > import os
> >
> > A()
> > B()
> >
> > which results in:
> >
> > Traceback (most recent call last):
> > File "./test.py", line 12, in <module>
> > B()
> > File "./test.py", line 8, in B
> > os.stat("/")
> > UnboundLocalError: local variable 'os' referenced before assignment
> >
> > If I remove the "import os" from B(), it works as expected.
> >
> >>From what I've seen, its very unusual to have something operate

> >
> > "backwards" in scope in python. Can anyone explain why this happens?

>
> As the import-statement in a function/method-scope doesn't leak the
> imported names into the module scope, python treats them as locals.
> Which makes your code equivalent to
>
>
> x = 1000
>
> def foo():
> print x
> x = 10
>
> Throws the same error. The remedy is to inform python that a specific
> name belongs to global scope, using the "global"-statement.
>
> def foo():
> global x
> print x
> x = 10
>
>
> Beware though that then of course *assigning* to x is on global level.
> This shouldn't be of any difference in your case though, because of the
> import-only-once-mechanics of python.
>
> Diez
>


So...it should not work

def A():
import __builtin__
import os
__builtin__.os = os

A()
os.stat("/")

but it does. Why ? B() cannot see the import, but the global level can ?

Thanks.

Eduardo.

--
Esta mensagem foi verificada pelo sistema de antivÝrus e
acredita-se estar livre de perigo.



All times are GMT. The time now is 11:51 AM.

Powered by vBulletin®. Copyright ©2000 - 2014, vBulletin Solutions, Inc.
SEO by vBSEO ©2010, Crawlability, Inc.