Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > Python Embedding Importing relative modules

Reply
Thread Tools

Python Embedding Importing relative modules

 
 
moerchendiser2k3
Guest
Posts: n/a
 
      07-05-2010
Hi all,

I have a serious problem I haven't solved yet, hope one of you can
help me. The first thing is, I embedded Python into my app and I
execute several scripts in this environment.

The problem is, the scripts don't import modules from their relative
path. I guess this is related to the sys.path ['',...] and the current
working directory which is set to the directory of my host
application.

I could set that path manually, but all the scripts might be stored in
different locations. So now I try to find a way to handle that. Any
suggestions?

A solution would be, that each script, appends its own directory to
the system path, but this might lead to problems. Imagine all of them
have a module called 'foo.py' and its not the same. This might lead to
name conflicts, wouldnt it?

Btw, I found a source code line in the documentation, where I should
really get rid of the ['', ...] path in the system path due to
security reasons.

import sys; sys.path.pop(0)


Hope one of you can help me out here. Really thanks!!

Bye,

moerchendiser2k3
 
Reply With Quote
 
 
 
 
Aahz
Guest
Posts: n/a
 
      07-06-2010
In article <(E-Mail Removed)>,
moerchendiser2k3 <(E-Mail Removed)> wrote:
>
>I have a serious problem I haven't solved yet, hope one of you can
>help me. The first thing is, I embedded Python into my app and I
>execute several scripts in this environment.
>
>The problem is, the scripts don't import modules from their relative
>path. I guess this is related to the sys.path ['',...] and the current
>working directory which is set to the directory of my host
>application.
>
>I could set that path manually, but all the scripts might be stored in
>different locations. So now I try to find a way to handle that. Any
>suggestions?


Set sys.path to include each script's base dir before running it, then
restore after each script.
--
Aahz ((E-Mail Removed)) <*> http://www.pythoncraft.com/

"If you don't know what your program is supposed to do, you'd better not
start writing it." --Dijkstra
 
Reply With Quote
 
 
 
 
moerchendiser2k3
Guest
Posts: n/a
 
      07-06-2010
>Set sys.path to include each script's base dir before running it, then
>restore after each script.


That works, but doesnt solve the problem.

ScriptA.py has a module in its directory called 'bar.py'
ScriptB.py has a module in its directory called 'bar.py'

Imagine the 'bar.py' modules dont have the same content, so
they are not equal.

Now when the first bar.py is imported, the second import for
a "import bar" imports the first one, because its already
stored in sys.modules.
 
Reply With Quote
 
Aahz
Guest
Posts: n/a
 
      07-06-2010
In article <(E-Mail Removed)>,
moerchendiser2k3 <(E-Mail Removed)> wrote:
>Aahz:
>>
>>Set sys.path to include each script's base dir before running it, then
>>restore after each script.

>
>That works, but doesnt solve the problem.
>
>ScriptA.py has a module in its directory called 'bar.py'
>ScriptB.py has a module in its directory called 'bar.py'
>
>Imagine the 'bar.py' modules dont have the same content, so they are
>not equal.
>
>Now when the first bar.py is imported, the second import for a "import
>bar" imports the first one, because its already stored in sys.modules.


Good point, you'll need to save/restore sys.modules, too. That gets you
90-95% of complete namespace separation; if you need more than that, your
best bet is to use separate processes. Full-blown namepace isolation is
a *hard* problem, just look at all the past attempts to create secure
Python (and what you're trying to do is roughly equivalent).
--
Aahz ((E-Mail Removed)) <*> http://www.pythoncraft.com/

"If you don't know what your program is supposed to do, you'd better not
start writing it." --Dijkstra
 
Reply With Quote
 
moerchendiser2k3
Guest
Posts: n/a
 
      07-06-2010
Good idea. Just one thing I thought about:

Imagine I load them parallel so the GIL might
interrupt the save-process of sys.modules,

[ENSURE GIL]
Load Script
Save sys.modules
[interrupt]
Load Script
Save sys.modules
....

so this might run into several other problems.

But maybe I change that so I load the scripts in a row.
Thanks for your shoulder I can cry on

Bye,

moerchendiser2k3
 
Reply With Quote
 
Aahz
Guest
Posts: n/a
 
      07-06-2010
In article <(E-Mail Removed)>,
moerchendiser2k3 <(E-Mail Removed)> wrote:
>
>Imagine I load them parallel so the GIL might interrupt the
>save-process of sys.modules,
>
>[ENSURE GIL]
>Load Script
>Save sys.modules
>[interrupt]
>Load Script
>Save sys.modules
>...
>
>so this might run into several other problems.


Very yes; if you're trying to maintain script independence, you should
either run them sequentially (so you can safely save/restore the
environment) or use processes.
--
Aahz ((E-Mail Removed)) <*> http://www.pythoncraft.com/

"If you don't know what your program is supposed to do, you'd better not
start writing it." --Dijkstra
 
Reply With Quote
 
Thomas Jollans
Guest
Posts: n/a
 
      07-06-2010
On 07/06/2010 09:11 PM, Aahz wrote:
> In article <(E-Mail Removed)>,
> moerchendiser2k3 <(E-Mail Removed)> wrote:
>> Aahz:
>>>
>>> Set sys.path to include each script's base dir before running it, then
>>> restore after each script.

>>
>> That works, but doesnt solve the problem.
>>
>> ScriptA.py has a module in its directory called 'bar.py'
>> ScriptB.py has a module in its directory called 'bar.py'
>>
>> Imagine the 'bar.py' modules dont have the same content, so they are
>> not equal.
>>
>> Now when the first bar.py is imported, the second import for a "import
>> bar" imports the first one, because its already stored in sys.modules.

>
> Good point, you'll need to save/restore sys.modules, too. That gets you
> 90-95% of complete namespace separation; if you need more than that, your
> best bet is to use separate processes. Full-blown namepace isolation is
> a *hard* problem, just look at all the past attempts to create secure
> Python (and what you're trying to do is roughly equivalent).


I believe Python (at least in the 3.x series) supports multiple
interpreter instances per process.
 
Reply With Quote
 
Aahz
Guest
Posts: n/a
 
      07-06-2010
In article <(E-Mail Removed)>,
Thomas Jollans <(E-Mail Removed)> wrote:
>On 07/06/2010 09:11 PM, Aahz wrote:
>> In article <(E-Mail Removed)>,
>> moerchendiser2k3 <(E-Mail Removed)> wrote:
>>> Aahz:
>>>>
>>>> Set sys.path to include each script's base dir before running it, then
>>>> restore after each script.
>>>
>>> That works, but doesnt solve the problem.
>>>
>>> ScriptA.py has a module in its directory called 'bar.py'
>>> ScriptB.py has a module in its directory called 'bar.py'
>>>
>>> Imagine the 'bar.py' modules dont have the same content, so they are
>>> not equal.
>>>
>>> Now when the first bar.py is imported, the second import for a "import
>>> bar" imports the first one, because its already stored in sys.modules.

>>
>> Good point, you'll need to save/restore sys.modules, too. That gets you
>> 90-95% of complete namespace separation; if you need more than that, your
>> best bet is to use separate processes. Full-blown namepace isolation is
>> a *hard* problem, just look at all the past attempts to create secure
>> Python (and what you're trying to do is roughly equivalent).

>
>I believe Python (at least in the 3.x series) supports multiple
>interpreter instances per process.


Perhaps things have changed in 3.x, but although 2.x supposedly supported
multiple interpreter instances per process, the impression I got was that
most people would rather poke their eyes out with a dull stick than try
to make multiple interpreters per process actually work.

Multiple processes are easy, OTOH.
--
Aahz ((E-Mail Removed)) <*> http://www.pythoncraft.com/

"If you don't know what your program is supposed to do, you'd better not
start writing it." --Dijkstra
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Importing v reloading modules modules Peter Peyman Puk Python 0 03-19-2010 05:09 PM
Importing modules from packages with relative imports John Millikin Python 0 05-30-2008 09:16 PM
after embedding and extending python (using swig) problem importing (non-core) modules stefan Python 3 12-08-2004 07:11 PM
Importing modules from within other modules Tobiah Python 2 09-14-2003 09:18 PM



Advertisments