Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > PYTHONPATH issue with sibling package names

Reply
Thread Tools

PYTHONPATH issue with sibling package names

 
 
Stuart Moffatt
Guest
Posts: n/a
 
      09-10-2009
Environment: Eclipse 3.4.2, Windows XP Pro SP2, Pydev 1.4.4, python
2.6

When I work in eclipse with java, I like to break up my client and
server packages, like this:

client-project/src/org/me/client

server-project/src/org/me/api
server-project/src/org/me/dao
server-project/src/org/me/entity
server-project/src/org/me/<etc>

Then, when I need to call API code from the client, I make sure the
API src is on the path.

I am trying setup pydev projects to do the same thing, but running
into an ImportError because my client code can't see the server src,
even though it is on the path.

Specifically, I am trying to import an entity from the server code
into the client, like this:

from org.me.entity import MyEntity

If I do this from any module in the server project it is fine (because
the src path is in the same eclipse project). But if I do it from
anywhere in the client code I get the ImportError

From what I can tell, python asks for the "closest" module path, which
is the current project. It finds org.me, but there is only the client
sub-package. The org.me.entity sibling is in another eclipse project,
but even though that path is on the PYTHONPATH, python stopped looking
after it found a similarly named parent package.

Is there a trusted way to make sure python looks through all paths for
sibling packages? Can I load 'org.me.client' from one path in
PYTHONPATH and 'org.me.*' from another path in PYTHONPATH? I imagine
if I try to force python to load the second package first that the
same thing will happen in reverse.
 
Reply With Quote
 
 
 
 
Diez B. Roggisch
Guest
Posts: n/a
 
      09-10-2009
Stuart Moffatt wrote:

> Environment: Eclipse 3.4.2, Windows XP Pro SP2, Pydev 1.4.4, python
> 2.6
>
> When I work in eclipse with java, I like to break up my client and
> server packages, like this:
>
> client-project/src/org/me/client
>
> server-project/src/org/me/api
> server-project/src/org/me/dao
> server-project/src/org/me/entity
> server-project/src/org/me/<etc>
>
> Then, when I need to call API code from the client, I make sure the
> API src is on the path.
>
> I am trying setup pydev projects to do the same thing, but running
> into an ImportError because my client code can't see the server src,
> even though it is on the path.
>
> Specifically, I am trying to import an entity from the server code
> into the client, like this:
>
> from org.me.entity import MyEntity
>
> If I do this from any module in the server project it is fine (because
> the src path is in the same eclipse project). But if I do it from
> anywhere in the client code I get the ImportError
>
> From what I can tell, python asks for the "closest" module path, which
> is the current project. It finds org.me, but there is only the client
> sub-package. The org.me.entity sibling is in another eclipse project,
> but even though that path is on the PYTHONPATH, python stopped looking
> after it found a similarly named parent package.
>
> Is there a trusted way to make sure python looks through all paths for
> sibling packages? Can I load 'org.me.client' from one path in
> PYTHONPATH and 'org.me.*' from another path in PYTHONPATH? I imagine
> if I try to force python to load the second package first that the
> same thing will happen in reverse.


The solution you are searching for is called "namespace packages", and you
can read more about it here:

http://peak.telecommunity.com/DevCen...space-packages


Do yourself a favor though, and don't use those several-steps-namespaces.
This is Python, not Java - two levels at *most*, normally a
project-namespace should be enough.

Diez
 
Reply With Quote
 
 
 
 
Stuart Moffatt
Guest
Posts: n/a
 
      09-10-2009
On Sep 10, 10:12*am, "Diez B. Roggisch" <(E-Mail Removed)> wrote:
> Stuart Moffatt wrote:
> > Environment: Eclipse 3.4.2, Windows XP Pro SP2, Pydev 1.4.4, python
> > 2.6

>
> > When I work in eclipse with java, I like to break up my client and
> > server packages, like this:

>
> > client-project/src/org/me/client

>
> > server-project/src/org/me/api
> > server-project/src/org/me/dao
> > server-project/src/org/me/entity
> > server-project/src/org/me/<etc>

>
> > Then, when I need to call API code from the client, I make sure the
> > API src is on the path.

>
> > I am trying setup pydev projects to do the same thing, but running
> > into an ImportError because my client code can't see the server src,
> > even though it is on the path.

>
> > Specifically, I am trying to import an entity from the server code
> > into the client, like this:

>
> > * *from org.me.entity import MyEntity

>
> > If I do this from any module in the server project it is fine (because
> > the src path is in the same eclipse project). But if I do it from
> > anywhere in the client code I get the ImportError

>
> > From what I can tell, python asks for the "closest" module path, which
> > is the current project. It finds org.me, but there is only the client
> > sub-package. The org.me.entity sibling is in another eclipse project,
> > but even though that path is on the PYTHONPATH, python stopped looking
> > after it found a similarly named parent package.

>
> > Is there a trusted way to make sure python looks through all paths for
> > sibling packages? Can I load 'org.me.client' from one path in
> > PYTHONPATH and 'org.me.*' from another path in PYTHONPATH? I imagine
> > if I try to force python to load the second package first that the
> > same thing will happen in reverse.

>
> The solution you are searching for is called "namespace packages", and you
> can read more about it here:
>
> *http://peak.telecommunity.com/DevCen...space-packages
>
> Do yourself a favor though, and don't use those several-steps-namespaces.
> This is Python, not Java - two levels at *most*, normally a
> project-namespace should be enough.
>
> Diez



Diez,

Thanks for the tips re: namespace packages. Yeah, I know this is
python, but for very large projects (or multiple projects for very
large clients) it is just more flexible to stick to the reverse-dot
java notation of "domain.organization.project.namespace".

All I had to do was make sure that top- and mid-level folders had an
__init__.py with this line:

__import__('pkg_resources').declare_namespace(__na me__)

in it (and nothing else).

So, for my example, I had:

client-project/src/org/__init__.py
client-project/src/org/me/__init__.py
server-project/src/org/__init__.py
server-project/src/org/me/__init__.py

....all with the special namespace declaration line above.

Note that the lowest level folders in the package:

client-project/src/org/me/client
server-project/src/org/me/api
server-project/src/org/me/<etc>

....all have __init__.py files with actual module code, and therefore
do NOT have the namespace declaration line.

To expose the server code (org.me.api, org.me.dao, etc) I have a
setup.py in the server-project/src folder with:

from setuptools import setup
setup(name='myserver',
version='1.0',
namespace_packages = ['org','org.me'],
packages=['org.me.api','org.me.dao','org.me.entity'],
)

A similar setup.py can be created for the client library too. Notice
that the setup has every level (except the last) in the
namespace_packages list argument. This is how it walks down the tree
and builds the namespace properly.

With this setup file I can make an .egg or .zip distribution using:

cd server-project/src
python setup.py bdist

or

python setup.py bdist_egg

For my dev environment in eclipse I also wanted the convenience of
editing server code AND have the latest code on the path for my client
project (without rebuilding a dist). To do this I just:

cd server-project/src
python setup.py develop

....which puts a symlink (even does it properly in Windows) in my
python/lib/site-packages folder. Now in my client code I can do a
regular import, even though the code lives in another project, and
though portions of the code base are at different PYTHONPATHs:

from org.me.entity import MyEntity # from server-project/src
from org.me.api import MyAPI # from server-project/src
from org.me.client import MyClient # from client-project/src

When my code is complete, the myserver-1.0-py2.6.egg and myclient-1.0-
py2.6.egg can be included in other python modules by loading compiled
source from inside the egg with:

from pkg_resources import require
require("myserver>=1.0")
require("myclient>=1.0")

and then:

from org.me.entity import MyEntity # from compiled code in
myserver-1.0-py2.6.egg
from org.me.api import MyAPI # from compiled code in myserver-1.0-
py2.6.egg
from org.me.client import MyClient # from compiled code in
myclient-1.0-py2.6.egg


Again, it might seem like a headache to some python developers, but it
replicates fairly well the java way of keeping code organized, but
"merging" it into a virtual namespace (encapsulated in an egg archive)
at run-time.

Hope that helps other java developers in packaging python code.

Stuart
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
importing package contents from multiple places in PYTHONPATH joshetomlinson@gmail.com Python 1 03-24-2008 06:41 PM
Collapse sibling nodes in 2.0 Treeview christoffer.lantz@gmail.com ASP .Net 1 02-16-2006 12:15 AM
how to find not the next sibling but the 2nd sibling or find sibling "a" OR sinbling "b" localpricemaps@gmail.com Python 11 01-23-2006 07:04 PM
Accessing Sibling Objects E Tepp Java 2 12-18-2003 04:37 PM
xpath following-sibling - null return looks for object reference? Kathy Burke ASP .Net 0 08-03-2003 10:43 PM



Advertisments