Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > PEP: import version

Reply
Thread Tools

PEP: import version

 
 
Manlio Perillo
Guest
Posts: n/a
 
      05-13-2004
Hi.
I'm a new user of Python but I have noted a little problem.
Python is a very good language but it is evolving, in particular its
library is evolving.
This can be a problem when, ad example, a module change its interface
or its implementation in a fundamental way (an example: wxPython).
This, I think, can be resolved by allowing an user to explicitly say
what version of a module it wants (sush as version numbers in Linux
shared objects).

A possible syntax is this:
import wxPython version '2.4'

When the version number is missing, the latest version is loaded.

The syntax of the version number can be expanded.
Ad example, when a module change totally its interface, instead of
renaming it, one can be do (its only an example):
import mymodule version '0'

Another case is when the module is implemented also in C:
import picke version 'C'


This new syntax can be implemented by using version numbers on the
module file name, or by introducing a new special variable at module
level: __moduleversion__

A module implementator can check against this variable:

def foo():
if __moduleversion__ < 2:
# compatibility
pass
else:
return something

(It would be better if the check can be done at module loading time)


Regards Manlio Perillo
 
Reply With Quote
 
 
 
 
Andrew Bennetts
Guest
Posts: n/a
 
      05-13-2004
On Thu, May 13, 2004 at 07:46:51AM +0000, Manlio Perillo wrote:
[...]
> A possible syntax is this:
> import wxPython version '2.4'


I'd like to see versioned imports too, but it looks like a tricky problem to
me. Some questions that immediately spring to mind reading your proposal:

How would this work with from X import Y? How would this work with packages
(e.g. import X.Y.Z)?

e.g. how would you add version requirements to these statements:

from gtk import main

from twisted.web import server

For the first one, "from gtk version '2.0' import main" doesn't look too
bad.

But what about the second one -- can only top-level modules/packages have
versions? e.g. what if the "twisted.web" package has a version that's
independent of the top-level "twisted" container package? Would "from
twisted.web version '1.3.0'" mean Twisted version 1.3.0, or Twisted Web
version 1.3.0?

What happens if the same program tries to import different (and possibly
conflicting) versions of the same module, e.g.:

import gtk version '2.0'
import gtk version '1.2'

Should an ImportError be raised? Should it succeed (assuming both versions
are available)? Is it even possible to have two different versions of the
same library/module installed in parallel, and if so, how?

I'd love to see a proposal that addresses all of these points

> Another case is when the module is implemented also in C:
> import picke version 'C'


Alternative implementations of the same module for reasons of optimisation
is a different problem, and I don't think it's a good idea to try solve it
the same way.

-Andrew.


 
Reply With Quote
 
 
 
 
Peter Otten
Guest
Posts: n/a
 
      05-13-2004
Manlio Perillo wrote:

> A possible syntax is this:
> import wxPython version '2.4'


Why not just

import wxPython2_4 as wxPython

and a symlink or something equivalent pointing to the latest version
to also allow

import wxPython

> Another case is when the module is implemented also in C:
> import picke version 'C'


Presenting the user with the choice of two equivalent modules is a mistake
IMHO. Just rename cPickle to pickle and move pickle.py into the
docs/demo/educational section. Or, more general, provide a Python version
as a fallback:

<pickle.py>
try
# try
from cPickle import *
except ImportError:
# python implementation
</pickle.py>

Peter


 
Reply With Quote
 
Skip Montanaro
Guest
Posts: n/a
 
      05-13-2004

Manlio> This, I think, can be resolved by allowing an user to explicitly
Manlio> say what version of a module it wants (sush as version numbers
Manlio> in Linux shared objects).

Manlio> A possible syntax is this:
Manlio> import wxPython version '2.4'

Somehow that version has to map to a different file being loaded, so it
seems to me the proper place to record this information is in sys.path (or
before the program runs, PYTHONPATH). Voila! No syntax changes required.

Skip

 
Reply With Quote
 
David Fraser
Guest
Posts: n/a
 
      05-13-2004
Peter Otten wrote:
> Manlio Perillo wrote:
>
>
>>A possible syntax is this:
>>import wxPython version '2.4'

>
>
> Why not just
>
> import wxPython2_4 as wxPython
>
> and a symlink or something equivalent pointing to the latest version
> to also allow
>
> import wxPython


We've been discussing different approaches to allow version import on
the wxPython list. See the wiki entry at
http://wiki.wxpython.org/index.cgi/VersionSelection

My latest code is attached, which allows selection via command-line
parameters, assuming wxPython is installed in well-named subdirectories.
See the wxpython users list for more info

David

import os
import sys
import glob

"""wxselect selects an appropriate wxPython module and adds it to sys.path
Version selection is implemented with environment variables"
Order of precedence is:
WXPYTHON_PATH
WXPYTHON_VERSION (looks for a version starting with this - 2.4 or 2.4.2.4 are valid)
WXPYTHON_MINVERSION (requires at least this version)
Otherwise the latest available version is used
"""

class EnvConfig:
"""reads environment variables of the form MODULE_KEY and stores them as self.key"""
def __init__(self, modulename, keys):
for key in keys:
setattr(self, key, os.getenv("%s_%s" % (modulename.upper(), key.upper())))

class VersionFinder:
"""Finds Versions of a module using module-x.y.z directory names and selects best match for environment variables"""
keys = ("minversion", "version", "path", "pythonpath")
def __init__(self, modulename, versionimportlist = None, versionattrlist = ["ver", "version", "VERSION", "VERSION_STRING"]):
"""construct a VersionFinder for the given modulename"""
self.modulename = modulename
if versionimportlist:
self.versionimportlist = versionimportlist
else:
self.versionimportlist = [os.path.join(self.modulename, "__version__.py")]
self.versionattrlist = versionattrlist
self.findversions()

def findversions(self):
"""finds all versions of this module by looking at module-x.y.z directories in the Python Path"""
self.versions = {}
for path in sys.path:
filenames = glob.glob(os.path.join(path, '%s-*' % self.modulename))
for filename in filenames:
if os.path.isfile(filename) and filename.lower().endswith(os.extsep + "pth"):
versionname = os.path.splitext(os.path.basename(filename))[0]
versiondirs = open(filename).readlines()
versionpaths = []
for versiondir in versiondirs:
versionpaths.extend(self.readversionpath(versiondi r.strip()))
elif os.path.isdir(filename):
versionname = os.path.basename(filename)
versionpaths = self.readversionpath(filename)
else:
continue
version = versionname[len("%s-" % self.modulename):]
if version not in self.versions:
self.versions[version] = versionpaths
return self.versions

def readversionpath(self, versiondir):
"""reads any .pth files in the versiondir and returns the path required for the version"""
versionpaths = [versiondir]
versionpthfiles = glob.glob(os.path.join(versiondir, '*.pth'))
for pthfile in versionpthfiles:
for line in open(pthfile, "r").readlines():
versionpath = line.strip()
if not versionpath: continue
if not os.path.isabs(versionpath):
versionpath = os.path.join(os.path.dirname(versiondir), versionpath)
versionpaths.append(versionpath)
return versionpaths

def readpathversion(self, versionpath):
"""reads the module version from the given path"""
import imp
for versionimportpath in self.versionimportlist:
versionfilename = os.path.join(versionpath, versionimportpath)
if os.path.isfile(versionfilename):
versionmodule = imp.load_source(os.path.basename(versionfilename), versionfilename, open(versionfilename, 'r'))
if versionmodule is not None:
for versionattrname in self.versionattrlist:
version = getattr(versionmodule, versionattrname, None)
if version is not None:
return version
return None

def getversionpath(self, version):
"""looks up the pathsep-joined path for the given version"""
return os.path.pathsep.join(self.versions[version])

def listversions(self):
"""lists known versions"""
return self.versions.keys()

def getbestversion(self, possibleversions):
"""finds the best version out of the possibilities"""
if possibleversions:
return max(possibleversions)

def getconfig(self, path=None, version=None, minversion=None):
"""reads the environment variables and intelligently chooses version and path"""
config = EnvConfig(self.modulename, self.keys)
if path:
config.path = path
if version:
config.version = version
if minversion:
config.minversion = minversion
if config.path:
config.version = self.readpathversion(config.path)
else:
if config.version:
possibleversions = [version for version in self.listversions() if version.startswith(config.version)]
elif config.minversion:
possibleversions = [version for version in self.listversions() if version >= config.minversion]
else:
possibleversions = self.listversions()
config.version = self.getbestversion(possibleversions)
if config.version:
config.path = self.getversionpath(config.version)
return config

def setpath(self, path=None, version=None, minversion=None):
"""removes other versions from the path and appends the selected path"""
allpaths = []
map(allpaths.extend, self.versions.values())
self.removefrompath(allpaths)
config = self.getconfig(path, version, minversion)
self.appendtopath(config.path)

def appendtopath(self, paths):
"""takes a pathsep-separated path list and adds elements to the Python path at the end"""
if paths:
pathlist = paths.split(os.path.pathsep)
pathlist = [path for path in pathlist if path and os.path.isdir(path)]
sys.path.extend(pathlist)

def prependtopath(self, paths):
"""takes a pathsep-separated path list and adds elements to the Python path at the beginning"""
if paths:
pathlist = paths.split(os.path.pathsep)
pathlist = [path for path in pathlist if path and os.path.isdir(path)]
sys.path = pathlist + sys.path

def removefrompath(self, pathlist):
"""removes all known versions from the PythonPath"""
if pathlist:
pathlist = [os.path.abspath(path) for path in pathlist if path and os.path.isdir(path)]
sys.path = [path for path in sys.path if os.path.abspath(path) not in pathlist]

wx25versionfile = os.path.join("wx", "__version__.py")
wx25versionattr = "VERSION_STRING"
wx24versionfile = os.path.join("wxPython", "__version__.py")
wx24versionattr = "wxVERSION_STRING"
wxversionimportlist = [wx25versionfile, wx24versionfile]
wxversionattrlist = [wx25versionattr, wx24versionattr]
wxVersionFinder = VersionFinder("wxPython", versionimportlist = wxversionimportlist, versionattrlist = wxversionattrlist)

if __name__ == "__main__":
print "wxPython version selector"
print "available versions:"
for version, path in wxVersionFinder.versions.iteritems():
print "%s: %s" % (version, path)
print
config = wxVersionFinder.getconfig()
print "selected: %s in %s" % (config.version, config.path)
else:
wxVersionFinder.setpath()
# wxVersionFinder.appendtopath(wxVersionFinder.getco nfig().path)


 
Reply With Quote
 
Christopher Barker
Guest
Posts: n/a
 
      05-13-2004
Skip Montanaro wrote:
> Somehow that version has to map to a different file being loaded, so it
> seems to me the proper place to record this information is in sys.path (or
> before the program runs, PYTHONPATH). Voila! No syntax changes required.


This really needs to be done at runtime, not ahead of time with PYTHONPATH.

We have been discussing this over at wxPython-users, and a number f
workable solutions have been proposed. However, it would be great if
there was an "official" Pythonic way to do it, rather than each package
author coming up with their own system. So, I'm all for a PEP, even if
it is just for a standard approach, and no change to python whatsoever.
If someone is working on a proposal, you might want to consider these
(from the wxPython Wiki):

Basic Goals:

* Multiple versions of wxPython being installed at the same time
* Two or more programs, each using a different version could run at
once.
* Easy for developers - for testing with multiple versions etc
* Easy for users - their apps detect the right version to run with
* Cross-Platform solution

More detail:

* The ability for the programmer to specify the version used in the
code, ideally only on (or before) the first import of wxPython
* A default version that gets imported with "import wx", just like
it's always been
* The ability to specify the version wanted with an environment
variable, so that multiple versions could be easily tested
* The ability to specify multiple versions that all work, in a
ordered list of some sort.
* The ability to specify a "minimal" version, and higher versions
could be used.
* The ability for "from wx.lib import Something" to work as well,
respecting the version

-Chris


--
Christopher Barker, Ph.D.
Oceanographer

NOAA/OR&R/HAZMAT (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception

http://www.velocityreviews.com/forums/(E-Mail Removed)
 
Reply With Quote
 
Skip Montanaro
Guest
Posts: n/a
 
      05-13-2004

Chris> Skip Montanaro wrote:
>> Somehow that version has to map to a different file being loaded, so
>> it seems to me the proper place to record this information is in
>> sys.path (or before the program runs, PYTHONPATH). Voila! No syntax
>> changes required.


Chris> This really needs to be done at runtime, not ahead of time with
Chris> PYTHONPATH.

Chris> We have been discussing this over at wxPython-users,

Perhaps I'm missing something... If you need to futz around at runtime,
futz around with sys.path. Can you summarize the wxPython discussions for
those of us who don't use that toolkit (and have apparently never run into
this problem in such an intractable way that we felt the need to change
Python because of it)? In particular it would be helpful to understand why
you don't know what version of wxPython you want to use before running the
program.

Thanks,

Skip

 
Reply With Quote
 
Skip Montanaro
Guest
Posts: n/a
 
      05-13-2004

Chris> Basic Goals:

Chris> * Multiple versions of wxPython being installed at the same time

Presumably in different directories, so setting PYTHONPATH, LD_LIBRARY_PATH,
etc will distiguish them for applications.

Chris> * Two or more programs, each using a different version could run
Chris> at once.

This doesn't require a change to the language. This requires a proper
setting of PYTHONPATH and/or LD_LIBRARY_PATH, etc.

Chris> * Easy for developers - for testing with multiple versions etc

That's fine, and precludes the use of something like a hardcoded

import wxPython version '2.4'

as someone else suggested. In fact, this just cries out for either a
WXPYTHON_VERSION environment variable which the wxPython framework could use
or for setting PYTHONPATH.

Chris> * Easy for users - their apps detect the right version to run
Chris> with

Getting the right version of dependent packages is an installation issue.
This should be detectable from either configure script, Makefile or
distutils setup.py script.

Chris> * Cross-Platform solution

Distribute your apps with .bat file front ends on Windows, shell scripts
on Unix.

Chris> More detail:

Chris> * The ability for the programmer to specify the version used in
Chris> the code, ideally only on (or before) the first import of
Chris> wxPython

I fail to understand why that's a problem that needs to be addressed in the
Python program. This is information which is know before the program is
run. It belongs in an environment variable or front-end shell/bat script.

Chris> * The ability to specify the version wanted with an environment
Chris> variable, so that multiple versions could be easily tested

This is the correct way to do this.

Chris> * The ability to specify multiple versions that all work, in a
Chris> ordered list of some sort.

Your installation program should detect all installed versions and pick the
latest version which intersects with the set of multiple working versions.

Chris> * The ability to specify a "minimal" version, and higher versions
Chris> could be used.

Again, this goes in your setup.py script (or configure, or Makefile, or
whatever). It's an install-time decision.

Chris> * The ability for "from wx.lib import Something" to work as well,
Chris> respecting the version

Assuming your installation software worked correctly this should be a
no-brainer.

Skip

 
Reply With Quote
 
Christopher Barker
Guest
Posts: n/a
 
      05-13-2004
Skip Montanaro wrote:

> Perhaps I'm missing something... If you need to futz around at runtime,
> futz around with sys.path. Can you summarize the wxPython discussions for
> those of us who don't use that toolkit (and have apparently never run into
> this problem in such an intractable way that we felt the need to change
> Python because of it)?



Actually, I was personally advocating an approach that did not require
changing python, but would hopefully be standardized. I posted a
question to this newsgroup about it a short while back, and I think
PyGTK and PMW both do this in some way or another, so it's not a totally
unique problem.

> In particular it would be helpful to understand why
> you don't know what version of wxPython you want to use before running > the program.


I know what version I want when writing it, but not necessarily when
running it. This is analogous to why I ALWAYS use:

#!/usr/bin/env python2.3

rather than:

#!/usr/bin/env python

at the top of my scripts. RedHat doing the later created a major
pain-in-the-behind for many of us. I only wish there were a way for
Windows Python to handle that kind of thing gracefully. MacPython uses a
"PythonLauncher" mini-app to take care of it (and for other reasons).

Anyway, as for wxPython (and many other packages, I imagine), I have a
handful of small utilities that work just fine with Python version X and
wxPython version Y. If I upgrade python while keeping the old one in
place, they all still work (thanks to my specifying the version on the
#! line), However, if I add wxPython version Z, I'm stuck. The way it's
ordinarily installed is in site-packages/wx. If I do a standard install,
it blows away version Y, and all my existing programs are broken. What
I, and other "early adopters" of new versions do is do a custom install
and run a script that re-names or sym-links site-packages/wx to the
version we want. That helps, but then at any given moment, only one is
the active version, so I can't run a new and old program at the same
time. So what some of us would like is for the standard way that
wxPython is installed to allow for versioning. The Wiki page pointed to
earlier has some suggestions about how to do this.


> Chris> Basic Goals:
> Chris> * Multiple versions of wxPython being installed at the same time



> Presumably in different directories, so setting PYTHONPATH, LD_LIBRARY_PATH,
> etc will distiguish them for applications.



Well, yes, but unfortunately, the standard install puts it in
site-packages, so you have to do a custom install to make this work.

> Chris> * Two or more programs, each using a different version could run
> Chris> at once. This doesn't require a change to the language. This requires a proper
> setting of PYTHONPATH and/or LD_LIBRARY_PATH, etc.



Well, yes, but that's a PITA that I don't want to have to go through. I
want to be able to distribute apps also, and not everyone is going to
have the same paths as me, and certainly not the same on different OSs.

> Chris> * Easy for developers - for testing with multiple versions etc
>
> That's fine, and precludes the use of something like a hardcoded
>
> import wxPython version '2.4'



No, it doesn't. Change the 2.4 to 2.5 and you test again. However, there
have been requests for:

> a
> WXPYTHON_VERSION environment variable which the wxPython framework could use
> or for setting PYTHONPATH.



yup, that's one option, but good for developers, not good for users.

>
> Chris> * Easy for users - their apps detect the right version to run
> Chris> with



> Getting the right version of dependent packages is an installation issue.
> This should be detectable from either configure script, Makefile or
> distutils setup.py script.



Actually this is poorly worked, I don't want the app to detect anything,
I want to to specify something:

import wx-2_5 as wx

for instance.

> Chris> * Cross-Platform solution
>
> Distribute your apps with .bat file front ends on Windows, shell scripts
> on Unix.



Yuck, yuck yuck! First of all, I don't know a .bat from a hole in the
ground (though I suppose I need to learn this to deal with python
versioning anyway..sigh), and second of all, I'd much rather not have to
write a wrapper script around a twenty line utility script (or a 2000
line one either!), and third of all, the wrapper script would be
platform and individual system dependent

> Chris> * The ability for the programmer to specify the version used in
> Chris> the code, ideally only on (or before) the first import of
> Chris> wxPython



> I fail to understand why that's a problem that needs to be addressed in the
> Python program. This is information which is know before the program is
> run. It belongs in an environment variable or front-end shell/bat script.



A front-end script script is one solution to these problems, but it's
another layer I would rather not have to deal with! Another issue is for
OS_X BundleBuilder Bundles. You can build them to use the Python
installation, so I'd like to be able to to the version specifying inside
the python script, not outside of it.

> Chris> * The ability to specify the version wanted with an environment Chris> variable, so that multiple versions could be easily tested
>
> This is the correct way to do this.



It is ONE way to so it, not THE way.

> Chris> * The ability to specify multiple versions that all work, in a Chris> ordered list of some sort.
> Your installation program should detect all installed versions and pick the
> latest version which intersects with the set of multiple working versions.



Again, simple solutions for simple problems. I don't want to write a
complex installer. Frankly, I think this is getting too fancy (though it
would be nice). I'm just as happy to have my apps Require a particular
version, that way I know I've tested there. Disk space it cheap.

> Chris> * The ability to specify a "minimal" version, and higher versions Chris> could be used.
>
> Again, this goes in your setup.py script (or configure, or Makefile, or
> whatever). It's an install-time decision.



Again with the installer! What's wrong with dropping a script in
/usr/local/bin? Granted when you get beyond small utilities, there is
often a need for a proper installer, but in that case, you could be
building stand alone apps with PY2EXE or whatever anyway.

> Chris> * The ability for "from wx.lib import Something" to work as well, Chris> respecting the version



> Assuming your installation software worked correctly this should be a
> no-brainer.



This was addressing why you couldn't just do:

import wx-2_5 as wx

because then you'd have to change all the "wx" to "wx-2-2_5" everywhere
else which could get ugly. One thing that makes this harder, at least
for wxPython, is that there is a legacy way that wxPython has been
installed and imported, and people would like to preserve that, so that

import wx

and

from wx.lib import Something

and

import wx.lib.Something

need to all still work.

By the way, as my thought about this started with wxPython, and now I'm
advocating a more general solution, I realize there are some
differences. A big one is that you could never use more than one version
of wxPython in a single program, but you might very well be able to use
multiple versions of other packages in the same program. So, while for
wxPython it makes sense for __main__ to specify the wxPython version,
and all other modules that "import wx" get than version, this may not be
the case with other packages, where each module using it might want a
different version.

This is really analogous to shared libraries. I see a Python package as
no different to a shared lib in function. When you compile and link and
app, the version of the shared lib is specified. When it is run, the
linker looks for an links that version. The user doesn't have to set an
environment variable first, or change the lib path, or anything else.
All I'm looking for is that same functionality. Does anyone remember the
old DOS days of DLL hell? why would we want that? (Is Windows any better
now? I'm wouldn't know)


-Chris

--
Christopher Barker, Ph.D.
Oceanographer

NOAA/OR&R/HAZMAT (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception

(E-Mail Removed)
 
Reply With Quote
 
A. Lloyd Flanagan
Guest
Posts: n/a
 
      05-14-2004
Christopher Barker <(E-Mail Removed)> wrote in message news:<c80v79$h2o$(E-Mail Removed)>...
> differences. A big one is that you could never use more than one version
> of wxPython in a single program, but you might very well be able to use
> multiple versions of other packages in the same program. So, while for


Using multiple versions in one program sounds like a recipe for
disaster to me, even with some sort of language support. Also, we
shouldn't make the normal single-version case more complex or less
efficient to support a case that should be extremely rare.
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Re: Where to get stand alone Dot Net Framework version 1.1, version2.0, version 3.0, version 3.5, version 2.0 SP1, version 3.0 SP1 ? MowGreen [MVP] ASP .Net 5 02-09-2008 01:55 AM
Re: Where to get stand alone Dot Net Framework version 1.1, version 2.0, version 3.0, version 3.5, version 2.0 SP1, version 3.0 SP1 ? PA Bear [MS MVP] ASP .Net 0 02-05-2008 03:28 AM
Re: Where to get stand alone Dot Net Framework version 1.1, version 2.0, version 3.0, version 3.5, version 2.0 SP1, version 3.0 SP1 ? V Green ASP .Net 0 02-05-2008 02:45 AM
to use import java.lang.* or import java.lang.Math or none at all? JPractitioner Java 13 02-24-2006 08:48 PM
IMPORT STATIC; Why is "import static" file scope? Why not class scope? Paul Opal Java 12 10-10-2004 11:01 PM



Advertisments