Re: imputil.py, is this a bug ?

En Fri, 06 Nov 2009 18:33:37 -0300, Stef Mientki <>
escribió:

> I get an error compiling with pyjamas, in the standard module imputil,
> _import_top_module

Note that imputil is undocumented in 2.5, deprecated in 2.6 and
definitively gone in 3.0

> AttributeError: 'unicode' object has no attribute 'import_top'
>
> def _import_top_module(self, name):
> # scan sys.path looking for a location in the filesystem that
> contains
> # the module, or an Importer object that can import the module.
> for item in sys.path:
> if isinstance(item, _StringType):
> module = self.fs_imp.import_from_dir(item, name)
> else:
> module = item.import_top(name)
> if module:
> return module
> return None
>
> It seems that elements of sys.path can be of the type unicode
> so by adding the next 2 lines, everything works ok.
> elif isinstance ( item, basestring ) :
> module = self.fs_imp.import_from_dir ( str(item), name)
>
> is this a bug ?
> (I'm using Python 2.5.2 on Windows )

Yes, seems to be a bug. But given the current status of imputil, it's not
likely to be fixed; certainly not in 2.5 which only gets security fixes
now.

I cannot test it at this moment, but I'd use the unicode item directly
(that is, self.fs_imp.import_from_dir(item, name)). Or perhaps
item.encode(sys.getdefaultfilesystemencoding()). str(item) definitively
won't work with directory names containing non-ascii characters.

Why are you using imputil in the first place?

--
Gabriel Genellina

Gabriel Genellina

On Nov 7, 2:20 am, "Gabriel Genellina" <gagsl-...@yahoo.com.ar> wrote:
> Yes, seems to be a bug. But given the current status of imputil, it's not
> likely to be fixed; certainly not in 2.5 which only gets security fixes
> now.

well, that bug's not the only one. the other one that i found, which
i have been specifically ordered not to report (that or _any_ python
bugs, of which there have been several discovered in the past eight
months), will have to wait until the python developers rescind that
order.

if the python community is lucky, by the time that decision is made, i
will not have forgotten what those bugs are.


> (that is, self.fs_imp.import_from_dir(item, name)). Or perhaps
> item.encode(sys.getdefaultfilesystemencoding()). str(item) definitively
> won't work with directory names containing non-ascii characters.
>
> Why are you using imputil in the first place?

it's an absolutely necessary and integral part of pyjamas-desktop
"platform overrides".

it's absolutely essential to track, in exactly the same manner in
which python "normally" performs importing, and to give the platform-
specific "overrides" a chance to get in there, first.

so, it is absolutely essential to have a correct working version of
imputil.py - and due to the bugs present, and the unwillingness of the
python team to fix those bugs, pyjamas-desktop is forced to maintain a
copy of imputil.py

the "platform" is set to e.g. hulahop, pywebkitgtk or mshtml,
depending on the decision made by the user or the developer to use a
particular browser engine. the platform name is stored in
pyjd.platform in exactly the same way that the system name is stored
in sys.platform.

the way that the platform-specific overrides works is to perform AST
translation of the module, and then to look for the exact same module
but in platform/{modulename}{platformname}.py and perform AST
translation of _that_ module as well.

then, at the top level, any global functions in the platform-specific
AST tree *replace* those in the "main" AST. likewise, a node-walk
along all methods in all classes of the platform-specific AST tree.

in this way, god-awful messy code like this:

Widget.py
class Widget:
def do_something(self):
if platform == 'mshtml':
# do something terrible and ugly
elif platform == 'pywebkitgtk':
# do something only marginally better
else:
# do the default W3C standards-compliant thing

def a_standard_function(self):
# do something normal that all the browser engines get right

can be replaced by three files, each of which encodes *only* the
logic associated with the god-awful ugliness of each browser:

Widget.py
class Widget:
def do_something(self):
# do the default W3C standards-compliant thing

def a_standard_function(self):
# do something normal that all the browser engines get right

platform/Widgetpywebkitgtk.py
class Widget:
def do_something(self):
# do something only marginally better

platform/Widgetmshtml.py
class Widget:
def do_something(self):
# do something terrible and ugly


a similar trick could in fact be deployed to drastically simplify the
layout of e.g. distutils, _espeeecially_ the compiler and linker
modules, by using sys.platform as the "override", or, given that
that's not entirely the whole decision-making process, as i noted when
doing the mingw32 port of python, it would be better to set something
like distutils.platform and to use that.

however, although the technique could be used in the distutils/
ccompiler.py case, the level of complexity of the code (the size of
each "override"), and the small number of actual modules, means that
there isn't actually that much benefit in deploying this AST-
overriding technique.

in the pyjamas API, however, with over 75 user-interface modules, and
over 1200 functions, any one of which could be over-ridden by _eight_
separate platforms, each with their own quirks that can usually be
handled with one or two lines of code, the AST-merging idea that james
tauber came up with is an absolute god-send.

l.

lkcl

lkcl wrote:
> On Nov 7, 2:20 am, "Gabriel Genellina" <gagsl-...@yahoo.com.ar> wrote:
>> Yes, seems to be a bug. But given the current status of imputil, it's not
>> likely to be fixed; certainly not in 2.5 which only gets security fixes
>> now.
>
> well, that bug's not the only one. the other one that i found, which
> i have been specifically ordered not to report

Imputil has been deprecated and has in 3.1 and will-be in 2.7 replaced
by importlib, written in Python. So reporting more bugs in imputil is
rather useless noise. So either use your patched version or try shifting
to the 2.7 importlib and see if it runs well enough on earlier versions.

>(that or _any_ python bugs, of which there have been several
discovered in the past eight
> months), will have to wait until the python developers rescind that
> order.

Without seeing documentary proof, I am dubious that you have been
ordered to not properly report bugs in supported, not deprecated,
modules. If really so, post reports here and interested parties can try
to verify and possibly report them themselves. I, for instance, would be
interested in 3.1 bugs that do not require outside resources to verify.

Terry Jan Reedy

Terry Reedy