Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > crossplatform py2exe - would it be useful?

Reply
Thread Tools

crossplatform py2exe - would it be useful?

 
 
Thomas Heller
Guest
Posts: n/a
 
      08-06-2003
I'm currently working on a new version of py2exe, which will require
Python 2.3 and later, because it uses the zipimport mechanism.

Since py2exe is a distutils extension, and since C compilers are
commonly available on most platforms except Windows, it would be fairly
easy to let py2exe generate a C source file installing this import hook,
and let distutils' C compiler build an executable file from this.

Would this be useful, or would I be wasting my time, since McMillan
installer, cx_freeze, and tools/freeze already do it?

At the end of this post you'll find excerpts from the readme file, which
is currently the only documentation available.

Thomas

The readme file:

A new and improved py2exe for Python 2.3
========================================

Uses the zipimport mechanism, so it requires Python 2.3 or later. The
zipimport mechanism is able to handle the early imports of the
warnings and also the encodings module which is done by Python.

Creates a single directory, which must be deployed completely.

(Most of this is based on ideas of Mark Hammond Can create any
number of console and gui executables in this directory, plus
optionally a windows service exe, plus optionally an exe and dll com
server. The com servers can expose one or more com object classes.

All pure Python files are contained in a single zip archive, which is
shared by all the executables. The zip archive may also be used by
programs embedding Python. Since extension modules cannot be imported
from zipfiles, a simple pure Python loader is included in the zipfile
which loads the extension from the file system (without requiring that
the directory is in sys.path).

It would be nice if the executables could be run with only a single
sys.path entry containing the absolute filename of the zipfile, but it
seems for dll com servers the executable's directory is also
needed. The absolute filenames are constructed at runtime from the
directory containing the executable, and the zipfile name specified at
build time.

The way has changed how build targets are specified in the setup
script. py2exe installs it own Distribution subclass, which enables
additional keyword arguments to the setup function:

console = [...] # list of scripts to convert into console executables
windows = [...] # list of scripts to convert into gui executables
com_servers = [...] # list of fully qualified class names to build into the exe com server
service = [...] # list of fully qualified class names to build into a service executable
zipfile = "xxx.zip" # filename of the zipfile containing the pure Python modules

All of the above arguments are optional. The zipfile name defaults to
'library.zip'.
 
Reply With Quote
 
 
 
 
Kevin Cazabon
Guest
Posts: n/a
 
      08-06-2003
Hi Thomas;

I've tried Freeze before, but I'm hooked on py2exe because it's so
SIMPLE and flexible, and handles all the weird cases automatically...
For those of us that aren't C gurus, anything that makes the process
easier is welcome.

To clarify your proposal, would you simply build the executable using
the same setup.py script on other platforms too, or would you have to
do more manual steps?

Thanks for the great work so far... py2exe ROCKS! q:]

Kevin.


Thomas Heller <(E-Mail Removed)> wrote in message news:<(E-Mail Removed)>...
> I'm currently working on a new version of py2exe, which will require
> Python 2.3 and later, because it uses the zipimport mechanism.
>
> Since py2exe is a distutils extension, and since C compilers are
> commonly available on most platforms except Windows, it would be fairly
> easy to let py2exe generate a C source file installing this import hook,
> and let distutils' C compiler build an executable file from this.
>
> Would this be useful, or would I be wasting my time, since McMillan
> installer, cx_freeze, and tools/freeze already do it?
>
> At the end of this post you'll find excerpts from the readme file, which
> is currently the only documentation available.
>
> Thomas
>
> The readme file:
>
> A new and improved py2exe for Python 2.3
> ========================================
>
> Uses the zipimport mechanism, so it requires Python 2.3 or later. The
> zipimport mechanism is able to handle the early imports of the
> warnings and also the encodings module which is done by Python.
>
> Creates a single directory, which must be deployed completely.
>
> (Most of this is based on ideas of Mark Hammond Can create any
> number of console and gui executables in this directory, plus
> optionally a windows service exe, plus optionally an exe and dll com
> server. The com servers can expose one or more com object classes.
>
> All pure Python files are contained in a single zip archive, which is
> shared by all the executables. The zip archive may also be used by
> programs embedding Python. Since extension modules cannot be imported
> from zipfiles, a simple pure Python loader is included in the zipfile
> which loads the extension from the file system (without requiring that
> the directory is in sys.path).
>
> It would be nice if the executables could be run with only a single
> sys.path entry containing the absolute filename of the zipfile, but it
> seems for dll com servers the executable's directory is also
> needed. The absolute filenames are constructed at runtime from the
> directory containing the executable, and the zipfile name specified at
> build time.
>
> The way has changed how build targets are specified in the setup
> script. py2exe installs it own Distribution subclass, which enables
> additional keyword arguments to the setup function:
>
> console = [...] # list of scripts to convert into console executables
> windows = [...] # list of scripts to convert into gui executables
> com_servers = [...] # list of fully qualified class names to build into the exe com server
> service = [...] # list of fully qualified class names to build into a service executable
> zipfile = "xxx.zip" # filename of the zipfile containing the pure Python modules
>
> All of the above arguments are optional. The zipfile name defaults to
> 'library.zip'.

 
Reply With Quote
 
 
 
 
Alex Martelli
Guest
Posts: n/a
 
      08-06-2003
Thomas Heller wrote:

> I'm currently working on a new version of py2exe, which will require
> Python 2.3 and later, because it uses the zipimport mechanism.
>
> Since py2exe is a distutils extension, and since C compilers are
> commonly available on most platforms except Windows, it would be fairly
> easy to let py2exe generate a C source file installing this import hook,
> and let distutils' C compiler build an executable file from this.
>
> Would this be useful, or would I be wasting my time, since McMillan
> installer, cx_freeze, and tools/freeze already do it?


I think it would be a WONDERFUL idea: py2exe is the simplest to use
of all the tools you mention, and it would save a lot of ink^H^H^H pixels,
each time I point people to it, to be able to omit the blurb about "if
you're interested in deploying to Windows platform only, then"....


Alex

 
Reply With Quote
 
Oren Tirosh
Guest
Posts: n/a
 
      08-07-2003
On Wed, Aug 06, 2003 at 08:36:20PM +0200, Thomas Heller wrote:
> I'm currently working on a new version of py2exe, which will require
> Python 2.3 and later, because it uses the zipimport mechanism.


Now that zipimport is part of Python the code required for bootstrapping
a py2exe runtime is just:

myscript -c "import sys; sys.path.insert(0, sys.executable); import foo"

This reduces the difference between the custom interpreter supplied with
py2exe and the standard interpreter to just a few lines of C.

The obvious question is - why not go all the way and put this little
hook into the standard Python distribution? This way py2exe could be a
platform-independent pure Python application. In fact, py2exe wouldn't
actually be necessary because anyone could create a zip file manually and
append it to the executable but it's more convenient to have a tool that
automates the process and finds the required dependencies.

Oren

 
Reply With Quote
 
Alex Martelli
Guest
Posts: n/a
 
      08-07-2003
Oren Tirosh wrote:

> On Wed, Aug 06, 2003 at 08:36:20PM +0200, Thomas Heller wrote:
>> I'm currently working on a new version of py2exe, which will require
>> Python 2.3 and later, because it uses the zipimport mechanism.

>
> Now that zipimport is part of Python the code required for bootstrapping
> a py2exe runtime is just:
>
> myscript -c "import sys; sys.path.insert(0, sys.executable); import foo"
>
> This reduces the difference between the custom interpreter supplied with
> py2exe and the standard interpreter to just a few lines of C.
>
> The obvious question is - why not go all the way and put this little
> hook into the standard Python distribution? This way py2exe could be a
> platform-independent pure Python application. In fact, py2exe wouldn't
> actually be necessary because anyone could create a zip file manually and
> append it to the executable but it's more convenient to have a tool that
> automates the process and finds the required dependencies.


Sounds like a good idea to me, if a sensible name is chosen for the
"main module" (I propose 'main'. Take it to Python-Dev...? I even
wonder if it's unobtrusive enough to be considered (as a "bugfix"...
for 2.3.1, rather than having to get into 2.4 and thus wait a LONG time...


Alex


 
Reply With Quote
 
Thomas Heller
Guest
Posts: n/a
 
      08-07-2003
Oren Tirosh <(E-Mail Removed)> writes:

> On Wed, Aug 06, 2003 at 08:36:20PM +0200, Thomas Heller wrote:
>> I'm currently working on a new version of py2exe, which will require
>> Python 2.3 and later, because it uses the zipimport mechanism.

>
> Now that zipimport is part of Python the code required for bootstrapping
> a py2exe runtime is just:
>
> myscript -c "import sys; sys.path.insert(0, sys.executable); import foo"
>


Yes, something like this is also what I am thinking now. And 'myscript'
is just a copy of the standard interpreter. Although the sys.path entry
must be present *before* Py_Initialize is called.

> This reduces the difference between the custom interpreter supplied with
> py2exe and the standard interpreter to just a few lines of C.
>
> The obvious question is - why not go all the way and put this little
> hook into the standard Python distribution? This way py2exe could be a
> platform-independent pure Python application. In fact, py2exe wouldn't
> actually be necessary because anyone could create a zip file manually and
> append it to the executable but it's more convenient to have a tool that
> automates the process and finds the required dependencies.


Yes, and modulefinder is now in the standard library.

OTOH, py2exe does a little bit more: It has a mechanism to supply
modules to include which modulefinder doesn't find, exclude modules
which are unneeded although found, can detect whether Tkinter is used
and copy it, scan (on Windows) extensions for dlls they need (wxPython
needs the wxWindows dll, for example), handle hidden imports from C code
in Python itself and extensions, and so on.

And it works around the fact that extension modules cannot be loaded
from zipfiles, it creates a pure Python loader included in the zip for
them.

Having said that, I would have nothing against py2exe included in the
standard distribution, and the hooks in place.

Thanks,

Thomas
 
Reply With Quote
 
Thomas Heller
Guest
Posts: n/a
 
      08-07-2003
Alex Martelli <(E-Mail Removed)> writes:

> Oren Tirosh wrote:
>
>> On Wed, Aug 06, 2003 at 08:36:20PM +0200, Thomas Heller wrote:
>>> I'm currently working on a new version of py2exe, which will require
>>> Python 2.3 and later, because it uses the zipimport mechanism.

>>
>> Now that zipimport is part of Python the code required for bootstrapping
>> a py2exe runtime is just:
>>
>> myscript -c "import sys; sys.path.insert(0, sys.executable); import foo"
>>
>> This reduces the difference between the custom interpreter supplied with
>> py2exe and the standard interpreter to just a few lines of C.
>>
>> The obvious question is - why not go all the way and put this little
>> hook into the standard Python distribution? This way py2exe could be a
>> platform-independent pure Python application. In fact, py2exe wouldn't
>> actually be necessary because anyone could create a zip file manually and
>> append it to the executable but it's more convenient to have a tool that
>> automates the process and finds the required dependencies.

>
> Sounds like a good idea to me, if a sensible name is chosen for the
> "main module" (I propose 'main'.


My choice would have been __main__ Is it really the correct way to
'import __main__' instead of 'running' it?

> Take it to Python-Dev...? I even wonder if it's unobtrusive enough to
> be considered (as a "bugfix"... for 2.3.1, rather than having to
> get into 2.4 and thus wait a LONG time...
>
>
> Alex


Thomas
 
Reply With Quote
 
Alex Martelli
Guest
Posts: n/a
 
      08-07-2003
Thomas Heller wrote:
...
>>> myscript -c "import sys; sys.path.insert(0, sys.executable); import foo"

...
>> Sounds like a good idea to me, if a sensible name is chosen for the
>> "main module" (I propose 'main'.

>
> My choice would have been __main__ Is it really the correct way to
> 'import __main__' instead of 'running' it?


Well, most main scripts ARE coded with the "if __name__=='__main__':"
convention, after all, so an "import __main__" can be seen as a way
to just piggyback on that existing convention rather than inventing a
new one in addition. So, I concede it's better than "import main".


Alex

 
Reply With Quote
 
Thomas Heller
Guest
Posts: n/a
 
      08-07-2003
Alex Martelli <(E-Mail Removed)> writes:

> Thomas Heller wrote:
> ...
>>>> myscript -c "import sys; sys.path.insert(0, sys.executable); import foo"

> ...
>>> Sounds like a good idea to me, if a sensible name is chosen for the
>>> "main module" (I propose 'main'.

>>
>> My choice would have been __main__ Is it really the correct way to
>> 'import __main__' instead of 'running' it?

>
> Well, most main scripts ARE coded with the "if __name__=='__main__':"
> convention, after all, so an "import __main__" can be seen as a way
> to just piggyback on that existing convention rather than inventing a
> new one in addition. So, I concede it's better than "import main".
>


How would the hook be triggered? The zipimporter code would probably add
argv[0] to sys.path, and then try to 'import __main__' or something like
this. The problem is that a standalone executable python would have to
disable the standard Python command line flags and environment
variables, but they are parse *before* Py_Initialize() is called.

And I hope that the options set by the command line flags and env vars
should now come from the __main__ script itself.

Thomas
 
Reply With Quote
 
Alex Martelli
Guest
Posts: n/a
 
      08-07-2003
Thomas Heller wrote:

> Alex Martelli <(E-Mail Removed)> writes:
>
>> Thomas Heller wrote:
>> ...
>>>>> myscript -c "import sys; sys.path.insert(0, sys.executable); import
>>>>> foo"

>> ...
>>>> Sounds like a good idea to me, if a sensible name is chosen for the
>>>> "main module" (I propose 'main'.
>>>
>>> My choice would have been __main__ Is it really the correct way to
>>> 'import __main__' instead of 'running' it?

>>
>> Well, most main scripts ARE coded with the "if __name__=='__main__':"
>> convention, after all, so an "import __main__" can be seen as a way
>> to just piggyback on that existing convention rather than inventing a
>> new one in addition. So, I concede it's better than "import main".
>>

>
> How would the hook be triggered? The zipimporter code would probably add
> argv[0] to sys.path, and then try to 'import __main__' or something like
> this. The problem is that a standalone executable python would have to
> disable the standard Python command line flags and environment
> variables, but they are parse *before* Py_Initialize() is called.


Ah, yes, good point. So, the executable needs to know whether to do
the usual commandline and environment processing, or not, _before_
calling Py_Inizialize. One approach might be to trigger this based
on the executable's own *name* -- do the full commandline and environment
processing if and only if the executable's name starts with (case-
insensitive, probably, to be safe...) the six letters 'python', but
not otherwise. There are, no doubt, other alternative ways, too, but
this one seems dirt-simple and practically sufficient.


> And I hope that the options set by the command line flags and env vars
> should now come from the __main__ script itself.


I'm not sure I understand what you mean. Anyway, I do see that if
my 'foobar.exe' is a python.exe + appended zipfile, then running
'foobar -i' should just put '-i' in sys.argv[1], and NOT gobble it up
to mean "enter interactive mode", for example.


Alex

 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
crossplatform standalone python apps Gabriel Rossetti Python 4 10-23-2008 08:26 PM
If you'd like to see c.l.c++.crossplatform Tomás Ó hÉilidhe C++ 3 12-11-2007 03:55 PM
Crossplatform (Windows/Linux) RMI System?? Hotlips C++ 0 02-28-2007 08:35 PM
crossplatform curses-style apps copx Python 1 09-28-2004 10:15 PM
looking for crossplatform layer between gui and database Alois Weber C++ 1 04-17-2004 07:44 PM



Advertisments