Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > When does a binary extension gets the file extension '.pyd' and whenis it '.so'

Reply
Thread Tools

When does a binary extension gets the file extension '.pyd' and whenis it '.so'

 
 
llothar
Guest
Posts: n/a
 
      04-04-2008
On windows everything is '.pyd' but there seems to be two ways to get
this on unix?
Why and what is the rule?
 
Reply With Quote
 
 
 
 
Fredrik Lundh
Guest
Posts: n/a
 
      04-04-2008
llothar wrote:

> On windows everything is '.pyd' but there seems to be two ways to get
> this on unix?


If you attempt to import the module "spam" on Windows, Python looks for
"spam.dll" and "spam.pyd" (in addition to "spam.py/spam.pyw/spam.pyc" etc)

On most Unix platforms, Python looks for "spam.so" and "spammodule.so".

You can check what suffixes a given Python version uses via the "imp"
module:

>>> import imp
>>> imp.get_suffixes()


To see *where* Python is looking as well, use the "-vv" flag:

$ python -vv -c "import spam"
...
# trying spam.so
# trying spammodule.so
# trying spam.py
# trying spam.pyc
... etc

(-vv prints loads of stuff, so you may want to use "grep" to filter out
the stuff you're interested in:

$ python -vv -c "import spam" 2>&1 | grep spam

or, under Windows:

> python -vv -c "import spam" 2> out.txt
> findstr spam out.txt


)

> Why and what is the rule?


If you want Python to be able to import your binary extension, make sure
to use a name it looks for.

</F>

 
Reply With Quote
 
 
 
 
llothar
Guest
Posts: n/a
 
      04-05-2008
Thanks, my question was not how can i make python to it find. I don't
have a problem.

My question was: Why does setup.py generated sometimes a pyd and
sometimes a so file?

There must be a rule behind this.

Unforunately setup.py is not well documented. Here i mean i need a
specification not a
tutorial, because i want to know something not do something.

Don't ask why i need to know: I need to know. And maybe somebody here
can drop a
line before i have to dig around in the source code.
 
Reply With Quote
 
Fredrik Lundh
Guest
Posts: n/a
 
      04-05-2008
llothar wrote:

> My question was: Why does setup.py generated sometimes a pyd and
> sometimes a so file?


setup.py picks an extension that happens to work on the platform you're
running setup.py on. doing otherwise would be pretty pointless.

</F>

 
Reply With Quote
 
llothar
Guest
Posts: n/a
 
      04-05-2008
On 5 Apr., 15:48, Fredrik Lundh <(E-Mail Removed)> wrote:
> llothar wrote:
> > My question was: Why does setup.py generated sometimes a pyd and
> > sometimes a so file?

>
> setup.py picks an extension that happens to work on the platform you're
> running setup.py on. doing otherwise would be pretty pointless.
>
> </F>


Unfortunately as pointless as the answers i got so far.


Okay i try it one more time:

I ship an application that compiles an python interpreter and
extension on a remote system.
It also needs to copy this created items around. So if i use setup.py
to create an
extension i need to know the file name of the generated file.

Damned this is trivial and a fundamental question and it is not
documented anywhere.

I have a clue at the moment that it might be ".so" when python is
compiled without shared library
and ".pyd" otherwise (configure option --enable-shared) . But this is
just a guess. Does anybody know?

And by the way: I think this is a bug and should be fixed. If the
platform does allow renaming the
extension of a DLL (does HP/UX allow this?) it should always be
".pyd"
 
Reply With Quote
 
Fredrik Lundh
Guest
Posts: n/a
 
      04-05-2008
llothar wrote:

> I ship an application that compiles an python interpreter and
> extension on a remote system.
> It also needs to copy this created items around. So if i use setup.py
> to create an
> extension i need to know the file name of the generated file.


so why not just ask setup.py to copy the files for you?

you can either use "install" with the --home option to copy all the
files to a given directory, or create a binary kit with "bdist" and
unpack the resulting archive at the target location.

> Unfortunately as pointless as the answers i got so far.


well, cutting the "I'm not going to tell you why I need to know this"
stuff might help people come up with solutions to your actual problem
instead of posting stuff that's informative but misses the point.

</F>

 
Reply With Quote
 
Steve Holden
Guest
Posts: n/a
 
      04-05-2008
llothar wrote:
> On 5 Apr., 15:48, Fredrik Lundh <(E-Mail Removed)> wrote:
>> llothar wrote:
>>> My question was: Why does setup.py generated sometimes a pyd and
>>> sometimes a so file?

>> setup.py picks an extension that happens to work on the platform you're
>> running setup.py on. doing otherwise would be pretty pointless.
>>
>> </F>

>
> Unfortunately as pointless as the answers i got so far.
>

Right, so you think people aren't trying to help you?

Put your paranoia back in your pocket
>
> Okay i try it one more time:
>
> I ship an application that compiles an python interpreter and
> extension on a remote system.
> It also needs to copy this created items around. So if i use setup.py
> to create an
> extension i need to know the file name of the generated file.
>
> Damned this is trivial and a fundamental question and it is not
> documented anywhere.
>
> I have a clue at the moment that it might be ".so" when python is
> compiled without shared library
> and ".pyd" otherwise (configure option --enable-shared) . But this is
> just a guess. Does anybody know?
>
> And by the way: I think this is a bug and should be fixed. If the
> platform does allow renaming the
> extension of a DLL (does HP/UX allow this?) it should always be
> ".pyd"


You display your ignorance here. The ".pyd" extension is used on Windows
as an alternative to ".dll", but both are recognized as shared
libraries. Personally I'm not really sure why they even chose to use
".pyd", which is confusing to most Windows users. In UNIX/Linux
environments ".so" is the standard extension for a shared library.

To depart from the platform standard would be unhelpful and confusing to
the majority of users. It's know use telling us what you think: tell us
instead the compelling reasons why your opinion is correct. Opinions,
after all, are so cheap that everyone can have one.

There are ways to build distributions of Python extensions (modules or
packages involving binary code from languages like C or C++), but you
will want to understand a bit more about computing in general (and work
on your social skills before you start to approach them.

regards
Steve
--
Steve Holden +1 571 484 6266 +1 800 494 3119
Holden Web LLC http://www.holdenweb.com/

 
Reply With Quote
 
Fredrik Lundh
Guest
Posts: n/a
 
      04-05-2008
Steve Holden wrote:

> You display your ignorance here. The ".pyd" extension is used on Windows
> as an alternative to ".dll", but both are recognized as shared
> libraries. Personally I'm not really sure why they even chose to use
> ".pyd", which is confusing to most Windows users. In UNIX/Linux
> environments ".so" is the standard extension for a shared library.


background:

http://mail.python.org/pipermail/pyt...er/001456.html

"Python cannot import every .dll file. Only .dll files that conform to
the convention for Python extension modules can be imported. (The
convention is that it must export an init<module> function.)

On most other platforms, shared libraries must have a specific
extension (e.g. .so on most Unix). Python allows you to drop such a
file into any directory where is looks for modules, and it will then
direct the dynamic load support to load that specific file.

This seems logical -- Python extensions must live in directories that
Python searches (Python must do its own search because the search
order is significant).

On Windows, Python uses the same strategy. The only modification is
that it is allowed to give the file a different extension, namely
..pyd, to indicate that this really is a Python extension and not a
regular DLL. This was mostly introduced because it is apparently
common to have an existing DLL "foo.dll" and write a Python wrapper
for it that is also called "foo". Clearly, two files foo.dll are too
confusing, so we let you name the wrapper foo.pyd. But because the
file format is essentially that of a DLL, we don't *require* this
renaming; some ways of creating DLLs in the first place may make it
difficult to do."

</F>

 
Reply With Quote
 
llothar
Guest
Posts: n/a
 
      04-05-2008
> Right, so you think people aren't trying to help you?

I think they are not reading the question.

> You display your ignorance here. The ".pyd" extension is used on Windows
> as an alternative to ".dll", but both are recognized as shared
> libraries. Personally I'm not really sure why they even chose to use
> ".pyd", which is confusing to most Windows users.


Here i agree. But having it's own identifiying extension has also some
small benefits.

> To depart from the platform standard would be unhelpful and confusing to
> the majority of users. It's know use telling us what you think: tell us
> instead the compelling reasons why your opinion is correct. Opinions,
> after all, are so cheap that everyone can have one.


Because i want a uniform solution. Either use "dllso" or use "pyd"
but
stay with one decision once made. At the moment when i build python on
my ubuntu system without "--enable-shared" i get a pyd file created,
if
i use "--enable-shared" it is a so file.

I don't know if this is a special case on Linux or the general on unix
systems
(i only have Linux, FreeBSD, NetBSD, OpenBSD, Solaris and MacOSX in
mind).

> There are ways to build distributions of Python extensions (modules or
> packages involving binary code from languages like C or C++), but you
> will want to understand a bit more about computing in general


Believe me nobody needs to teach me anything about general programming
anymore.

> (and work on your social skills


I don't think so. I asked a pretty simple question and as usual on
usenet nobody
read the question but answered to complete different topics. Answers
on usenet are
so cheap, everybody likes to give one - no matter if it is ontopic,
right or wrong.
And this does not really help.

My question is simple and person who knows setup.py and distools would
be able to
give the answer in a small sentence if there is a strategy behind it
and it's not
only a bug.

Unfortunately there is no python.core mailing list that i know so i
ask here.


 
Reply With Quote
 
Fredrik Lundh
Guest
Posts: n/a
 
      04-05-2008
llothar wrote:

> I don't think so. I asked a pretty simple question and as usual on
> usenet nobody read the question


did *you* read your own question? it took you three posts before you
mentioned what you were trying to do, and four posts before you bothered
to mention that you're seeing this behaviour on Ubuntu.

and for the record, Python doesn't look for PYD files on any of the Unix
boxes I have convenient access to right now. what Ubuntu version are
you using, what Python version do you have, and what does

$ python -c "import imp; print imp.get_suffixes()"

print on your machine?

</F>

 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Return of gets gets John Joyce Ruby 0 04-23-2007 01:38 PM
gets gets John Joyce Ruby 2 03-26-2007 04:00 PM
XMLHttpRequest gets data to display in element but data gets appended not replaces previous! libsfan01 Javascript 5 12-20-2006 06:25 AM
How to convert a .txt file extension to a .xls file extension? Steve ASP .Net 3 08-25-2006 05:43 PM
Not only the selected HREF gets surrounded, but the whole row gets surrounded Stefan Mueller HTML 5 07-10-2006 11:53 AM



Advertisments