Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > Python 2.6 and wrapping C libraries on Windows

Reply
Thread Tools

Python 2.6 and wrapping C libraries on Windows

 
 
L. Lindstrom
Guest
Posts: n/a
 
      04-30-2008
I have read that Python extension modules must link to the same C
run-time as the Python interpreter. This I can appreciate. But does this
requirement extend to the C libraries an extension module wraps. The
case in point is Pygame and SDL. The Pygame extension modules are built
with distutils, so for Python 2.6 using Visual Studio 2008 should ensure
the .pyd files link to msvcr90.dll. But SDL is built using Msys/MinGW
and the configure/make tool chain. This fails when linking to
msvcr90.dll since the small test programs configure builds lack manifest
files. They fail to load msvcr90.dll, raising an R6034 error instead. So
besides heap management and FILE pointers, is there any reason SDL, or
any C dependency, needs to link to the same C run-time as Python? If I
ensure SDL frees memory it allocates and does not directly access a file
opened by Python can I just use another C run-time such as msvcrt?


--
Lenard Lindstrom
"%s@%s.%s" % ('len-l', 'telus', 'net')
 
Reply With Quote
 
 
 
 
Christian Heimes
Guest
Posts: n/a
 
      04-30-2008
L. Lindstrom schrieb:
> I have read that Python extension modules must link to the same C
> run-time as the Python interpreter. This I can appreciate. But does this
> requirement extend to the C libraries an extension module wraps. The
> case in point is Pygame and SDL. The Pygame extension modules are built
> with distutils, so for Python 2.6 using Visual Studio 2008 should ensure
> the .pyd files link to msvcr90.dll. But SDL is built using Msys/MinGW
> and the configure/make tool chain. This fails when linking to
> msvcr90.dll since the small test programs configure builds lack manifest
> files. They fail to load msvcr90.dll, raising an R6034 error instead. So
> besides heap management and FILE pointers, is there any reason SDL, or
> any C dependency, needs to link to the same C run-time as Python? If I
> ensure SDL frees memory it allocates and does not directly access a file
> opened by Python can I just use another C run-time such as msvcrt?
>


Your analysis of the problem and the implication of mixing CRTs is
correct. However ...

It should be trivial to modify the build systemof SDL so that the
manifest is integrated into the DLLs. Everything else is a hack. It
*should* work and in reality it *does* work for most cases. But someday
you'll hit a solid wall and get strange and hard to debug segfaults.

It's in your own interest to get it right in the first place. And you'd
serve the Python community greatly by providing a nice tutorial how to
modify 3rd party builds. *hint*

If you need any help feel free to contact me. The new build system is
mostly my work with help from Martin, Amaury and other core developers.

Christian

 
Reply With Quote
 
 
 
 
sturlamolden
Guest
Posts: n/a
 
      04-30-2008
On Apr 30, 8:06 pm, "L. Lindstrom" <(E-Mail Removed)> wrote:

> I have read that Python extension modules must link to the same C
> run-time as the Python interpreter. This I can appreciate. But does this
> requirement extend to the C libraries an extension module wraps.


This somewhat of a misconception. You cannot reliably mix and blend
CRT resources across different CRTs. This is not really a Python
problem. It applies to any program. The reason this is important for
Python C extensions, is mainly the possibility of accessing a Python
file object as a pointer to a FILE struct in C. If you get a FILE*
pointer from one CRT, you should not pass it to another CRT's fread.
Likewise, if you allocate memory with one CRT's malloc(), you should
not release the memory with another CRT's free(). As long as your
libraries don't share CRT resources, it does not matter that the link
to different CRTs for their internal work.






 
Reply With Quote
 
L. Lindstrom
Guest
Posts: n/a
 
      05-01-2008
Christian Heimes wrote:
> L. Lindstrom schrieb:
>> I have read that Python extension modules must link to the same C
>> run-time as the Python interpreter. This I can appreciate. But does this
>> requirement extend to the C libraries an extension module wraps. The
>> case in point is Pygame and SDL. The Pygame extension modules are built
>> with distutils, so for Python 2.6 using Visual Studio 2008 should ensure
>> the .pyd files link to msvcr90.dll. But SDL is built using Msys/MinGW
>> and the configure/make tool chain. This fails when linking to
>> msvcr90.dll since the small test programs configure builds lack manifest
>> files. They fail to load msvcr90.dll, raising an R6034 error instead. So
>> besides heap management and FILE pointers, is there any reason SDL, or
>> any C dependency, needs to link to the same C run-time as Python? If I
>> ensure SDL frees memory it allocates and does not directly access a file
>> opened by Python can I just use another C run-time such as msvcrt?
>>

>
> Your analysis of the problem and the implication of mixing CRTs is
> correct. However ...
>
> It should be trivial to modify the build systemof SDL so that the
> manifest is integrated into the DLLs. Everything else is a hack. It
> *should* work and in reality it *does* work for most cases. But someday
> you'll hit a solid wall and get strange and hard to debug segfaults.
>
> It's in your own interest to get it right in the first place. And you'd
> serve the Python community greatly by providing a nice tutorial how to
> modify 3rd party builds. *hint*
>
> If you need any help feel free to contact me. The new build system is
> mostly my work with help from Martin, Amaury and other core developers.
>
> Christian
>

Linking to msvcr90.dll is possible with MinGW. The problem is with the
configure scripts. So I can run configure against msvcrt.dll, then
switch to mscvr90.dll for make. If this works I will make SDL and a test
program available on-line so someone can find the appropriate manifests.


--
Lenard Lindstrom
"%s@%s.%s" % ('len-l', 'telus', 'net')
 
Reply With Quote
 
L. Lindstrom
Guest
Posts: n/a
 
      05-01-2008
sturlamolden wrote:
> On Apr 30, 8:06 pm, "L. Lindstrom" <(E-Mail Removed)> wrote:
>
>> I have read that Python extension modules must link to the same C
>> run-time as the Python interpreter. This I can appreciate. But does this
>> requirement extend to the C libraries an extension module wraps.

>
> This somewhat of a misconception. You cannot reliably mix and blend
> CRT resources across different CRTs. This is not really a Python
> problem. It applies to any program. The reason this is important for
> Python C extensions, is mainly the possibility of accessing a Python
> file object as a pointer to a FILE struct in C. If you get a FILE*
> pointer from one CRT, you should not pass it to another CRT's fread.
> Likewise, if you allocate memory with one CRT's malloc(), you should
> not release the memory with another CRT's free(). As long as your
> libraries don't share CRT resources, it does not matter that the link
> to different CRTs for their internal work.
>
>
>
>
>
>

SDL has functions for separating memory allocation and file access.
Going back to msvcrt.dll is a last resort. I may have an idea on how to
build it for msvcr90.dll.

--
Lenard Lindstrom
"%s@%s.%s" % ('len-l', 'telus', 'net')
 
Reply With Quote
 
L. Lindstrom
Guest
Posts: n/a
 
      05-01-2008
L. Lindstrom wrote:
> Christian Heimes wrote:
>> L. Lindstrom schrieb:

[snip]
>>> [B]esides heap management and FILE pointers, is there any reason SDL, or
>>> any C dependency, needs to link to the same C run-time as Python? If I
>>> ensure SDL frees memory it allocates and does not directly access a file
>>> opened by Python can I just use another C run-time such as msvcrt?
>>>

>>
>> Your analysis of the problem and the implication of mixing CRTs is
>> correct. However ...
>>
>> It should be trivial to modify the build systemof SDL so that the
>> manifest is integrated into the DLLs. Everything else is a hack. It
>> *should* work and in reality it *does* work for most cases. But someday
>> you'll hit a solid wall and get strange and hard to debug segfaults.
>>

[snip]
> Linking to msvcr90.dll is possible with MinGW. The problem is with the
> configure scripts. So I can run configure against msvcrt.dll, then
> switch to mscvr90.dll for make. If this works I will make SDL and a test
> program available on-line so someone can find the appropriate manifests.
>
>

Here is my attempt to link SDL and a test program with msvcr90.dll. It
can be found at http://www3.telus.net/len_l/pygame . The md5sum is:

f5b71d9934d35c35a24b668ad8023146 *VC-2008-Run-Time-test.zip

Both lack a manifest. The test program and SDL work when a surrogate
run-time is provided, a renamed msvcr71.dll . So if someone can show me
the necessary manifest to make SDL use the C run-time installed by the
Python 2.6 installer I would appreciate it. SDL is built with MinGW so I
doubt I can distribute the run-time with it.

--
Lenard Lindstrom
"%s@%s.%s" % ('len-l', 'telus', 'net')
 
Reply With Quote
 
illume
Guest
Posts: n/a
 
      05-01-2008
Hi,

after a little research it appears that win9x is not supported by the
msvcr90.dll run time.

Can you confirm this Lenard?

Has anyone tested the new python binaries that link to msvcr90.dll on
win9x machines?

cheers,




On May 1, 3:05 pm, "L. Lindstrom" <(E-Mail Removed)> wrote:
> L. Lindstrom wrote:
> > Christian Heimes wrote:
> >> L. Lindstrom schrieb:

> [snip]
> >>> [B]esides heap management and FILE pointers, is there any reason SDL, or
> >>> any C dependency, needs to link to the same C run-time as Python? If I
> >>> ensure SDL frees memory it allocates and does not directly access a file
> >>> opened by Python can I just use another C run-time such as msvcrt?

>
> >> Your analysis of the problem and the implication of mixing CRTs is
> >> correct. However ...

>
> >> It should be trivial to modify the build systemof SDL so that the
> >> manifest is integrated into the DLLs. Everything else is a hack. It
> >> *should* work and in reality it *does* work for most cases. But someday
> >> you'll hit a solid wall and get strange and hard to debug segfaults.

>
> [snip]
> > Linking to msvcr90.dll is possible with MinGW. The problem is with the
> > configure scripts. So I can run configure against msvcrt.dll, then
> > switch to mscvr90.dll for make. If this works I will make SDL and a test
> > program available on-line so someone can find the appropriate manifests.

>
> Here is my attempt to link SDL and a test program with msvcr90.dll. It
> can be found athttp://www3.telus.net/len_l/pygame. The md5sum is:
>
> f5b71d9934d35c35a24b668ad8023146 *VC-2008-Run-Time-test.zip
>
> Both lack a manifest. The test program and SDL work when a surrogate
> run-time is provided, a renamed msvcr71.dll . So if someone can show me
> the necessary manifest to make SDL use the C run-time installed by the
> Python 2.6 installer I would appreciate it. SDL is built with MinGW so I
> doubt I can distribute the run-time with it.
>
> --
> Lenard Lindstrom
> "%s@%s.%s" % ('len-l', 'telus', 'net')


 
Reply With Quote
 
illume
Guest
Posts: n/a
 
      05-01-2008
hi again,

I should have said, the msvcr90.dll does not work on win9x machines -
as well as not being supported by ms.


cu,



On May 1, 4:02 pm, illume <(E-Mail Removed)> wrote:
> Hi,
>
> after a little research it appears that win9x is not supported by the
> msvcr90.dll run time.
>
> Can you confirm this Lenard?
>
> Has anyone tested the new python binaries that link to msvcr90.dll on
> win9x machines?
>
> cheers,
>
> On May 1, 3:05 pm, "L. Lindstrom" <(E-Mail Removed)> wrote:
>
> > L. Lindstrom wrote:
> > > Christian Heimes wrote:
> > >> L. Lindstrom schrieb:

> > [snip]
> > >>> [B]esides heap management and FILE pointers, is there any reason SDL, or
> > >>> any C dependency, needs to link to the same C run-time as Python? If I
> > >>> ensure SDL frees memory it allocates and does not directly access a file
> > >>> opened by Python can I just use another C run-time such as msvcrt?

>
> > >> Your analysis of the problem and the implication of mixing CRTs is
> > >> correct. However ...

>
> > >> It should be trivial to modify the build systemof SDL so that the
> > >> manifest is integrated into the DLLs. Everything else is a hack. It
> > >> *should* work and in reality it *does* work for most cases. But someday
> > >> you'll hit a solid wall and get strange and hard to debug segfaults.

>
> > [snip]
> > > Linking to msvcr90.dll is possible with MinGW. The problem is with the
> > > configure scripts. So I can run configure against msvcrt.dll, then
> > > switch to mscvr90.dll for make. If this works I will make SDL and a test
> > > program available on-line so someone can find the appropriate manifests.

>
> > Here is my attempt to link SDL and a test program with msvcr90.dll. It
> > can be found athttp://www3.telus.net/len_l/pygame. The md5sum is:

>
> > f5b71d9934d35c35a24b668ad8023146 *VC-2008-Run-Time-test.zip

>
> > Both lack a manifest. The test program and SDL work when a surrogate
> > run-time is provided, a renamed msvcr71.dll . So if someone can show me
> > the necessary manifest to make SDL use the C run-time installed by the
> > Python 2.6 installer I would appreciate it. SDL is built with MinGW so I
> > doubt I can distribute the run-time with it.

>
> > --
> > Lenard Lindstrom
> > "%s@%s.%s" % ('len-l', 'telus', 'net')


 
Reply With Quote
 
Christian Heimes
Guest
Posts: n/a
 
      05-01-2008
illume schrieb:
> Hi,
>
> after a little research it appears that win9x is not supported by the
> msvcr90.dll run time.
>
> Can you confirm this Lenard?
>
> Has anyone tested the new python binaries that link to msvcr90.dll on
> win9x machines?


It doesn't matter to use because Python 2.6 and 3.0 require at least
Windows 2000 SP4. The 9x, ME and NT series aren't supported any more.

Christian

 
Reply With Quote
 
illume
Guest
Posts: n/a
 
      05-01-2008
Ah, why is that?

There's still at least 1.1% of people using win98, if you believe this
source of stats:
http://www.w3schools.com/browsers/browsers_os.asp

I just noticed that win9x winme and win nt are all being dropped from
python.

I know they are old and crufty, but there's still heaps of people
using them.

Someone pointed me to the pep, where the un-support seems planned:
http://www.python.org/dev/peps/pep-0011/


Seems like a lot of people using it, so it's still worthwhile making
2.6 work with win98.





On May 1, 10:09*pm, Christian Heimes <(E-Mail Removed)> wrote:
> illume schrieb:
>
> > Hi,

>
> > after a little research it appears that win9x is not supported by the
> > msvcr90.dll run time.

>
> > Can you confirm thisLenard?

>
> > Has anyone tested the new python binaries that link to msvcr90.dll on
> > win9x machines?

>
> It doesn't matter to use because Python 2.6 and 3.0 require at least
> Windows 2000 SP4. The 9x, ME and NT series aren't supported any more.
>
> Christian


 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
standard libraries don't behave like standard 'libraries' Sriram Srinivasan Python 13 11-12-2009 06:05 PM
Wrapping LabVIEW and DAQmx Libraries for Python xkenneth Python 1 06-08-2009 07:49 PM
How come Python software, tools and libraries is so good? A hug toGvr and the Python community! cnb Python 0 08-27-2008 11:50 PM
Using mandatory libraries (custom class loading vs. expanding libraries) Karsten Wutzke Java 21 06-29-2007 09:25 PM
Tutorial on wrapping C/C++ libraries for use in Ruby? Ritesh Tijoriwala Ruby 0 08-04-2006 06:21 PM



Advertisments