Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > How does one get from "ImportError: DLL load failed:..." to a culprit.dll and symbol?

Reply
Thread Tools

How does one get from "ImportError: DLL load failed:..." to a culprit.dll and symbol?

 
 
Chris Cormie
Guest
Posts: n/a
 
      02-23-2009
Hi,

I've been Googling around on a moderately common Windows Python problem:
a mismatch between the symbols a python extension thinks are available
and the contents of the associated DLL. Python users running into this
problem are greeted with:

import <somemodule>
"ImportError: DLL load failed: The specified procedure could not be found."

That's it: no mention of which dll nor which symbol is causing the
problem. Both these pieces of information are helpful/important in
resolving this class of problem: how does one get Python to dump this
debugging data?

(>python -v -d
doesn't help, BTW)

Regards,
Chris
 
Reply With Quote
 
 
 
 
John Machin
Guest
Posts: n/a
 
      02-23-2009
On Feb 23, 2:56*pm, Chris Cormie <(E-Mail Removed)> wrote:
> Hi,
>
> I've been Googling around on a moderately common Windows Python problem:
> a mismatch between the symbols a python extension thinks are available
> and the contents of the associated DLL. Python users running into this
> problem are greeted with:
>
> import <somemodule>
> "ImportError: DLL load failed: The specified procedure could not be found.."
>
> That's it: no mention of which dll nor which symbol is causing the
> problem. Both these pieces of information are helpful/important in
> resolving this class of problem: how does one get Python to dump this
> debugging data?
>
> (>python -v -d
> doesn't help, BTW)


Use -vv ... you need to know exactly which of possibly many
incarnations of somemodule.pyd Python is trying to import from.

Then get a copy of the Dependency Walker from http://www.dependencywalker.com/

Open c:\the\failing\somemodule.pyd with the DW. Look at grumble
messages in the bottom pane. Note: you often get 1 or 2 warning
messages with a pyd that's not having problems. Scroll through all the
modules in the middle pane, looking for grumbles.

If that not-very-technical description [all I've ever needed] doesn't
help, you'll need to read the DW help file (HTFF1K) or wait till
someone who knows what they are doing comes along

By the way, I presume that you are aware that extensions on Windows
are tied to a particular version of Python ... you would have got a
different error message if you had a version mismatch (I think). If
your problem is not obvious, come back here with:
-- what version of Python you are running
-- what DLLs you find in the middle pane whose names match
(1) MSVC*.DLL
(2) PYTHON??.DLL

You may need to answer questions about the extension (e.g. compiler/
linker options used) -- is it your extension or a 3rd party's?

HTH,
John
 
Reply With Quote
 
 
 
 
Chris Cormie
Guest
Posts: n/a
 
      02-23-2009

> If that not-very-technical description [all I've ever needed] doesn't
> help, you'll need to read the DW help file (HTFF1K) or wait till
> someone who knows what they are doing comes along


LOL, I am that person
Your technique works well and it does provide the information and it is
a (roundabout) solution to this class of problems: thank you very much.
It doesn't answer the original question so I'll restate it:

How do you get *Python* to tell you the dll and the problem symbol? Not
external tools, Python. Python at a low level is calling LoadLibrary and
GetProcAddress to resolve symbols and the call fails. At that point it
has the name of the dll and the problem symbol to hand and yet strangely
only gives an opaque error message. How does one get Python to print out
the faulty DLL and symbol?

(Even if it means a debug build of Python and adding the printf
yourself, I just want to know where we stand on this issue! )

Best Regards,
Chris
 
Reply With Quote
 
John Machin
Guest
Posts: n/a
 
      02-23-2009
On Feb 23, 11:41*pm, Chris Cormie <(E-Mail Removed)> wrote:
> > If that not-very-technical description [all I've ever needed] doesn't
> > help, you'll need to read the DW help file (HTFF1K) or wait till
> > someone who knows what they are doing comes along

>
> LOL, I am that person


It wasn't apparent, and still isn't.

> Your technique works well and it does provide the information and it is
> a (roundabout) solution to this class of problems: *thank you very much..
> It doesn't answer the original question so I'll restate it:
>
> How do you get *Python* to tell you the dll and the problem symbol? Not
> external tools, Python. Python at a low level is calling LoadLibrary and
> GetProcAddress to resolve symbols and the call fails. At that point it
> has the name of the dll and the problem symbol to hand and yet strangely
> only gives an opaque error message. How does one get Python to print out
> the faulty DLL and symbol?


If you know all that, then the answer is to submit a patch, isn't it??


> (Even if it means a debug build of Python and adding the printf
> yourself,


printf? Maybe you shouldn't submit a patch after all.

 
Reply With Quote
 
Gabriel Genellina
Guest
Posts: n/a
 
      02-23-2009
En Mon, 23 Feb 2009 10:41:20 -0200, Chris Cormie
<(E-Mail Removed)> escribió:

>> If that not-very-technical description [all I've ever needed] doesn't
>> help, you'll need to read the DW help file (HTFF1K) or wait till
>> someone who knows what they are doing comes along

>
> LOL, I am that person
> Your technique works well and it does provide the information and it is
> a (roundabout) solution to this class of problems: thank you very much.
> It doesn't answer the original question so I'll restate it:
>
> How do you get *Python* to tell you the dll and the problem symbol? Not
> external tools, Python. Python at a low level is calling LoadLibrary and
> GetProcAddress to resolve symbols and the call fails. At that point it
> has the name of the dll and the problem symbol to hand and yet strangely
> only gives an opaque error message. How does one get Python to print out
> the faulty DLL and symbol?


You can't, because it isn't Python who's trying to load the symbol - the
*only* symbol that Python attempts to import itself is "initFOO" from
FOO.pyd, and when that fails you get an explicit message.
The error you see must come from the extension itself, and propagates into
Python as an ImportError.

--
Gabriel Genellina

 
Reply With Quote
 
Mark Hammond
Guest
Posts: n/a
 
      02-23-2009
On 23/02/2009 11:41 PM, Chris Cormie wrote:
>
>> If that not-very-technical description [all I've ever needed] doesn't
>> help, you'll need to read the DW help file (HTFF1K) or wait till
>> someone who knows what they are doing comes along

>
> LOL, I am that person


LOL sounds right!

> How do you get *Python* to tell you the dll and the problem symbol? Not
> external tools, Python. Python at a low level is calling LoadLibrary and
> GetProcAddress to resolve symbols and the call fails.


It is the LoadLibrary that is failing; it should be obvious that if it
was a simple GetProcAddress that was failing, Python would simply throw
an exception rather than displaying the ugly dialog box you see.

The problem is that some *other* DLL in the chain of dependencies is
failing to load; please re-adjust your perceptions of your own knowledge
and re-read John's initial response - if you follow his instructions
that should all become quite obvious.

Mark
 
Reply With Quote
 
Chris Cormie
Guest
Posts: n/a
 
      02-24-2009
Mark Hammond wrote:
> On 23/02/2009 11:41 PM, Chris Cormie wrote:
>>
>>> If that not-very-technical description [all I've ever needed] doesn't
>>> help, you'll need to read the DW help file (HTFF1K) or wait till
>>> someone who knows what they are doing comes along

>>
>> LOL, I am that person

>
> LOL sounds right!
>
>> How do you get *Python* to tell you the dll and the problem symbol? Not
>> external tools, Python. Python at a low level is calling LoadLibrary and
>> GetProcAddress to resolve symbols and the call fails.

>
> It is the LoadLibrary that is failing; it should be obvious that if it
> was a simple GetProcAddress that was failing, Python would simply throw
> an exception rather than displaying the ugly dialog box you see.


I'm talking about the whole class of errors when loading an extension,
the LoadLibary succeeds but the GetProcAddress fails. Not one specific
error, a class of errors and *by definition* from the original question
LoadLibrary succeeds. That's the point: at some point in this common
scenario we have the path to the dll and a symbol we are resolving, and
the symbol name, yet for whatever reason the GetProcAddress lookup
fails, resulting in a surprisingly uninformative errors message.

Regards,
Chris.
 
Reply With Quote
 
Chris Cormie
Guest
Posts: n/a
 
      02-24-2009
Gabriel Genellina wrote:
> En Mon, 23 Feb 2009 10:41:20 -0200, Chris Cormie
> <(E-Mail Removed)> escribió:
>
>>> If that not-very-technical description [all I've ever needed] doesn't
>>> help, you'll need to read the DW help file (HTFF1K) or wait till
>>> someone who knows what they are doing comes along

>>
>> LOL, I am that person
>> Your technique works well and it does provide the information and it
>> is a (roundabout) solution to this class of problems: thank you very
>> much. It doesn't answer the original question so I'll restate it:
>>
>> How do you get *Python* to tell you the dll and the problem symbol?
>> Not external tools, Python. Python at a low level is calling
>> LoadLibrary and GetProcAddress to resolve symbols and the call fails.
>> At that point it has the name of the dll and the problem symbol to
>> hand and yet strangely only gives an opaque error message. How does
>> one get Python to print out the faulty DLL and symbol?

>
> You can't, because it isn't Python who's trying to load the symbol - the
> *only* symbol that Python attempts to import itself is "initFOO" from
> FOO.pyd, and when that fails you get an explicit message.
> The error you see must come from the extension itself, and propagates
> into Python as an ImportError.


Right -- now we are getting somewhere! That makes a lot of sense.

What I was expecting to see next was an explicit GetProcAddress() call
in the extension code that was failing in initFOO() or elsewhere in the
extension code. But there are no such calls.

I think what might have been making this confusing is that the root of
my example problem is link time.

Let me explain by giving the specific example case I have to hand.
I orginally got on to this class of opaque runtime errors caused by
missing symbols through building the pycURL extension on MinGW. As of
Python 2.6, Python links against the new msvcr90.dll C Runtime. MinGW
maintains a set of .a files for system DLLs including the C Runtime.
When building pycURL naturally I want to link against libmsvcr90.a so
Python 2.6 and pycURL are using the same C Rutime. Problem was, MinGW's
.a was wrong: it exports symbols that msvcr90.dll does not in fact
possess. Building pycURL with the faulty libmsvcr90.a causes the opaque
error message: "ImportError: DLL load failed"

Now I expected this meant an explict LoadLibrary() GetProcAddress()
sequence was failing in the extension but perhaps that's not what
happens at all?
Chris




 
Reply With Quote
 
Chris Cormie
Guest
Posts: n/a
 
      02-24-2009
John Machin wrote:
> On Feb 23, 11:41 pm, Chris Cormie <(E-Mail Removed)> wrote:
>>> If that not-very-technical description [all I've ever needed] doesn't
>>> help, you'll need to read the DW help file (HTFF1K) or wait till
>>> someone who knows what they are doing comes along

>> LOL, I am that person

>
> It wasn't apparent, and still isn't.


Yikes, enough with the rudeness people. I'm asking a reasonable question
in a good faith.
 
Reply With Quote
 
Chris Cormie
Guest
Posts: n/a
 
      02-24-2009
Mark Hammond wrote:
> On 23/02/2009 11:41 PM, Chris Cormie wrote:
>>
>>> If that not-very-technical description [all I've ever needed] doesn't
>>> help, you'll need to read the DW help file (HTFF1K) or wait till
>>> someone who knows what they are doing comes along

>>
>> LOL, I am that person

>
> LOL sounds right!
>
>> How do you get *Python* to tell you the dll and the problem symbol? Not
>> external tools, Python. Python at a low level is calling LoadLibrary and
>> GetProcAddress to resolve symbols and the call fails.

>
> It is the LoadLibrary that is failing; it should be obvious that if it
> was a simple GetProcAddress that was failing, Python would simply throw
> an exception rather than displaying the ugly dialog box you see.
>
> The problem is that some *other* DLL in the chain of dependencies is
> failing to load; please re-adjust your perceptions of your own knowledge
> and re-read John's initial response - if you follow his instructions
> that should all become quite obvious.
>
> Mark


I think an example would help rather than me jabbering on about
LoadLibary/GetProcAddress .

It really is a case of Python loads module x depending on DLL y that is
missing symbol z, yet the error message is just "load x failed" when in
theory it could be "load x failed: y doesn't have z".

However I think I am confusing people (and myself!) by assuming it's
always as explicit as a LoadLibary/GetProcAddress sequence that is
failing when people see this error.

Let me explain by giving the specific example case I have to hand.
I originally got on to this class of opaque runtime errors caused by
missing symbols through building the pycURL extension on MinGW. As of
Python 2.6, Python links against the new msvcr90.dll C Runtime. MinGW
maintains a set of .a files for system DLLs including the C Runtime.
When building pycURL naturally I want to link against libmsvcr90.a so
Python 2.6 and pycURL are using the same C Runtime. Problem was, MinGW's
.a was wrong: it exports symbols that msvcr90.dll does not in fact
possess. Building pycURL with the faulty libmsvcr90.a causes the opaque
error message: "ImportError: DLL load failed"

I think Gabriel's got the part of the answer: she said "it isn't Python
who's trying to load the symbol - the *only* symbol that Python attempts
to import itself is "initFOO" from FOO.pyd"

So my next question is: given the above missing-symbol problem, is the
system able/willing to report the bad symbol reference in the .pyd when
Python attempts to import initFOO?

Regards,
Chris.

 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Re: How include a large array? Edward A. Falk C Programming 1 04-04-2013 08:07 PM
How to determine if a DLL is a COM DLL or .NET DLL Anushi ASP .Net 5 10-28-2004 01:59 PM
Why does Ruby use both tcl83.dll and tk83.dll (instead of just tk83.dll)? H. Simpson Ruby 4 08-03-2004 04:45 PM
mprapi.dll --> samlib.dll --> ntdll.dll issue. Some1 Computer Support 4 04-05-2004 02:02 AM
msvcrt.dll, msvcirt.dll, msvcrt20.dll and msvcrt40.dll, explanation please! Snoopy NZ Computing 16 08-25-2003 12:34 PM



Advertisments