Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > Threading issue: [Error 9]: bad file descriptor

Reply
Thread Tools

Threading issue: [Error 9]: bad file descriptor

 
 
Kevin
Guest
Posts: n/a
 
      07-06-2003
Has anyone else run into random IOErrors ([Error 9] bad file descriptor) in
multi-threaded Python apps?

I've searched the newsgroups, and the only references I can find to that
seem to be sporadically related to threading, but nobody seems to have a
"fix" or even a clear understanding of what the issue is.

The app I'm running into this with (once in a while... not really repeatable
predictably) is using 3 threads, and I'm using locks around all shared
objects. The error seems to be happening mostly when I'm trying to delete a
temporary file that was originally created by a different thread.

ANy suggestions? Thanks,

Kevin.


 
Reply With Quote
 
 
 
 
Andrew MacIntyre
Guest
Posts: n/a
 
      07-06-2003
On Sun, 6 Jul 2003, Kevin wrote:

> Has anyone else run into random IOErrors ([Error 9] bad file descriptor) in
> multi-threaded Python apps?


Not indicating which of the platforms with Python supported threads you're
experiencing this on doesn't give much scope for others to help...

--
Andrew I MacIntyre "These thoughts are mine alone..."
E-mail: http://www.velocityreviews.com/forums/(E-Mail Removed) (pref) | Snail: PO Box 370
(E-Mail Removed) (alt) | Belconnen ACT 2616
Web: http://www.andymac.org/ | Australia

 
Reply With Quote
 
 
 
 
Aahz
Guest
Posts: n/a
 
      07-06-2003
In article <KaQNa.394960$(E-Mail Removed) >,
Kevin <(E-Mail Removed)> wrote:
>
>Has anyone else run into random IOErrors ([Error 9] bad file
>descriptor) in multi-threaded Python apps?
>
>The app I'm running into this with (once in a while... not really
>repeatable predictably) is using 3 threads, and I'm using locks around
>all shared objects. The error seems to be happening mostly when I'm
>trying to delete a temporary file that was originally created by a
>different thread.


Sounds like the problem isn't locking per se, but lack of
synchronization. The simple answer: never use an external object in
multiple threads.
--
Aahz ((E-Mail Removed)) <*> http://www.pythoncraft.com/

"Not everything in life has a clue in front of it...." --JMS
 
Reply With Quote
 
Cliff Wells
Guest
Posts: n/a
 
      07-07-2003
On Sun, 2003-07-06 at 22:37, Kevin wrote:
> My oversight, thanks!
>
> I'm testing on Windows 2000 Professional, but have had other users report
> the same problem on other Windows flavours (NT, 9.


I experienced a similar problem a few years ago (on Windows NT Server).
A long-running threaded application would sporadically raise an
exception when writing to a log file. Never figured out the cause, but
"solved" it by catching the exception and reopening the file (IIRC).

Regards,

--
Cliff Wells, Software Engineer
Logiplex Corporation (www.logiplex.net)
(503) 978-6726 (800) 735-0555


 
Reply With Quote
 
David Bolen
Guest
Posts: n/a
 
      07-07-2003
"Kevin" <(E-Mail Removed)> writes:

> My oversight, thanks!
>
> I'm testing on Windows 2000 Professional, but have had other users report
> the same problem on other Windows flavours (NT, 9.


I can only speak for NT, but I have seen timing issues in the past
where there is some additional latency imposed by the system between
when I release all references to a file and when it can be removed
from the filesystem by the same process. This has affected me in C
code as well as Python. Perhaps you're running into the same thing,
although for me it normally showed up as a permission denied error.

Sometimes the higher level Python I/O exceptions can be a bit vague
(by their nature, a lot of underlying Win32 error codes eventually get
translated into a far fewer number of C RTL error codes, which in turn
bubble up as Python exceptions). One thing you could try, depending
on the operation in question, is to replace the Python operation (such
as os.remove) with a matching win32all module operation (such as
win32file.DeleteFile), which should raise a more specific exception.

In my past experiences, the most practical, albeit inelegant,
workaround was to retry a failed removal operation after several
seconds. Definitely a kluge, but I was never able to isolate any more
well-defined approach.

This of course assumes that you don't really have a threading race
condition under which you still do hold active handles to the file
that you are trying to remove. The fact that you're getting back an
invalid handle error certainly makes it seem that you make still be
looking at a race condition or inter-thread data corruption within
your application.

Depending on the structure of the application, if you can wrap (or if
you have already) all access to your file handles through a common
class you should be able to instrument it in order to at least get
some sort of trace as to how they are accessed, and perhaps catch one
of the failures. But as you probably know, debugging this sort of
thing can be nasty, particularly if it's truly sporadic in nature.

-- David
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
socket and threading ... bad file descriptor hg Python 1 09-14-2006 12:14 PM
threading.Event file descriptor Nicolas Fleury Python 4 04-04-2005 12:02 AM
file.readlines() - gives me error (bad file descriptor) wordsender@gmail.com Python 9 01-06-2005 02:02 PM
RE: file.readlines() - gives me error (bad file descriptor) Vishnu Python 0 01-06-2005 06:33 AM
File::Copy giving "Bad File Descriptor" from sysread Yary H Perl Misc 0 10-03-2004 05:33 PM



Advertisments