Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > bug with os.rename in 2.4.1?

Reply
Thread Tools

bug with os.rename in 2.4.1?

 
 
t123
Guest
Posts: n/a
 
      04-28-2009
I was wondering if anyone has seen this problem before?

I have a job that runs hourly. Part of the processing is to rename a
file using os.rename. The file name is the same every hour.
Sometimes the os.rename fails and the file does not get renamed. It
only happens occasionally. Most of the time it works just fine.

Appreciate any ideas.

Thanks
 
Reply With Quote
 
 
 
 
Daniel Fetchinson
Guest
Posts: n/a
 
      04-28-2009
> I was wondering if anyone has seen this problem before?
>
> I have a job that runs hourly. Part of the processing is to rename a
> file using os.rename. The file name is the same every hour.
> Sometimes the os.rename fails and the file does not get renamed. It
> only happens occasionally. Most of the time it works just fine.


Unless you send a minimal version of the code you use and the complete
traceback when an error occurs it will be impossible to help you.

Cheers,
Daniel

--
Psss, psss, put it down! - http://www.cafepress.com/putitdown
 
Reply With Quote
 
 
 
 
Gabriel Genellina
Guest
Posts: n/a
 
      04-28-2009
En Tue, 28 Apr 2009 14:15:54 -0300, Daniel Fetchinson
<(E-Mail Removed)> escribió:

>> I have a job that runs hourly. Part of the processing is to rename a
>> file using os.rename. The file name is the same every hour.
>> Sometimes the os.rename fails and the file does not get renamed. It
>> only happens occasionally. Most of the time it works just fine.

>
> Unless you send a minimal version of the code you use and the complete
> traceback when an error occurs it will be impossible to help you.


Also, don'f forget to tell us the OS / environment you're using.
On Windows, you cannot rename a file while it is open by any process.

--
Gabriel Genellina

 
Reply With Quote
 
t123
Guest
Posts: n/a
 
      04-28-2009
It's running on solaris 9. Here is some of the code. It's actually
at the beginning of the job. The files are ftp'd over. The first
thing that happens is that the files get renamed before any processing
of the file. And when it fails, it always fails at the first file,
comm.dat. What I can't understand is why the inconsistent behavior.

try:
if os.path.exists(paths.xferin_dir+'/COMM.DAT'):
os.rename(paths.xferin_dir+'/COMM.DAT',paths.xferin_dir+'/
COMM.DAT'+'.0')
if os.path.exists(paths.xferin_dir+'/DESC.DAT'):
os.rename(paths.xferin_dir+'/DESC.DAT',paths.xferin_dir+'/
DESC.DAT'+'.0')
if os.path.exists(paths.xferin_dir+'/HEADER.DAT'):
os.rename(paths.xferin_dir+'/HEADER.DAT',paths.xferin_dir+'/
HEADER.DAT'+'.0')
if os.path.exists(paths.xferin_dir+'/LINE.DAT'):
os.rename(paths.xferin_dir+'/LINE.DAT',paths.xferin_dir+'/
LINE.DAT'+'.0')

except:
print "Unexpected error:", sys.exc_info()[0]
raise

On Apr 28, 12:25*pm, Duncan Booth <(E-Mail Removed)>
wrote:
> t123 <(E-Mail Removed)> wrote:
> > I was wondering if anyone has seen this problem before?

>
> > I have a job that runs hourly. *Part of the processing is to rename a
> > file using os.rename. *The file name is the same every hour.
> > Sometimes the os.rename fails and the file does not get renamed. *It
> > only happens occasionally. *Most of the time it works just fine.

>
> > Appreciate any ideas.

>
> My crystal ball says you are using Windows and don't always close the file
> before you try to rename it.


 
Reply With Quote
 
MRAB
Guest
Posts: n/a
 
      04-28-2009
t123 wrote:
> I was wondering if anyone has seen this problem before?
>
> I have a job that runs hourly. Part of the processing is to rename a
> file using os.rename. The file name is the same every hour.
> Sometimes the os.rename fails and the file does not get renamed. It
> only happens occasionally. Most of the time it works just fine.
>
> Appreciate any ideas.
>

This can happen if an antivirus checker, or Windows indexing service
(assuming you're using Windows), or some such background task, is
accessing the file at the time. It's a good idea to wait a short time
and then retry, perhaps more than once, before giving up. Just write a
short function to do this.
 
Reply With Quote
 
Gabriel Genellina
Guest
Posts: n/a
 
      04-28-2009
En Tue, 28 Apr 2009 14:43:01 -0300, t123 <(E-Mail Removed)> escribió:

> It's running on solaris 9. Here is some of the code. It's actually
> at the beginning of the job. The files are ftp'd over. The first
> thing that happens is that the files get renamed before any processing
> of the file. And when it fails, it always fails at the first file,
> comm.dat. What I can't understand is why the inconsistent behavior.
>
> try:
> if os.path.exists(paths.xferin_dir+'/COMM.DAT'):
> os.rename(paths.xferin_dir+'/COMM.DAT',paths.xferin_dir+'/
> COMM.DAT'+'.0')
> if os.path.exists(paths.xferin_dir+'/DESC.DAT'):
> os.rename(paths.xferin_dir+'/DESC.DAT',paths.xferin_dir+'/
> DESC.DAT'+'.0')
> if os.path.exists(paths.xferin_dir+'/HEADER.DAT'):
> os.rename(paths.xferin_dir+'/HEADER.DAT',paths.xferin_dir+'/
> HEADER.DAT'+'.0')
> if os.path.exists(paths.xferin_dir+'/LINE.DAT'):
> os.rename(paths.xferin_dir+'/LINE.DAT',paths.xferin_dir+'/
> LINE.DAT'+'.0')
>
> except:
> print "Unexpected error:", sys.exc_info()[0]
> raise


And the error is...?

--
Gabriel Genellina

 
Reply With Quote
 
Steven D'Aprano
Guest
Posts: n/a
 
      04-29-2009
On Tue, 28 Apr 2009 14:30:02 -0500, Nick Craig-Wood wrote:

> t123 <(E-Mail Removed)> wrote:
>> It's running on solaris 9. Here is some of the code. It's actually
>> at the beginning of the job. The files are ftp'd over. The first
>> thing that happens is that the files get renamed before any processing
>> of the file. And when it fails, it always fails at the first file,
>> comm.dat. What I can't understand is why the inconsistent behavior.
>>
>> try:
>> if os.path.exists(paths.xferin_dir+'/COMM.DAT'):
>> os.rename(paths.xferin_dir+'/COMM.DAT',paths.xferin_dir+'/

>
> This code is inherently racy... What if two copies of your code started
> simultaneously? They might both run the os.path.exists but only one
> will succeed in the os.rename.
>
> You could write instead
>
> try:
> os.rename(paths.xferin_dir+'/COMM.DAT',paths.xferin_dir+'/

COMM.DAT'+'.0')
> except OSError:
> pass
>
> Which isn't racy.



The race condition is still there. The only difference is that in the
first case it fails noisily, with an exception, and in the second it
fails quietly and does nothing.



--
Steven
 
Reply With Quote
 
Steven D'Aprano
Guest
Posts: n/a
 
      05-01-2009
On Thu, 30 Apr 2009 03:30:04 -0500, Nick Craig-Wood wrote:

>> The race condition is still there. The only difference is that in the
>> first case it fails noisily, with an exception, and in the second it
>> fails quietly and does nothing.

>
> I'd argue that since os.rename implements the syscall rename() and that
> is defined to be atomic (give or take) then the above is atomic and
> can't possibly be racy.


Ah, you're right, at least for systems where rename is atomic.

What I was trying to say is that the risk of the file *not* being renamed
is still there, but in the first case the code failed noisily with an
exception and in the second it just failed quietly. But that's wrong, the
test is to see if the source file exists, not whether the destination
file exists. If the destination file exists, and you have write-access to
it, then it will be over-written no matter what.


--
Steven
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
*bug* *bug* *bug* David Raleigh Arnold Firefox 12 04-02-2007 03:13 AM
ASP.NET Login control bug or SQL 2005 bug? RedEye ASP .Net 2 12-13-2005 10:57 AM
Re: BUG? OR NOT A BUG? John ASP .Net 2 09-21-2005 10:31 AM
Bug Parade Bug 4953793 Michel Joly de Lotbiniere Java 4 12-02-2003 05:05 AM
how to report bug to g++ ? got a bug and fixed up source code DarkSpy C++ 4 06-27-2003 09:05 AM



Advertisments