«

I'm trying to safely rename a file without over-writing any existing
files, and I've run into a problem with file locks. Here's a naive way of
renaming without over-writing:

import os
oldname = "ham.txt"
newname = "spam.txt"
if not os.path.exists(newname):
os.rename(oldname, newname)

It's naive because there's a race-condition: if file newname is created
by another process after the call to exists(), but before the call to
rename(), then it will be over-written.

Here's my current solution, based on advice given by people on this
thread:
http://mail.python.org/pipermail/pyt...er/411432.html

and this recipe:
http://code.activestate.com/recipes/65203/

but it isn't working for me.


import os, fcntl
oldname = "ham.txt"
newname = "spam.txt"

def lock_destination(name):
fileno = os.open(name, os.O_CREAT | os.O_EXCL)
fcntl.flock(fileno, fcntl.LOCK_EX) # POSIX systems only
return fileno

# Create a test file to be renamed.
f = open(oldname, 'w')
f.write('this is my file\n')
f.close()
fileno = lock_destination(newname)

# At this point, I can see "ham.txt" plus an empty file
# "spam.txt" in my file browser

os.rename(oldname, newname)



The rename works, but here is my problem: after getting what I thought
was an exclusive lock on the new file, but before calling os.rename(), I
can still over-write it from another process:

$ echo "this comes from another process" > spam.txt
$ cat spam.txt
this comes from another process


This is despite running lock_destination("spam.txt"). What am I doing
wrong?

Before anyone asks, yes I have checked that the Python process and the
shell process are in the same working directory, and therefore are
writing to the same file:

>>> os.read(fileno, 200)
'this comes from another process\n'


I'm using Linux (Fedora).



--
Steven

Steven D'Aprano wrote:
<cut>
> import os, fcntl
> oldname = "ham.txt"
> newname = "spam.txt"
>
> def lock_destination(name):
> fileno = os.open(name, os.O_CREAT | os.O_EXCL)
> fcntl.flock(fileno, fcntl.LOCK_EX) # POSIX systems only
> return fileno
>
> # Create a test file to be renamed.
> f = open(oldname, 'w')
> f.write('this is my file\n')
> f.close()
> fileno = lock_destination(newname)
>
> # At this point, I can see "ham.txt" plus an empty file
> # "spam.txt" in my file browser
>
> os.rename(oldname, newname)
>
> The rename works, but here is my problem: after getting what I thought
> was an exclusive lock on the new file, but before calling os.rename(), I
> can still over-write it from another process:
>
> $ echo "this comes from another process" > spam.txt
> $ cat spam.txt
> this comes from another process
>
<cut>
This is weird, you would indeed expect it to be locked, I tried it on
FreeBSD and it behaves the same.
I guess the behavior of os.rename should be changed so it raises an
exception when the source destination exists independent on the below
platform.

I tried searching for another way to do it but the closest I came was to
popen mv and raise an exception if it asks to overwrite the existing file.

Sorry I can't be of more help.
--
mph

On Dec 6, 12:25*am, Steven D'Aprano <st...@REMOVE-THIS-
cybersource.com.au> wrote:

> The rename works, but here is my problem: after getting what I thought
> was an exclusive lock on the new file, but before calling os.rename(), I
> can still over-write it from another process:
>
> $ echo "this comes from another process" > spam.txt
> $ cat spam.txt
> this comes from another process

What happens if you try to delete spam.txt at this point instead of
over-write it?

Steven D'Aprano wrote:
> I'm trying to safely rename a file without over-writing any existing
> files, and I've run into a problem with file locks. Here's a naive way of
> renaming without over-writing
By default on a Linux filesystem, flock() gives you an _advisory_ lock.
Other processes can touch the file unless they explicitly check the lock.

Try

man mount (see under the -o mand)

and

man fcntl (see under file locking)

Jeffrey Straszheim