Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Perl > Perl Misc > Correct file locking techniques

Reply
Thread Tools

Correct file locking techniques

 
 
Robert TV
Guest
Posts: n/a
 
      06-26-2004
Hi, I am asking foir advice on which form of code i've written below is the
correct and safe way to lock a file. I've done some reading on" use Fcntl
qw(EFAULT :flock);" and I understand the basics. I've also read back on
many posts regarding file locking and there seems to be much debate. For
this code, I simply want to increase a value.

#!/usr/bin/perl

use Fcntl qw(EFAULT :flock);
use CGI::Carp qw(fatalsToBrowser);

open (NUMBER, "<number.txt") or die "Can't open file: $!";
flock (NUMBER, LOCK_EX) or die "Can't lock file: $!";
$value = <NUMBER>;
close(NUMBER);

$value = $value++;

open (NUMBER, ">number.txt") or die "Can't open file: $!";
flock (NUMBER, LOCK_EX) or die "Can't lock file: $!";
print NUMBER $value;
close(NUMBER);

The above code seems to work ok. My question is when the first user locks
and reads the file, does the locking prevent other users from reading the
file? The $value is then increased and printed back to the file.

Another concern of mine is ... User#1 locks file, reads file value, then
close and releases lock. While User#1 is writing the new value back to the
file, another user came and read the unlocked file BEFORE the value was
increased. So he gets the same value as User#1. User#2 then increases the
value and writes the same data as User#1. In the end, the locking technique
failed ... am I right?

Another method I came up with is locking and entire read/write session
within a what I will call "parent lock" Here is the code:

#!/usr/bin/perl

use Fcntl qw(EFAULT :flock);
use CGI::Carp qw(fatalsToBrowser);

open (PARENT, "<locker.txt") or die "Can't open file: $!"; #an empty file
flock (PARENT, LOCK_EX) or die "Can't lock file: $!";

open (NUMBER, "<number.txt") or die "Can't open file: $!";
flock (NUMBER, LOCK_SH) or die "Can't lock file: $!";
$value = <NUMBER>;
close(NUMBER);

$value = $value++;

open (NUMBER, ">number.txt") or die "Can't open file: $!";
flock (NUMBER, LOCK_SH) or die "Can't lock file: $!";
print NUMBER $value;
close(NUMBER);

close(PARENT); #closes the parent lock

Now User #1 locks the parent file and moves on to the read/write session.
All other users get stuck back at the parent lock until released. I had to
use LOCK_SH for the read/write locks for some reason ... if all locks were
set to EX the program would stall. Is this technique better? Is it
effective? Or perhaps they are both ineffective?

In this next example, I don't want to increase a value, just append to the
file on a new line. So I do this:

#!/usr/bin/perl

use Fcntl qw(EFAULT :flock);
use CGI::Carp qw(fatalsToBrowser);

open (DATA, ">>locker.txt") or die "Can't open file: $!";
flock (DATA,LOCK_EX) or die "Can't lock file: $!";
print DATA "$data\n";
close(DATA);

Im going to assume that this is ok.

It's just when increasing a value that concerns me ... multipul users
reading same value when the read lock is released. Your suggestions are
greatly welcomes. TIA

Robert


 
Reply With Quote
 
 
 
 
Ben Morrow
Guest
Posts: n/a
 
      06-27-2004

Quoth "Robert TV" <(E-Mail Removed)>:
> Hi, I am asking foir advice on which form of code i've written below is the
> correct and safe way to lock a file. I've done some reading on" use Fcntl
> qw(EFAULT :flock);" and I understand the basics. I've also read back on
> many posts regarding file locking and there seems to be much debate. For
> this code, I simply want to increase a value.
>
> #!/usr/bin/perl
>
> use Fcntl qw(EFAULT :flock);
> use CGI::Carp qw(fatalsToBrowser);
>
> open (NUMBER, "<number.txt") or die "Can't open file: $!";
> flock (NUMBER, LOCK_EX) or die "Can't lock file: $!";
> $value = <NUMBER>;
> close(NUMBER);
>
> $value = $value++;
>
> open (NUMBER, ">number.txt") or die "Can't open file: $!";
> flock (NUMBER, LOCK_EX) or die "Can't lock file: $!";
> print NUMBER $value;
> close(NUMBER);
>
> The above code seems to work ok. My question is when the first user locks
> and reads the file, does the locking prevent other users from reading the
> file?


No. It simply prevents other processes from *obtaining a lock* on the
file.

> The $value is then increased and printed back to the file.
>
> Another concern of mine is ... User#1 locks file, reads file value, then
> close and releases lock. While User#1 is writing the new value back to the
> file, another user came and read the unlocked file BEFORE the value was
> increased. So he gets the same value as User#1. User#2 then increases the
> value and writes the same data as User#1. In the end, the locking technique
> failed ... am I right?


You are. You must read/modify/write under the same, exclusive, lock;
anyone who is just reading can use a shared lock.

> Another method I came up with is locking and entire read/write session
> within a what I will call "parent lock" Here is the code:


This would be correct, but there is no need for two layers of locks. The
inner lock is made completely redundant by the outer.

Correct technique is:

Processes that will only read:

open read-only
flock LOCK_SH
read the data
close

Processes that will read and modify:

open read-write ('+<')
flock LOCK_EX
read the data
make the modifications
seek to the start of the file (perldoc -f seek)
truncate the file (perldoc -f truncate)
write the new data
close

Alternatively, you can use a lockfile instead of proper flock locking;
this can be simpler, but you have to be careful to clean up after
yourself:

sysopen a lockfile with mode O_CREAT | O_EXCL
if the open fails, sleep for a bit and try again
open/close/read/write the data file as you will
unlink the lockfile

One problem with this is you may wish to check the age of the lockfile
and summerarily (sp?) unlink it if it appears to be too old...

> I had to use LOCK_SH for the read/write locks for some reason ... if
> all locks were set to EX the program would stall.


On some systems you cannot LOCK_EX a file unless you opened it for
writing.

> In this next example, I don't want to increase a value, just append to the
> file on a new line. So I do this:
>
> #!/usr/bin/perl
>
> use Fcntl qw(EFAULT :flock);
> use CGI::Carp qw(fatalsToBrowser);
>
> open (DATA, ">>locker.txt") or die "Can't open file: $!";
> flock (DATA,LOCK_EX) or die "Can't lock file: $!";
> print DATA "$data\n";
> close(DATA);
>
> Im going to assume that this is ok.


Note that your readers will need to lock as well.

Ben

--
All persons, living or dead, are entirely coincidental.
http://www.velocityreviews.com/forums/(E-Mail Removed) Kurt Vonnegut
 
Reply With Quote
 
 
 
 
Bob Walton
Guest
Posts: n/a
 
      06-27-2004
Robert TV wrote:

> Hi, I am asking foir advice on which form of code i've written below is the
> correct and safe way to lock a file. I've done some reading on" use Fcntl



I have found that the most portable and least hassle way to lock files
is to use a lock file. This is a file which is used exclusively for the
purpose of establishing a lock -- it can be empty, or have any contents
you want, but the data in the lock file would be unused and unimportant.
This technique is particularly useful when doing stuff which is a bit
out of the ordinary, such as locking DBM-type files to which you wish to
tie a hash, for example.

If you want to write a file, open the lock file for write (that will, at
least on some systems, wipe out the contents of the lock file), then
establish an exclusive lock on it. Then go do whatever you wanted the
exclusive lock for (write another file, tie a hash to a DBM-type file,
or any other operation requiring exclusive access to a resource). Then,
when you are all done with your lock, close the lock file.

If you want to read a file, open the lock file for reading, and
establish a shared lock on it. Go read your data file or do whatever
you wanted the shared lock for. When done, close the lock file.

To the best of my knowledge, that works flawlessly on at least local
files on every OS that supports locking (that, by the way, rules out
Windoze 95/98/98SE). However, you should carefully and thoroughly read
everything your OS has to say about file locking, particularly if you
are considering locking files accessed via a network.

You will want to note that locking a file does not prevent other
processes from read/writing/deleting/whatevering a file. *All* it does
is establish a lock -- the other processes have to cooperate with the
lock in order for it to work. An exclusive lock means other processes
seeking a lock will block until the exclusive lock is released; a shared
lock means other processes seeking an exclusive lock will block until
all the shared locks are released.

I find it convenient to write a couple of subs to do the locking, like
[untested]:

sub lockex{
open LOCK,">lockfile.txt"
or die "Oops, couldn't open lockfile.txt for write, $!";
flock(LOCK,LOCK_EX)
or die "flock bombed out in lockex, $!";
}
sub locksh{
open LOCK,"lockfile.txt"
or die "Oops, couldn't open lockfile.txt for read, $!";
flock(LOCK,LOCK_SH)
or die "flock bombed out in locksh, $!";
}
sub unlock{
close LOCK or die "Oops, couldn't close lockfile.txt, $!";
}

HTH.


....
> Robert


--
Bob Walton
Email: http://bwalton.com/cgi-bin/emailbob.pl

 
Reply With Quote
 
Ben Morrow
Guest
Posts: n/a
 
      06-27-2004

Quoth (E-Mail Removed)d:
> Robert TV wrote:
>
> > Hi, I am asking foir advice on which form of code i've written below is the
> > correct and safe way to lock a file. I've done some reading on" use Fcntl

>
> I have found that the most portable and least hassle way to lock files
> is to use a lock file. This is a file which is used exclusively for the
> purpose of establishing a lock -- it can be empty, or have any contents
> you want, but the data in the lock file would be unused and unimportant.
> This technique is particularly useful when doing stuff which is a bit
> out of the ordinary, such as locking DBM-type files to which you wish to
> tie a hash, for example.


You can still perfectly well take a flock lock on the DBM file.

> If you want to write a file, open the lock file for write (that will, at
> least on some systems, wipe out the contents of the lock file), then
> establish an exclusive lock on it.


The usual reason for using a lockfile is to avoid making locking calls
altogether... a call to sysopen with O_CREAT | O_EXCL will atomically
test-and-create the named file, so there is no need to then lock it.

> To the best of my knowledge, that works flawlessly on at least local
> files on every OS that supports locking (that, by the way, rules out
> Windoze 95/98/98SE).


However, I think that win98 *does* support O_EXCL properly, so that
method will work.

> However, you should carefully and thoroughly read
> everything your OS has to say about file locking, particularly if you
> are considering locking files accessed via a network.


Yes. I belive most network filesystems do it properly; the exception is
NFSv2, and there are (at least on linux) instructions in open(2) for how
to safely lock files over NFS.

> I find it convenient to write a couple of subs to do the locking, like
> [untested]:
>
> sub lockex{
> open LOCK,">lockfile.txt"


Note that if you choose to put any useful information in the lockfile
(it is common to put the pid and possibly the hostname of the process
which created the lock, so later processes can check if you still exist)
you would need to sysopen it with O_RDWR | O_CREAT to ensure you didn't
destroy info about someone else's lock.

> or die "Oops, couldn't open lockfile.txt for write, $!";
> flock(LOCK,LOCK_EX)
> or die "flock bombed out in lockex, $!";
> }
> sub locksh{
> open LOCK,"lockfile.txt"
> or die "Oops, couldn't open lockfile.txt for read, $!";
> flock(LOCK,LOCK_SH)
> or die "flock bombed out in locksh, $!";
> }
> sub unlock{
> close LOCK or die "Oops, couldn't close lockfile.txt, $!";
> }


Yuck! What happens if you call this twice, for different files? You'll
implicitly close the global FH and destroy your lock. Try

{
my %locks;

sub lock {
my ($file, $type) = shift;

open $locks{$file}, '>', "$file.lock"
or die "can't create $file.lock: $!";

flock $locks{$file}, $type
or die "can't get lock on $file.lock: $!";
}

sub unlock {
close $locks{$_[0]};
}
}

Ben

--
Musica Dei donum optimi, trahit homines, trahit deos. |
Musica truces molit animos, tristesque mentes erigit. | (E-Mail Removed)
Musica vel ipsas arbores et horridas movet feras. |
 
Reply With Quote
 
Darren Dunham
Guest
Posts: n/a
 
      07-02-2004
Ben Morrow <(E-Mail Removed)> wrote:
>> If you want to write a file, open the lock file for write (that will, at
>> least on some systems, wipe out the contents of the lock file), then
>> establish an exclusive lock on it.


> The usual reason for using a lockfile is to avoid making locking calls
> altogether... a call to sysopen with O_CREAT | O_EXCL will atomically
> test-and-create the named file, so there is no need to then lock it.


Assuming NFS isn't involved.

I often use a separate lockfile because I don't want to modify my data
file directly, lest a crash leave it in an inconsistent state.

Instead I lock a separate file so that I can move a new file or files
into place and not have to worry about my lock moving with it.

> Yes. I belive most network filesystems do it properly; the exception is
> NFSv2, and there are (at least on linux) instructions in open(2) for how
> to safely lock files over NFS.


With increased cleanup requirements also, unfortunately. I tend not to
do it because things get so messy.

>> sub lockex{
>> open LOCK,">lockfile.txt"


> Note that if you choose to put any useful information in the lockfile
> (it is common to put the pid and possibly the hostname of the process
> which created the lock, so later processes can check if you still exist)
> you would need to sysopen it with O_RDWR | O_CREAT to ensure you didn't
> destroy info about someone else's lock.


The same as.. open (LOCK, "<+lockfile.txt").

--
Darren Dunham (E-Mail Removed)
Senior Technical Consultant TAOS http://www.taos.com/
Got some Dr Pepper? San Francisco, CA bay area
< This line left intentionally blank to confuse you. >
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Locking locking resolution Frontpage raiderhawk General Computer Support 0 01-08-2008 01:42 AM
increase in cpu usage on locking and locking the system sowmya.rangineni@gmail.com Computer Support 0 06-15-2007 12:06 PM
Application locking to support optimisitc locking ? Timasmith Java 4 11-01-2006 12:42 AM
Uploaded File Empty but in correct folder with correct name froil Perl Misc 12 03-02-2006 01:21 PM
Confused about locking a file via file.flock(File::LOCK_EX) Ludwigi Beethoven Ruby 5 07-26-2003 03:26 PM



Advertisments