Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Perl > Perl Misc > File locking using threads

Reply
Thread Tools

File locking using threads

 
 
Prince Al
Guest
Posts: n/a
 
      10-23-2009
Hi,

I am attempting to write a some code that spawns various threads. Each
thread will need to access a common file and to ensure that this file
is only written to by one thread at a time, it needs to be locked.
However, I am having trouble even creating a test script to try out
the logic, which is below. What happens is that both threads are able
to acquire the lock, even though the first thread should have it and
release it before the second thread is able to lock the file. Any
assistance will be very gratefully received!

Cheers

Tim

#!usr/bin/perl

use strict;
use threads;

my $thr_1 = threads->new(\&flock_test,5, 20, "Thread 1");
my $thr_2 = threads->new(\&flock_test,10, 5, "Thread 2");

grep {$_->join;} ($thr_1,$thr_2);
# Just In Case!
while ( my(@list)=threads->list()) {
print "$#list\n";
grep { $_->join } @list;
};
exit;

sub flock_test {
open(my $fh, ">", "/tmp/flock_test.dat");
my $sleep_1 = $_[0];
my $sleep_2 = $_[1];
my $thread_name = $_[2];

sleep $sleep_1;

flock($fh,2) || die "Could not acquire the lock\n";
print "$thread_name: acquired lock!\n";
sleep $sleep_2;
close $fh;
print "$thread_name: released lock!\n";
}
 
Reply With Quote
 
 
 
 
Jens Thoms Toerring
Guest
Posts: n/a
 
      10-23-2009
Prince Al <(E-Mail Removed)> wrote:
> I am attempting to write a some code that spawns various threads. Each
> thread will need to access a common file and to ensure that this file
> is only written to by one thread at a time, it needs to be locked.
> However, I am having trouble even creating a test script to try out
> the logic, which is below. What happens is that both threads are able
> to acquire the lock, even though the first thread should have it and
> release it before the second thread is able to lock the file. Any
> assistance will be very gratefully received!


> #!usr/bin/perl


> use strict;
> use threads;


> my $thr_1 = threads->new(\&flock_test,5, 20, "Thread 1");
> my $thr_2 = threads->new(\&flock_test,10, 5, "Thread 2");


> grep {$_->join;} ($thr_1,$thr_2);
> # Just In Case!
> while ( my(@list)=threads->list()) {
> print "$#list\n";
> grep { $_->join } @list;
> };
> exit;


> sub flock_test {
> open(my $fh, ">", "/tmp/flock_test.dat");
> my $sleep_1 = $_[0];
> my $sleep_2 = $_[1];
> my $thread_name = $_[2];


> sleep $sleep_1;


> flock($fh,2) || die "Could not acquire the lock\n";


Works ok on my machine. My guess is that on yours LOCK_EX (that's
what you need here) isn't 2. Better use

use Fcntl ':flock';

to import the constants used for flock() and then write

flock($fh,LOCK_EX) || die "Could not acquire the lock\n";

Regards, Jens
--
\ Jens Thoms Toerring ___ http://www.velocityreviews.com/forums/(E-Mail Removed)
\__________________________ http://toerring.de
 
Reply With Quote
 
 
 
 
QoS@invalid.net
Guest
Posts: n/a
 
      10-24-2009

Prince Al <(E-Mail Removed)> wrote in message-id: <(E-Mail Removed)>

>
> Hi,
>
> I am attempting to write a some code that spawns various threads. Each
> thread will need to access a common file and to ensure that this file
> is only written to by one thread at a time, it needs to be locked.
> However, I am having trouble even creating a test script to try out
> the logic, which is below. What happens is that both threads are able
> to acquire the lock, even though the first thread should have it and
> release it before the second thread is able to lock the file. Any
> assistance will be very gratefully received!
>
> Cheers
>
> Tim
>
> #!usr/bin/perl
>
> use strict;
> use threads;
>
> my $thr_1 = threads->new(\&flock_test,5, 20, "Thread 1");
> my $thr_2 = threads->new(\&flock_test,10, 5, "Thread 2");
>
> grep {$_->join;} ($thr_1,$thr_2);
> # Just In Case!
> while ( my(@list)=threads->list()) {
> print "$#list\n";
> grep { $_->join } @list;
> };
> exit;
>
> sub flock_test {
> open(my $fh, ">", "/tmp/flock_test.dat");
> my $sleep_1 = $_[0];
> my $sleep_2 = $_[1];
> my $thread_name = $_[2];
>
> sleep $sleep_1;
>
> flock($fh,2) || die "Could not acquire the lock\n";
> print "$thread_name: acquired lock!\n";
> sleep $sleep_2;
> close $fh;
> print "$thread_name: released lock!\n";
> }


You might like threads::shared for inter thread communication.

You set up your shared variables then launch your threads.
Have the threads monitor and update the shared variables
to determine the status of the file or other things.

All threads will have access to the entire %shash.

foreach my $tID (1..2) {
warn 'Launching thread: [' . $tID . "]\n";
foreach my $l qw (fileIsLocked, go, stop, quit,) {
share($shash{1}{$l});
$shash{$tID}{$l} = 0;
}
$threads{$tID} = threads->new(\&worker, 1);
warn 'Thread: [' . $tID . "] is active\n";
}

sub worker #------------------------------------------------------------
{
#called from main
my $TID = $_[0] || 0;

while(1) {
if ($shash{$TID}{quit} == 1) {
last;
}
elsif ($shash{$TID}{fileIsLocked} == 1) {

}
#etc...
sleep (1);
}
return (1);
}

Hth,
J

 
Reply With Quote
 
QoS@invalid.net
Guest
Posts: n/a
 
      10-24-2009

(E-Mail Removed) wrote in message-id: <w_EEm.8817$(E-Mail Removed)>

>
>
> Prince Al <(E-Mail Removed)> wrote in message-id: <(E-Mail Removed)>
>
> >
> > Hi,
> >
> > I am attempting to write a some code that spawns various threads. Each
> > thread will need to access a common file and to ensure that this file
> > is only written to by one thread at a time, it needs to be locked.
> > However, I am having trouble even creating a test script to try out
> > the logic, which is below. What happens is that both threads are able
> > to acquire the lock, even though the first thread should have it and
> > release it before the second thread is able to lock the file. Any
> > assistance will be very gratefully received!
> >
> > Cheers
> >
> > Tim
> >
> > #!usr/bin/perl
> >
> > use strict;
> > use threads;
> >
> > my $thr_1 = threads->new(\&flock_test,5, 20, "Thread 1");
> > my $thr_2 = threads->new(\&flock_test,10, 5, "Thread 2");
> >
> > grep {$_->join;} ($thr_1,$thr_2);
> > # Just In Case!
> > while ( my(@list)=threads->list()) {
> > print "$#list\n";
> > grep { $_->join } @list;
> > };
> > exit;
> >
> > sub flock_test {
> > open(my $fh, ">", "/tmp/flock_test.dat");
> > my $sleep_1 = $_[0];
> > my $sleep_2 = $_[1];
> > my $thread_name = $_[2];
> >
> > sleep $sleep_1;
> >
> > flock($fh,2) || die "Could not acquire the lock\n";
> > print "$thread_name: acquired lock!\n";
> > sleep $sleep_2;
> > close $fh;
> > print "$thread_name: released lock!\n";
> > }

>
> You might like threads::shared for inter thread communication.
>
> You set up your shared variables then launch your threads.
> Have the threads monitor and update the shared variables
> to determine the status of the file or other things.
>
> All threads will have access to the entire %shash.
>
> foreach my $tID (1..2) {
> warn 'Launching thread: [' . $tID . "]\n";
> foreach my $l qw (fileIsLocked, go, stop, quit,) {
> share($shash{1}{$l});
> $shash{$tID}{$l} = 0;
> }
> $threads{$tID} = threads->new(\&worker, 1);
> warn 'Thread: [' . $tID . "] is active\n";
> }
>
> sub worker #------------------------------------------------------------
> {
> #called from main
> my $TID = $_[0] || 0;
>
> while(1) {
> if ($shash{$TID}{quit} == 1) {
> last;
> }
> elsif ($shash{$TID}{fileIsLocked} == 1) {
>
> }
> #etc...
> sleep (1);
> }
> return (1);
> }
>
> Hth,
> J


Hrmph, typo, just half way through my am coffee and still a bit tired.
For this to work right you would have to fix above thread launching code
replacing the 1's within the foreach my $tID (1..2) {} block with $tID


 
Reply With Quote
 
dba.9999@gmail.com
Guest
Posts: n/a
 
      10-25-2009
In article <(E-Mail Removed)>,
Prince Al says...

>sub flock_test {
> open(my $fh, ">", "/tmp/flock_test.dat");
> my $sleep_1 = $_[0];
> my $sleep_2 = $_[1];
> my $thread_name = $_[2];
>
> sleep $sleep_1;



why do you even have to lock the file via flock. Threads can lock
variables and that is faster and safer. Just create a shared variable
which you can use as a semaphore between threads. Unless that variable
is locked first, no thread can write to the file. Release the lock
once written.

I have done this in my of my scripts and it works great.

 
Reply With Quote
 
Prince Al
Guest
Posts: n/a
 
      10-25-2009
Hi,

Many thanks for the replies so far - very much appreciated! I tried
your various suggestions on my Linux machine and they worked like a
charm, however, moving this code onto the work servers did not prove
quite so fruitful... This are running Solaris (HP-UX B.11.11 U) and
given the way most things are at my company, configured incorrectly!

To answer a question posed above, the reason I want to lock a file
instead of using shared variables is because there will potentially be
many instances of the final script running and I want a way to manage
machine resources. I am writing a data load scheduler by the way...
So, when an instance of the script wants to grab a resource, it locks
the file and subtracts 1 (for example) from the number contained in
the file and when it finishes, it locks the file and adds 1 to the
contents of the file. In a nutshell.

Hopefully that is clear enough! Again, thanks in advance for any
assitance

Cheers

Tim
 
Reply With Quote
 
Jens Thoms Toerring
Guest
Posts: n/a
 
      10-25-2009
Prince Al <(E-Mail Removed)> wrote:
> Many thanks for the replies so far - very much appreciated! I tried
> your various suggestions on my Linux machine and they worked like a
> charm, however, moving this code onto the work servers did not prove
> quite so fruitful... This are running Solaris (HP-UX B.11.11 U) and
> given the way most things are at my company, configured incorrectly!


Out of curiosity, How was it mis-configured not to let you get a
lock on a file? The only thing I can think of at the moment is
that the file to be locked is on an NFS mounted disk and the
normal (non-Perl but POSIX) flock() function often does not
work with NFS - and usually Perl's flock() function is based on
POSIX's flock() function. If that should be the case you might
be better off using the POSIX fcntl() function, with one way to
do that would be to use a module I uploaded to CPAN, called
File::FcntlLock (current version is 0.12).

Regards, Jens
--
\ Jens Thoms Toerring ___ (E-Mail Removed)
\__________________________ http://toerring.de
 
Reply With Quote
 
Prince Al
Guest
Posts: n/a
 
      10-25-2009
On Oct 25, 6:55*pm, (E-Mail Removed) (Jens Thoms Toerring) wrote:
> Out of curiosity, How was it mis-configured not to let you get a
> lock on afile? The only thing I can think of at the moment is
> that thefileto be locked is on an NFS mounted disk ...


Hi Jens,

Sorry, after re-reading my post, I realised that bit might have been
unclear! I have no idea if there is a mis-configuration - I was just
musing out loud and was wondering if there could be such a scenario to
prevent this from working. Instead of placing the file to be locked
in /tmp, I have placed it in my home directory to test the scenario
about different mounts out, with the same results...

My current script is pasted below, oh, and the version of Perl we have
on the server is 5.8.3, if that is any use.

Thanks again for any help!

Cheers

Tim

#!usr/bin/perl

use strict;
use threads;
use Fcntl qw/:flock/;

my @args_1 = (5, 20, "Thread 1");
my @args_2 = (10, 5, "Thread 2");


my $thr_1 = threads->new(\&flock_test,5, 20, "Thread 1");
my $thr_2 = threads->new(\&flock_test,10, 5, "Thread 2");

$_->join for $thr_1,$thr_2;

sub flock_test {
open(my $fh, ">", "/home/thill/flock_test.dat");
my $sleep_1 = $_[0];
my $sleep_2 = $_[1];
my $thread_name = $_[2];

sleep $sleep_1;

#flock($fh,LOCK_EX) || die "Could not acquire the lock\n";

print "$thread_name: acquired lock!\n";
sleep $sleep_2;
close $fh;
print "$thread_name: released lock!\n";
}

 
Reply With Quote
 
Jens Thoms Toerring
Guest
Posts: n/a
 
      10-25-2009
Prince Al <(E-Mail Removed)> wrote:
> On Oct 25, 6:55*pm, (E-Mail Removed) (Jens Thoms Toerring) wrote:
> > Out of curiosity, How was it mis-configured not to let you get a
> > lock on afile? The only thing I can think of at the moment is
> > that thefileto be locked is on an NFS mounted disk ...


> Sorry, after re-reading my post, I realised that bit might have been
> unclear! I have no idea if there is a mis-configuration - I was just
> musing out loud and was wondering if there could be such a scenario to
> prevent this from working. Instead of placing the file to be locked
> in /tmp, I have placed it in my home directory to test the scenario
> about different mounts out, with the same results...


> My current script is pasted below, oh, and the version of Perl we have
> on the server is 5.8.3, if that is any use.


> #!usr/bin/perl


That's rather likely meant to be

#!/usr/bin/perl

> use strict;
> use threads;
> use Fcntl qw/:flock/;


> my @args_1 = (5, 20, "Thread 1");
> my @args_2 = (10, 5, "Thread 2");


> my $thr_1 = threads->new(\&flock_test,5, 20, "Thread 1");
> my $thr_2 = threads->new(\&flock_test,10, 5, "Thread 2");


> $_->join for $thr_1,$thr_2;


> sub flock_test {
> open(my $fh, ">", "/home/thill/flock_test.dat");
> my $sleep_1 = $_[0];
> my $sleep_2 = $_[1];
> my $thread_name = $_[2];


> sleep $sleep_1;


> #flock($fh,LOCK_EX) || die "Could not acquire the lock\n";


> print "$thread_name: acquired lock!\n";
> sleep $sleep_2;
> close $fh;
> print "$thread_name: released lock!\n";
> }


Mmm, since you commented out the line where flock() is called
I would expect that you won't get a lock but also no error
message that locking failed

Another thing I notice, which might result in trouble, is that
both threads open the file in write mode ('>'). That leads to
both threads emptying the file (i.e. possibly modifying it)
before asking for the lock which, as far as I have understood
your intentions, isn't what you're planning to do. I guess it
would make more sense to use '+<' (or '>>') instead of '>' when
opening the file to avoid that.

And, BTW, using 'use warnings;' is usually rather useful

Regards, Jens
--
\ Jens Thoms Toerring ___ (E-Mail Removed)
\__________________________ http://toerring.de
 
Reply With Quote
 
Prince Al
Guest
Posts: n/a
 
      10-25-2009
On Oct 25, 8:46*pm, (E-Mail Removed) (Jens Thoms Toerring) wrote:
> #!/usr/bin/perl


Thanks - corrected.

> Mmm, since you commented out the line where flock() is called
> I would expect that you won't get a lock but also no error
> message that locking failed


Dang - not making a good first impression here, am I?! I did actually
notice this just before I replied and tested the code without the
crucial line commented out, but same result. I obviously forgot to re-
copy the updated code...

> Another thing I notice, which might result in trouble, is that
> both threads open the file in write mode ('>'). That leads to
> both threads emptying the file (i.e. possibly modifying it)
> before asking for the lock which, as far as I have understood
> your intentions, isn't what you're planning to do. I guess it
> would make more sense to use '+<' (or '>>') instead of '>' when
> opening the file to avoid that.


Fair point, thank you.

> And, BTW, using 'use warnings;' is usually rather useful


Done
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Locking locking resolution Frontpage raiderhawk General Computer Support 0 01-08-2008 01:42 AM
JDBC Threads Locking up sameergn@gmail.com Java 1 06-18-2007 06:57 PM
increase in cpu usage on locking and locking the system sowmya.rangineni@gmail.com Computer Support 0 06-15-2007 12:06 PM
Application locking to support optimisitc locking ? Timasmith Java 4 11-01-2006 12:42 AM
threads: not locking read-only objects Christopher T King Python 3 08-07-2004 09:53 AM



Advertisments