Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Perl > Perl Misc > flock or IPC semaphore for atomic appends

Reply
Thread Tools

flock or IPC semaphore for atomic appends

 
 
jtd
Guest
Posts: n/a
 
      11-14-2003
Hi all,

I'm running linux 2.4.20. I have 50-100 processes appending to a
single file. Each process appends a block of binary data between 1
byte to 100KB in size, at the rate of at least 1 per second. Using
flock appears to ensure that the appends are atomic, but it does use
up a lot of CPU time compared to not locking at all.

My questions:
1) Is my assumption that flock will ensure that appends are atomic
correct?
2) Why is flock processor intensive? Does flock write something to
disk when locking/unlocking?
3) Would IPC semaphores be a better idea since it is an in-memory
structure?

open(F, ">> append.txt");
binmode(F);
while(1) {
flock(F, LOCK_EX);
print F (large block of binary data)
flock(F, LOCK_UN);
sleep(rand(1));
}
close(F);

Thanks for any suggestions,
jtd
 
Reply With Quote
 
 
 
 
A. Sinan Unur
Guest
Posts: n/a
 
      11-14-2003
http://www.velocityreviews.com/forums/(E-Mail Removed) (jtd) wrote in
news:(E-Mail Removed) m:

> 2) Why is flock processor intensive? Does flock write something to
> disk when locking/unlocking?


From perldoc -f flock:

Two potentially non-obvious but traditional "flock" semantics
are that it waits indefinitely until the lock is granted, and
that its locks merely advisory.

I haven't looked at the source, but I think it is this waiting indefinitely
part (polling?) that's causing CPU usage to go up.

Sinan.

--
A. Sinan Unur
(E-Mail Removed)
Remove dashes for address
Spam bait: (E-Mail Removed)
 
Reply With Quote
 
 
 
 
Anno Siegel
Guest
Posts: n/a
 
      11-14-2003
A. Sinan Unur <(E-Mail Removed)> wrote in comp.lang.perl.misc:
> (E-Mail Removed) (jtd) wrote in
> news:(E-Mail Removed) m:
>
> > 2) Why is flock processor intensive? Does flock write something to
> > disk when locking/unlocking?

>
> From perldoc -f flock:
>
> Two potentially non-obvious but traditional "flock" semantics
> are that it waits indefinitely until the lock is granted, and
> that its locks merely advisory.
>
> I haven't looked at the source, but I think it is this waiting indefinitely
> part (polling?) that's causing CPU usage to go up.


No. Typically, file locking is event driven and no polling goes on.
Even when a lock is released implicitly (on close), the system knows
about it and can activate the new lock owner.

Anno
 
Reply With Quote
 
A. Sinan Unur
Guest
Posts: n/a
 
      11-14-2003
http://www.velocityreviews.com/forums/(E-Mail Removed)-berlin.de (Anno Siegel) wrote in
news:bp2snb$otk$(E-Mail Removed)-Berlin.DE:

> A. Sinan Unur <(E-Mail Removed)> wrote in comp.lang.perl.misc:
>> (E-Mail Removed) (jtd) wrote in
>> news:(E-Mail Removed) m:
>>
>> > 2) Why is flock processor intensive? Does flock write something to
>> > disk when locking/unlocking?

....
>> I haven't looked at the source, but I think it is this waiting
>> indefinitely part (polling?) that's causing CPU usage to go up.

>
> No. Typically, file locking is event driven and no polling goes on.
> Even when a lock is released implicitly (on close), the system knows
> about it and can activate the new lock owner.


OK. Thanks for the correction.

Sinan.
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
[ANN] atomic 0.0.1 - An atomic reference for Ruby Charles Oliver Nutter Ruby 5 06-08-2010 01:04 PM
Help Needed : Race condition while removing semaphore(sysvsem semaphore) techi_C C Programming 2 08-10-2006 12:27 PM
atomic cross-window IPC? bwucke@gmail.com Javascript 5 12-24-2005 11:26 AM
File upload/download from database. Download appends aspx page to end of file Ryan Taylor ASP .Net 2 11-10-2004 08:31 PM
Response.redirect appends Port number to URL Sanjay Patil ASP .Net 0 05-10-2004 03:04 PM



Advertisments