Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Java > Design Question: Managing multiple writes to flat file

Reply
Thread Tools

Design Question: Managing multiple writes to flat file

 
 
Rhino
Guest
Posts: n/a
 
      03-24-2005
I have a variety of applications that need to be able to write diagnostics
to the same flat file. While there won't be that many situations where any
of these applications are running concurrently, it could happen. What is the
best way to write my applications to handle the case where Program A wants
to write to the flat file while Program B is already writing to it?

Am I safe in assuming that there is no safe way to have multiple Java
programs simultaneously writing to the same flat file? If I am, then am I
correct in assuming that each program that wants to write to the flat file
needs to first verify that the file is not already in use? If so, what is
the correct way to determine if some other program is writing to a file? I
don't see anything in the File class that indicates if a file is in use,
unless perhaps canRead() does that; the API is very vague on that so I'm not
sure if canRead() is actually reporting on whether a file is already in use.

Now, assuming that I'm right about having to verify that no other program is
using the file, what should my program do if it finds that the file is in
use when it wants to write to it? Should the program that wants the file be
forced to exit? Or should it go on about its work and simply not try to
write to the file after all?

How do loggers handle this situation? What happens if two concurrent
applications want to write to the same log file at the same time?

--
Rhino
---
rhino1 AT sympatico DOT ca
"There are two ways of constructing a software design. One way is to make it
so simple that there are obviously no deficiencies. And the other way is to
make it so complicated that there are no obvious deficiencies." - C.A.R.
Hoare


 
Reply With Quote
 
 
 
 
Elie De Brauwer
Guest
Posts: n/a
 
      03-24-2005
Rhino wrote:
> I have a variety of applications that need to be able to write diagnostics
> to the same flat file. While there won't be that many situations where any
> of these applications are running concurrently, it could happen. What is the
> best way to write my applications to handle the case where Program A wants
> to write to the flat file while Program B is already writing to it?
>
> Am I safe in assuming that there is no safe way to have multiple Java
> programs simultaneously writing to the same flat file?


Hello,

Suppose in this case you have a multithreaded logging application, which
has a single synchronized log(String s); alike function, why shouldn't
this work ? The multithreading part enables it to handle multiple
applications that want to log but the synchronized logging method
ensures that only one file is logging at the same time.

So both program A and B use a logger program C. You could even do remote
logging by letting program C listen on the network or use other
mechanisms to connect to program C including divine intervention

hth

 
Reply With Quote
 
 
 
 
Matt Humphrey
Guest
Posts: n/a
 
      03-24-2005

"Rhino" <(E-Mail Removed)> wrote in message
news:9AD0e.9843$(E-Mail Removed).. .
> I have a variety of applications that need to be able to write diagnostics
> to the same flat file. While there won't be that many situations where any
> of these applications are running concurrently, it could happen. What is

the
> best way to write my applications to handle the case where Program A wants
> to write to the flat file while Program B is already writing to it?


Acquire a lock for the file, write to it, release the lock. See the bottom
of the message for the details on how to do this using a ServerSocket as the
lock.

>
> Am I safe in assuming that there is no safe way to have multiple Java
> programs simultaneously writing to the same flat file?


Whether or not simultaneous writing is possible (e.g. implicit locks,
atomicity, flushing, etc) depends on the OS rather than Java.

> If I am, then am I
> correct in assuming that each program that wants to write to the flat file
> needs to first verify that the file is not already in use? If so, what is
> the correct way to determine if some other program is writing to a file? I
> don't see anything in the File class that indicates if a file is in use,
> unless perhaps canRead() does that; the API is very vague on that so I'm

not
> sure if canRead() is actually reporting on whether a file is already in

use.

You have to arrange the lock yourself--see the details below

> Now, assuming that I'm right about having to verify that no other program

is
> using the file, what should my program do if it finds that the file is in
> use when it wants to write to it?


Clearly, it would have to wait. You want writing to the file to be quick so
that you acquire the lock, open the file, write to the file, close the file
and release the lock. As an added bonus, you want a lock that is
automatically discarded if your program fails.

> Should the program that wants the file be
> forced to exit? Or should it go on about its work and simply not try to
> write to the file after all?


It should wait and try again.

>
> How do loggers handle this situation? What happens if two concurrent
> applications want to write to the same log file at the same time?


Some heavy-duty logging systems use a logging server. Applications write to
the server (where concurrency is ok) and the server writes to the file. For
most ordinary systems simple locking is ok.

You can arrange for two or more independent applications (Java or not) to
write to the same flat file in a controlled way. First, pick a TCP port
number to be your lock. Any rarely-used number (e.g. 7112) will be fine
provided that all your applications know about. When any of the
applications needs to write to the file it first tries to allocate a server
socket for that port (e.g. new ServerSocket (LOCK_PORT...). If the socket
gets allocated, the program can open the file, write to it and close the
file. It should also then close the socket. If it cannot allocate the
socket it should wait a short time and try again.

TCP ports are global to the machine and issued uniquely and atomically--only
one program can have it at any time. Having the port means you have the
lock. If any program can't allocate the port it means it's already in use.
The program should wait a bit and try again. If the writes are reasonably
quick it will get in.

Note that you don't actually have to do anything with the socket. Just
allocate it and release it.

Cheers,
Matt Humphrey http://www.velocityreviews.com/forums/(E-Mail Removed) http://www.iviz.com/


 
Reply With Quote
 
Rhino
Guest
Posts: n/a
 
      03-24-2005

"Elie De Brauwer" <(E-Mail Removed)> wrote in message
news:LnE0e.45054$(E-Mail Removed)-ops.be...
> Rhino wrote:
> > I have a variety of applications that need to be able to write

diagnostics
> > to the same flat file. While there won't be that many situations where

any
> > of these applications are running concurrently, it could happen. What is

the
> > best way to write my applications to handle the case where Program A

wants
> > to write to the flat file while Program B is already writing to it?
> >
> > Am I safe in assuming that there is no safe way to have multiple Java
> > programs simultaneously writing to the same flat file?

>
> Hello,
>
> Suppose in this case you have a multithreaded logging application, which
> has a single synchronized log(String s); alike function, why shouldn't
> this work ? The multithreading part enables it to handle multiple
> applications that want to log but the synchronized logging method
> ensures that only one file is logging at the same time.
>
> So both program A and B use a logger program C. You could even do remote
> logging by letting program C listen on the network or use other
> mechanisms to connect to program C including divine intervention
>

I'm going to have to think about this a bit.

Thanks for your reply!

Rhino


 
Reply With Quote
 
Rhino
Guest
Posts: n/a
 
      03-24-2005

"Matt Humphrey" <(E-Mail Removed)> wrote in message
news:(E-Mail Removed)...
>
> "Rhino" <(E-Mail Removed)> wrote in message
> news:9AD0e.9843$(E-Mail Removed).. .
> > I have a variety of applications that need to be able to write

diagnostics
> > to the same flat file. While there won't be that many situations where

any
> > of these applications are running concurrently, it could happen. What is

> the
> > best way to write my applications to handle the case where Program A

wants
> > to write to the flat file while Program B is already writing to it?

>
> Acquire a lock for the file, write to it, release the lock. See the

bottom
> of the message for the details on how to do this using a ServerSocket as

the
> lock.
>
> >
> > Am I safe in assuming that there is no safe way to have multiple Java
> > programs simultaneously writing to the same flat file?

>
> Whether or not simultaneous writing is possible (e.g. implicit locks,
> atomicity, flushing, etc) depends on the OS rather than Java.
>
> > If I am, then am I
> > correct in assuming that each program that wants to write to the flat

file
> > needs to first verify that the file is not already in use? If so, what

is
> > the correct way to determine if some other program is writing to a file?

I
> > don't see anything in the File class that indicates if a file is in use,
> > unless perhaps canRead() does that; the API is very vague on that so I'm

> not
> > sure if canRead() is actually reporting on whether a file is already in

> use.
>
> You have to arrange the lock yourself--see the details below
>
> > Now, assuming that I'm right about having to verify that no other

program
> is
> > using the file, what should my program do if it finds that the file is

in
> > use when it wants to write to it?

>
> Clearly, it would have to wait. You want writing to the file to be quick

so
> that you acquire the lock, open the file, write to the file, close the

file
> and release the lock. As an added bonus, you want a lock that is
> automatically discarded if your program fails.
>
> > Should the program that wants the file be
> > forced to exit? Or should it go on about its work and simply not try to
> > write to the file after all?

>
> It should wait and try again.
>
> >
> > How do loggers handle this situation? What happens if two concurrent
> > applications want to write to the same log file at the same time?

>
> Some heavy-duty logging systems use a logging server. Applications write

to
> the server (where concurrency is ok) and the server writes to the file.

For
> most ordinary systems simple locking is ok.
>
> You can arrange for two or more independent applications (Java or not) to
> write to the same flat file in a controlled way. First, pick a TCP port
> number to be your lock. Any rarely-used number (e.g. 7112) will be fine
> provided that all your applications know about. When any of the
> applications needs to write to the file it first tries to allocate a

server
> socket for that port (e.g. new ServerSocket (LOCK_PORT...). If the socket
> gets allocated, the program can open the file, write to it and close the
> file. It should also then close the socket. If it cannot allocate the
> socket it should wait a short time and try again.
>
> TCP ports are global to the machine and issued uniquely and

atomically--only
> one program can have it at any time. Having the port means you have the
> lock. If any program can't allocate the port it means it's already in

use.
> The program should wait a bit and try again. If the writes are reasonably
> quick it will get in.
>
> Note that you don't actually have to do anything with the socket. Just
> allocate it and release it.
>

Thanks very much for your suggestions! They are clear and sound like they
should work well for my purposes.

Rhino


 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Lock xml file writes through multiple aspx pages vani ASP .Net 2 07-31-2010 02:04 PM
Multiple reads and writes with popen? davidcalvin C Programming 0 07-08-2008 11:26 PM
net::sftp write error with multiple packet writes csgonan@yahoo.com Perl Misc 3 10-22-2007 10:18 PM
Multiple writes to client socket Daniel Draper C Programming 3 04-04-2005 03:39 AM
java with ISAM , flat-file database with multiple keys Mohammad S Khan Java 3 08-31-2004 12:59 PM



Advertisments