Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > Re: 2 daemons write to a single file /w python file IO

Thread Tools

Re: 2 daemons write to a single file /w python file IO

Steven W. Orr
Posts: n/a
On Tuesday, Sep 11th 2007 at 21:17 -0700, quoth Andrey:

=>i have a newbie question about the file() function.
=>I have 2 daemons running on my linux box.
=>1 will record the IDs to a file - logs.txt
=>other 1 will open this file, read the IDs, and then "Clean up the
=>file" -logs.txt
=>Since these 2 daemons will run every 2-5mins, I think this will crash, isn't
=>it? When both daemons try to write to the file at the same time.
=>I am wondering if this won't crash, OR if there is some simple high-level
=>functions can lock the file while writing...
=>I also wonder if one side locked the file, what happens if the other side
=>try to open this locked file? raise error? so i also need to write a loop to
=>wait for the file to release locking?
=>seems a basic easy thing but i just cannot find an simple answer.
=>I always advoid this issue by putting them in mysql (well, fast and hassle
=>free for locking)

This is more of a unix question as opposed to a python question, but I'll
'splain it anyways

If two processes each have a channel opened to the same file then it is
very possible for the writes to collide. One way to mitigate the damage
would be to make sure that the two processes open the file for append
access instead of simple write access. This will ensure that the if
ProcessA writes data that ProcessB's next write won't overwrite the data
that A just wrote. But! It still won't work! You will still get
interleaving of data in your output file. They will all get out there, but
the data will on occasion look garbled because the interleaving won't look
like it was partitioned as the different processes intended.

So what's the solution? It turns out that data that is written to a pipe
(as opposed to data that is written to a file) is guaranteed to maintain
its partitioned integrity.

P1 >> fn
P2 >> fn

If you are the guy who starts P1 and P2 from a parent process(PP) then you
could structure your pipes so you end up with

PP > fn
/ \
/ \
P1 P2

and just make PP be a multiplexor.

Time flies like the wind. Fruit flies like a banana. Stranger things have .0.
happened but none stranger than this. Does your driver's license say Organ ..0
Donor?Black holes are where God divided by zero. Listen to me! We are all- 000
individuals! What if this weren't a hypothetical question?
steveo at
Reply With Quote

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off

Similar Threads
Thread Thread Starter Forum Replies Last Post
Single-instance daemons Jeffrey Barish Python 5 11-17-2008 05:15 AM
Python, Daemons and D-Bus PurpleServerMonkey Python 3 05-25-2008 11:40 AM
Thin errors out -> `gem_original_require': no such file to load --daemons Junkone Ruby 1 02-16-2008 03:00 PM
2 daemons write to a single file /w python file IO Andrey Python 0 09-12-2007 04:17 AM
How to write UNIX daemons in Python? Python 2 09-13-2006 03:38 PM