Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > Writing to same file from two threads

Reply
Thread Tools

Writing to same file from two threads

 
 
Jens Thoms Toerring
Guest
Posts: n/a
 
      02-26-2013
Hi,

I noticed in someone elses program that it writes single
lines to the same file from (what I call for loss of a better
name) the "main thread" of the program and from a thread sub-
sequentally started. This got me worried if it might result
in garbled output (i.e. having some output from A inside a
line written by B or vice versae) because the "main thread" or
the other thread could be interrupted during a call of write().
Is this a valid concern (and thus locking the file object is
required before writing to it) or am I guaranteed that this
can't happen? In the latter case I would be grateful for an
explanation what mechanism is responsible for this never to
happen.
Thanks and best regards, Jens

PS: I already have determined experimentally that a context
switch definitely can happen between two calls of write()
(and I expected nothing else), what I'm worried about are
context switches somewhere within the very innards of what
write() does.
--
\ Jens Thoms Toerring ___ http://www.velocityreviews.com/forums/(E-Mail Removed)
\__________________________ http://toerring.de
 
Reply With Quote
 
 
 
 
Paul Rubin
Guest
Posts: n/a
 
      02-26-2013
(E-Mail Removed) (Jens Thoms Toerring) writes:
> in garbled output (i.e. having some output from A inside a
> line written by B or vice versae) because the "main thread" or


Yes they do get garbled like that. Preferred Python style is put a
single thread in charge of all the i/o to that file, and communicate
with it by message passing through Queue objects. That is safer than
directly using locks.
 
Reply With Quote
 
 
 
 
Jens Thoms Toerring
Guest
Posts: n/a
 
      02-26-2013
Paul Rubin <(E-Mail Removed)> wrote:
> (E-Mail Removed) (Jens Thoms Toerring) writes:
> > in garbled output (i.e. having some output from A inside a
> > line written by B or vice versae) because the "main thread" or


> Yes they do get garbled like that. Preferred Python style is put a
> single thread in charge of all the i/o to that file, and communicate
> with it by message passing through Queue objects. That is safer than
> directly using locks.


Thank you for confirmig my suspicion But you have induced
another question: why is using a Queue safer than locking (not
that I doubt that it might be more elegant etc.). Is it "safer"
because it's less likely that one gets it wrong (e.g. by for-
grtting to acquire the lock) or is there something inherently
unsafe about locks?

Thank you and best regards, Jens
--
\ Jens Thoms Toerring ___ (E-Mail Removed)
\__________________________ http://toerring.de
 
Reply With Quote
 
Antoine Pitrou
Guest
Posts: n/a
 
      02-27-2013
Jens Thoms Toerring <jt <at> toerring.de> writes:
>
> Paul Rubin <no.email <at> nospam.invalid> wrote:
> > jt <at> toerring.de (Jens Thoms Toerring) writes:
> > > in garbled output (i.e. having some output from A inside a
> > > line written by B or vice versae) because the "main thread" or

>
> > Yes they do get garbled like that. Preferred Python style is put a
> > single thread in charge of all the i/o to that file, and communicate
> > with it by message passing through Queue objects. That is safer than
> > directly using locks.

>
> Thank you for confirmig my suspicion But you have induced
> another question: why is using a Queue safer than locking (not
> that I doubt that it might be more elegant etc.). Is it "safer"
> because it's less likely that one gets it wrong (e.g. by for-
> grtting to acquire the lock) or is there something inherently
> unsafe about locks?


For the record, binary files are thread-safe in Python 3, but text files
are not.
Locks are safe if you use them well. As you point out, if you forget
to acquire your lock, or if you devise a situation where there is a
deadlock between competing locks, you can have difficult to diagnose
issues. Queues have their internal locking all done for you.

Regards

Antoine.


 
Reply With Quote
 
Jens Thoms Toerring
Guest
Posts: n/a
 
      02-27-2013
Antoine Pitrou <(E-Mail Removed)> wrote:
> Jens Thoms Toerring <jt <at> toerring.de> writes:
> >
> > Paul Rubin <no.email <at> nospam.invalid> wrote:
> > > jt <at> toerring.de (Jens Thoms Toerring) writes:
> > > > in garbled output (i.e. having some output from A inside a
> > > > line written by B or vice versae) because the "main thread" or

> >
> > > Yes they do get garbled like that. Preferred Python style is put a
> > > single thread in charge of all the i/o to that file, and communicate
> > > with it by message passing through Queue objects. That is safer than
> > > directly using locks.

> >
> > Thank you for confirmig my suspicion But you have induced
> > another question: why is using a Queue safer than locking (not
> > that I doubt that it might be more elegant etc.). Is it "safer"
> > because it's less likely that one gets it wrong (e.g. by for-
> > grtting to acquire the lock) or is there something inherently
> > unsafe about locks?


> For the record, binary files are thread-safe in Python 3, but text files
> are not.
> Locks are safe if you use them well. As you point out, if you forget
> to acquire your lock, or if you devise a situation where there is a
> deadlock between competing locks, you can have difficult to diagnose
> issues. Queues have their internal locking all done for you.


Thank you for your kind answers!
Best regards, Jens
--
\ Jens Thoms Toerring ___ (E-Mail Removed)
\__________________________ http://toerring.de
 
Reply With Quote
 
Steven D'Aprano
Guest
Posts: n/a
 
      02-28-2013
On Wed, 27 Feb 2013 13:26:18 +0000, Antoine Pitrou wrote:

> For the record, binary files are thread-safe in Python 3, but text files
> are not.


Where is this documented please?



--
Steven
 
Reply With Quote
 
Antoine Pitrou
Guest
Posts: n/a
 
      03-01-2013
Steven D'Aprano <steve+comp.lang.python <at> pearwood.info> writes:
>
> On Wed, 27 Feb 2013 13:26:18 +0000, Antoine Pitrou wrote:
>
> > For the record, binary files are thread-safe in Python 3, but text files
> > are not.

>
> Where is this documented please?


In the documentation, of course
http://docs.python.org/3.3/library/i...ulti-threading

Regards

Antoine.




 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Problem two users writing to the same file at same time id10t error ASP .Net 3 08-31-2009 06:33 PM
Two processes writing to the same XML file at the same time? darrel ASP .Net 2 04-05-2006 05:30 PM
Prevent two users from accessing the same file at the same time Shawn ASP .Net 2 02-19-2006 03:11 AM
Any problems with writing the information into a file - Multi-users perform writing the same file at the same time ???? HNguyen ASP .Net 4 12-21-2004 01:53 PM
Two TCP Servers (threads) on the same computer Sona Java 1 09-28-2003 06:01 PM



Advertisments