Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > Make a small function thread safe

Reply
Thread Tools

Make a small function thread safe

 
 
Brad Tilley
Guest
Posts: n/a
 
      12-16-2011
Hey guys,

I have a C++ function that I'd like to replicate (as closely as
possible) in Python. Here's an example:

107 void increment_counter( unsigned int& counter )
108 {
109 boost::mutex::scoped_lock lock( counter_lock );
110 ++counter;
111 }

A thread locks the function on entrance and then releases it on exit.
What is the equivalent way to do this in Python?

Many thanks!

Brad
 
Reply With Quote
 
 
 
 
Tim Wintle
Guest
Posts: n/a
 
      12-16-2011
On Fri, 2011-12-16 at 05:21 -0800, Brad Tilley wrote:
> 107 void increment_counter( unsigned int& counter )
> 108 {
> 109 boost::mutex::scoped_lock lock( counter_lock );
> 110 ++counter;
> 111 }



with counter_lock:
counter += 1


.... where counter_lock is a threading.Lock instance.

(see docs for the threading module)

Tim

 
Reply With Quote
 
 
 
 
Tim Wintle
Guest
Posts: n/a
 
      12-16-2011
On Fri, 2011-12-16 at 09:24 -0500, Brad Tilley wrote:
> So something like this then:
>
> import threading
>
> shared_container = []
> lock = threading.Lock()
>
> class thread_example( threading.Thread ):
>
> def __init__( self ):
> threading.Thread.__init__ (self)
>
> def run(t):
> lock
> shared_container.append(t.name)


should be:
def run(t):
with lock:
shared_container.append(t.name)

(or lock.acquire() and lock.release() as you mentioned)

> # main
>
> threads = []
> for i in xrange(10):
> thread = thread_example()
> threads.append(thread)
>
> for thread in threads:
> thread.start()


you'll either need to lock again here, or join each thread:

for thread in threads:
thread.join()

> for item in shared_container:
> print item


Tim

 
Reply With Quote
 
Brad Tilley
Guest
Posts: n/a
 
      12-16-2011
On Dec 16, 9:36*am, Tim Wintle <(E-Mail Removed)> wrote:

> should be:
> * * * def run(t):
> * * * * * with lock:
> * * * * * * * shared_container.append(t.name)
>
> (or lock.acquire() and lock.release() as you mentioned)



Thanks Tim. The with statement is closer to the C++ code (IMO) more so
than the explicit acquire() and release() so I'll use that approach. I
appreciate your advice.

Brad
 
Reply With Quote
 
Lie Ryan
Guest
Posts: n/a
 
      12-16-2011
On 12/17/2011 01:30 AM, Brad Tilley wrote:
> Or perhaps run should look like this instead:
>
> def run(t):
> lock.acquire()
> shared_container.append(t.name <http://t.name>)
> lock.release()
>
> That seems a bit barbaric to me, not sure.


change that to:

def run(t):
with lock:
shared_container.append(t.name <http://t.name>)


the `with-statement` will call lock.acquire() and lock.release().

 
Reply With Quote
 
John Nagle
Guest
Posts: n/a
 
      12-17-2011
On 12/16/2011 2:08 PM, Lie Ryan wrote:
> On 12/17/2011 01:30 AM, Brad Tilley wrote:
>> Or perhaps run should look like this instead:
>>
>> def run(t):
>> lock.acquire()
>> shared_container.append(t.name <http://t.name>)
>> lock.release()
>>
>> That seems a bit barbaric to me, not sure.

>
> change that to:
>
> def run(t):
> with lock:
> shared_container.append(t.name <http://t.name>)
>
>
> the `with-statement` will call lock.acquire() and lock.release().
>


And, importantly, if you leave the "with" via an exception, the
lock will be unlocked.

John Nagle

 
Reply With Quote
 
RangerElf
Guest
Posts: n/a
 
      12-18-2011
Which is why the original .acquire() ... .release() idiom was wrong, this would better express the intent:

try:
lock.acquire()
shared_container.append(...)
finally:
lock.release()

But like everyone mentions, the with statement takes care of all that in a more readable and compact fashion.
 
Reply With Quote
 
Ian Kelly
Guest
Posts: n/a
 
      12-19-2011
On Sun, Dec 18, 2011 at 6:27 PM, Tim Delaney
<(E-Mail Removed)> wrote:
> On 18 December 2011 19:52, RangerElf <(E-Mail Removed)> wrote:
>>
>> Which is why the original .acquire() ... .release() idiom was wrong, this
>> would better express the intent:
>>
>> try:
>> *lock.acquire()
>> *shared_container.append(...)
>> finally:
>> *lock.release()

>
>
> No - this is very bad. The lock must be acquired outside the try: -
> otherwise if an exception is thrown while acquiring, you will try to release
> a lock that you have not acquired.
>
> Which again is why using with is a much better option - you can't make this
> kind of mistake.


Well, not unless you make the same mistake in the context manager itself.

from contextlib import contextmanager

@contextmanager
def bad_context_manager(lock):
try:
lock.acquire()
yield
finally:
lock.release()

class Lock(object):
def __init__(self):
self.is_locked = False
def acquire(self):
assert False
def release(self):
assert self.is_locked, "Tried to release lock without acquiring it"

with bad_context_manager(Lock()):
pass

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__
return self.gen.next()
File "<stdin>", line 7, in bad_context_manager
File "<stdin>", line 7, in release
AssertionError: Tried to release lock without acquiring it


Perhaps a (small) reason to avoid the contextmanager decorator and
implement __enter__ and __exit__ instead.
 
Reply With Quote
 
ting@thsu.org
Guest
Posts: n/a
 
      12-19-2011
On Dec 16, 8:21*am, Brad Tilley <(E-Mail Removed)> wrote:
> A thread locks the function on entrance and then releases it on exit.
> What is the equivalent way to do this in Python?


I'm not sure if this applies in your case, but much of the time, you
can use thread local storage, rather thread locking, in order to make
your code thread safe. You tend to run into far less bottleneck and
race condition issues by using thread local storage rather than thread
locking, whenever possible.
--
// T.Hsu
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
html5lib not thread safe. Is the Python SAX library thread-safe? John Nagle Python 5 03-12-2012 04:07 PM
Re: How to Easily Make a Function Thread-Safe CBFalconer C Programming 14 01-13-2009 10:45 PM
Re: How to Easily Make a Function Thread-Safe Ian Collins C Programming 4 01-13-2009 06:43 AM
os.ChDir() not thread-safe; was : Is tempfile.mkdtemp() thread-safe? Gabriel Rossetti Python 0 08-29-2008 08:30 AM
how to make SqlRoleProvider.ApplicationName thread safe Francis Reed ASP .Net 2 04-08-2006 03:03 AM



Advertisments