Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > Writing a thread-safe class

Reply
Thread Tools

Writing a thread-safe class

 
 
Timothy Madden
Guest
Posts: n/a
 
      09-11-2009
Hello

I would like to write a class with methods that can be accessed by many
threads at the same time.

For this I have a lock attribute in my class obtained with
threading.Lock(), in the constructor, and every method begins by
acquiring the lock and ends by releasing it

My problem is that the lock is still an attribute of the class, and the
very expression self.lock in the statement self.lock.acquire() is
performed before locking, thus not being thread-safe.

If I am correct self is a dictionary of object attributes, and if
another thread has the lock and creates a new attribute, or deletes one,
in the same time with my lookup for self.lock, than the lookup is
compromised.

How do people create thread-safe classes in python ?

Thank you,
Timothy Madden
 
Reply With Quote
 
 
 
 
Carl Banks
Guest
Posts: n/a
 
      09-12-2009
On Sep 11, 4:26*pm, Timothy Madden <(E-Mail Removed)> wrote:
> Hello
>
> I would like to write a class with methods that can be accessed by many
> threads at the same time.
>
> For this I have a lock attribute in my class obtained with
> threading.Lock(), in the constructor, and every method begins by
> acquiring the lock and ends by releasing it
>
> My problem is that the lock is still an attribute of the class, and the
> very expression self.lock in the statement self.lock.acquire() is
> performed before locking, thus not being thread-safe.
>
> If I am correct self is a dictionary of object attributes, and if
> another thread has the lock and creates a new attribute, or deletes one,
> in the same time with my lookup for self.lock, than the lookup is
> compromised.


You are not correct. Dictionary operation (like getting and setting
items) are atomic and limited to one thread at a time, thus thread-
safe.

(In fact, in CPython only one thread can execute Python code at a
time, in most cases. This is called the Global Interpreter Lock, or
GIL. If you search this newsgroup for information you will find a LOT
of discission about it. Other Python implementations such as Jython
may not use the GIL, but I am quite sure dictionary access is thread-
safe on those platforms also.)


> How do people create thread-safe classes in python ?


I think exactly the way you are doing it. (Some people might use
threading.RLock instead as a minor safety measure, but threading.Lock
will suffice.)


Carl Banks
 
Reply With Quote
 
 
 
 
Timothy Madden
Guest
Posts: n/a
 
      09-12-2009
Carl Banks wrote:
[...]
>
> You are not correct. Dictionary operation (like getting and setting
> items) are atomic and limited to one thread at a time, thus thread-
> safe.
>
> (In fact, in CPython only one thread can execute Python code at a
> time, in most cases. This is called the Global Interpreter Lock, or

[...]

I find that hard to believe, but I will look into it.

Thank you,
Timothy Madden
 
Reply With Quote
 
sturlamolden
Guest
Posts: n/a
 
      09-12-2009
On 12 Sep, 15:54, Timothy Madden <(E-Mail Removed)> wrote:

> I find that hard to believe, but I will look into it.


Carl Banks is correct.

There is a mutex called "the global interpreter lock" that takes care
of this. You can have multiple threads running, but access to the
Python interpreter is serialized.



 
Reply With Quote
 
News123
Guest
Posts: n/a
 
      09-14-2009
So what's recommended way for multicore machines?
Threads will probably only accelerate if the used C libraries are
releasing the GIL, right?

What's for example about PIL (Python Imaging library)?



Assuming, that the C library calls don't releas the GIL


Shoud I directly use use fork() and some kind of IPC? or are there some
special well established, recommendable commodity modules aiming for
rmultiprocessor job distribution?

So far I have just a single-core machine, but I'll be using a multicore
machine in the next weeks.

Then I'll probably find out

bye

N




sturlamolden wrote:
> On 12 Sep, 15:54, Timothy Madden <(E-Mail Removed)> wrote:
>
>> I find that hard to believe, but I will look into it.

>
> Carl Banks is correct.
>
> There is a mutex called "the global interpreter lock" that takes care
> of this. You can have multiple threads running, but access to the
> Python interpreter is serialized.
>
>
>

 
Reply With Quote
 
MRAB
Guest
Posts: n/a
 
      09-14-2009
News123 wrote:
> So what's recommended way for multicore machines?
> Threads will probably only accelerate if the used C libraries are
> releasing the GIL, right?
>
> What's for example about PIL (Python Imaging library)?
>
>
>
> Assuming, that the C library calls don't releas the GIL
>
>
> Shoud I directly use use fork() and some kind of IPC? or are there some
> special well established, recommendable commodity modules aiming for
> rmultiprocessor job distribution?
>
> So far I have just a single-core machine, but I'll be using a multicore
> machine in the next weeks.
>
> Then I'll probably find out
>

Have a look at the multiprocessing module.

> bye
>
> N
>
>
>
>
> sturlamolden wrote:
>> On 12 Sep, 15:54, Timothy Madden <(E-Mail Removed)> wrote:
>>
>>> I find that hard to believe, but I will look into it.

>> Carl Banks is correct.
>>
>> There is a mutex called "the global interpreter lock" that takes care
>> of this. You can have multiple threads running, but access to the
>> Python interpreter is serialized.
>>
>>
>>


 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Class A contains class B, class B points to class A Joseph Turian C++ 5 12-30-2005 03:24 PM
Nested Class, Member Class, Inner Class, Local Class, Anonymous Class E11 Java 1 10-12-2005 03:34 PM
A parameterized class (i.e. template class / class template) is not a class? christopher diggins C++ 16 05-04-2005 12:26 AM
Any problems with writing the information into a file - Multi-users perform writing the same file at the same time ???? HNguyen ASP .Net 4 12-21-2004 01:53 PM
instanciate a class in a jar file with class.forname, while my main class is in another jar cyril Java 2 08-25-2004 06:55 AM



Advertisments