Velocity Reviews

Velocity Reviews (http://www.velocityreviews.com/forums/index.php)
-   Python (http://www.velocityreviews.com/forums/f43-python.html)
-   -   Writing a thread-safe class (http://www.velocityreviews.com/forums/t697978-writing-a-thread-safe-class.html)

Timothy Madden 09-11-2009 11:26 PM

Writing a thread-safe class
 
Hello

I would like to write a class with methods that can be accessed by many
threads at the same time.

For this I have a lock attribute in my class obtained with
threading.Lock(), in the constructor, and every method begins by
acquiring the lock and ends by releasing it

My problem is that the lock is still an attribute of the class, and the
very expression self.lock in the statement self.lock.acquire() is
performed before locking, thus not being thread-safe.

If I am correct self is a dictionary of object attributes, and if
another thread has the lock and creates a new attribute, or deletes one,
in the same time with my lookup for self.lock, than the lookup is
compromised.

How do people create thread-safe classes in python ?

Thank you,
Timothy Madden

Carl Banks 09-12-2009 01:31 AM

Re: Writing a thread-safe class
 
On Sep 11, 4:26*pm, Timothy Madden <terminato...@gmail.com> wrote:
> Hello
>
> I would like to write a class with methods that can be accessed by many
> threads at the same time.
>
> For this I have a lock attribute in my class obtained with
> threading.Lock(), in the constructor, and every method begins by
> acquiring the lock and ends by releasing it
>
> My problem is that the lock is still an attribute of the class, and the
> very expression self.lock in the statement self.lock.acquire() is
> performed before locking, thus not being thread-safe.
>
> If I am correct self is a dictionary of object attributes, and if
> another thread has the lock and creates a new attribute, or deletes one,
> in the same time with my lookup for self.lock, than the lookup is
> compromised.


You are not correct. Dictionary operation (like getting and setting
items) are atomic and limited to one thread at a time, thus thread-
safe.

(In fact, in CPython only one thread can execute Python code at a
time, in most cases. This is called the Global Interpreter Lock, or
GIL. If you search this newsgroup for information you will find a LOT
of discission about it. Other Python implementations such as Jython
may not use the GIL, but I am quite sure dictionary access is thread-
safe on those platforms also.)


> How do people create thread-safe classes in python ?


I think exactly the way you are doing it. (Some people might use
threading.RLock instead as a minor safety measure, but threading.Lock
will suffice.)


Carl Banks

Timothy Madden 09-12-2009 01:54 PM

Re: Writing a thread-safe class
 
Carl Banks wrote:
[...]
>
> You are not correct. Dictionary operation (like getting and setting
> items) are atomic and limited to one thread at a time, thus thread-
> safe.
>
> (In fact, in CPython only one thread can execute Python code at a
> time, in most cases. This is called the Global Interpreter Lock, or

[...]

I find that hard to believe, but I will look into it.

Thank you,
Timothy Madden

sturlamolden 09-12-2009 03:49 PM

Re: Writing a thread-safe class
 
On 12 Sep, 15:54, Timothy Madden <terminato...@gmail.com> wrote:

> I find that hard to believe, but I will look into it.


Carl Banks is correct.

There is a mutex called "the global interpreter lock" that takes care
of this. You can have multiple threads running, but access to the
Python interpreter is serialized.




News123 09-14-2009 06:33 PM

Re: Writing a thread-safe class
 
So what's recommended way for multicore machines?
Threads will probably only accelerate if the used C libraries are
releasing the GIL, right?

What's for example about PIL (Python Imaging library)?



Assuming, that the C library calls don't releas the GIL


Shoud I directly use use fork() and some kind of IPC? or are there some
special well established, recommendable commodity modules aiming for
rmultiprocessor job distribution?

So far I have just a single-core machine, but I'll be using a multicore
machine in the next weeks.

Then I'll probably find out

bye

N




sturlamolden wrote:
> On 12 Sep, 15:54, Timothy Madden <terminato...@gmail.com> wrote:
>
>> I find that hard to believe, but I will look into it.

>
> Carl Banks is correct.
>
> There is a mutex called "the global interpreter lock" that takes care
> of this. You can have multiple threads running, but access to the
> Python interpreter is serialized.
>
>
>


MRAB 09-14-2009 07:06 PM

Re: Writing a thread-safe class
 
News123 wrote:
> So what's recommended way for multicore machines?
> Threads will probably only accelerate if the used C libraries are
> releasing the GIL, right?
>
> What's for example about PIL (Python Imaging library)?
>
>
>
> Assuming, that the C library calls don't releas the GIL
>
>
> Shoud I directly use use fork() and some kind of IPC? or are there some
> special well established, recommendable commodity modules aiming for
> rmultiprocessor job distribution?
>
> So far I have just a single-core machine, but I'll be using a multicore
> machine in the next weeks.
>
> Then I'll probably find out
>

Have a look at the multiprocessing module.

> bye
>
> N
>
>
>
>
> sturlamolden wrote:
>> On 12 Sep, 15:54, Timothy Madden <terminato...@gmail.com> wrote:
>>
>>> I find that hard to believe, but I will look into it.

>> Carl Banks is correct.
>>
>> There is a mutex called "the global interpreter lock" that takes care
>> of this. You can have multiple threads running, but access to the
>> Python interpreter is serialized.
>>
>>
>>




All times are GMT. The time now is 09:28 PM.

Powered by vBulletin®. Copyright ©2000 - 2014, vBulletin Solutions, Inc.
SEO by vBSEO ©2010, Crawlability, Inc.