Velocity Reviews

Velocity Reviews (http://www.velocityreviews.com/forums/index.php)
-   Python (http://www.velocityreviews.com/forums/f43-python.html)
-   -   Safely add a key to a dict only if it does not already exist? (http://www.velocityreviews.com/forums/t956681-safely-add-a-key-to-a-dict-only-if-it-does-not-already-exist.html)

Steven D'Aprano 01-19-2013 04:07 AM

Safely add a key to a dict only if it does not already exist?
 
I wish to add a key to a dict only if it doesn't already exist, but do it
in a thread-safe manner.

The naive code is:

if key not in dict:
dict[key] = value


but of course there is a race condition there: it is possible that
another thread may have added the same key between the check and the
store.

How can I add a key in a thread-safe manner?

Any solutions must work with built-in dicts, I cannot use a custom dict
subclass.


--
Steven

Chris Rebert 01-19-2013 04:15 AM

Re: Safely add a key to a dict only if it does not already exist?
 
On Friday, January 18, 2013, Steven D'Aprano wrote:

> I wish to add a key to a dict only if it doesn't already exist, but do it
> in a thread-safe manner.
>
> The naive code is:
>
> if key not in dict:
> dict[key] = value
>
>
> but of course there is a race condition there: it is possible that


another thread may have added the same key between the check and the
> store.
>
> How can I add a key in a thread-safe manner?
>


I'm not entirely sure, but have you investigated dict.setdefault() ?


--
Cheers,
Chris
--
http://rebertia.com


Lie Ryan 01-19-2013 05:00 AM

Re: Safely add a key to a dict only if it does not already exist?
 
On 19/01/13 15:15, Chris Rebert wrote:
> On Friday, January 18, 2013, Steven D'Aprano wrote:
>
> I wish to add a key to a dict only if it doesn't already exist, but
> do it
> in a thread-safe manner.
>
> The naive code is:
>
> if key not in dict:
> dict[key] = value
>
>
> but of course there is a race condition there: it is possible that
>
> another thread may have added the same key between the check and the
> store.
>
> How can I add a key in a thread-safe manner?
>
>
> I'm not entirely sure, but have you investigated dict.setdefault() ?


dict.setdefault() was not atomic on older python version, they were made
atomic in Python 2.7.3 and Python 3.2.3.

See bug13521 in the issue tracker http://bugs.python.org/issue13521

PS: The bug tracker seems down at the moment, so pulled this from
Google's cache:
https://webcache.googleusercontent.c...&client=ubuntu


Vito De Tullio 01-19-2013 07:25 AM

Re: Safely add a key to a dict only if it does not already exist?
 
Steven D'Aprano wrote:

> I wish to add a key to a dict only if it doesn't already exist, but do it
> in a thread-safe manner.
>
> The naive code is:
>
> if key not in dict:
> dict[key] = value
>
>
> but of course there is a race condition there: it is possible that
> another thread may have added the same key between the check and the
> store.
>
> How can I add a key in a thread-safe manner?


using locks?

import threading

lock = threading.Lock()
with lock:
if key not in dict:
dict[key] = value


--
ZeD


Vito De Tullio 01-19-2013 07:27 AM

Re: Safely add a key to a dict only if it does not already exist?
 
Chris Rebert wrote:

>> How can I add a key in a thread-safe manner?

> I'm not entirely sure, but have you investigated dict.setdefault() ?


but how setdefault makes sense in this context? It's used to set a default
value when you try to retrieve an element from the dict, not when you try to
set a new one ...

--
ZeD


Mitya Sirenef 01-19-2013 07:35 AM

Re: Safely add a key to a dict only if it does not already exist?
 
On 01/19/2013 02:27 AM, Vito De Tullio wrote:
> Chris Rebert wrote:
>
>>> How can I add a key in a thread-safe manner?

>> I'm not entirely sure, but have you investigated dict.setdefault() ?

> but how setdefault makes sense in this context? It's used to set a default
> value when you try to retrieve an element from the dict, not when you try to
> set a new one ...
>


I guess setdefault with a sentinel default value, then set to your
new value if d[k] is sentinel?

- mitya


--
Lark's Tongue Guide to Python: http://lightbird.net/larks/


Peter Otten 01-19-2013 08:02 AM

Re: Safely add a key to a dict only if it does not already exist?
 
Vito De Tullio wrote:

> Chris Rebert wrote:
>
>>> How can I add a key in a thread-safe manner?

>> I'm not entirely sure, but have you investigated dict.setdefault() ?

>
> but how setdefault makes sense in this context? It's used to set a default
> value when you try to retrieve an element from the dict, not when you try
> to set a new one ...


But it also sets the value if the key is not found:

"""
setdefault(...)
D.setdefault(k[,d]) -> D.get(k,d), also set D[k]=d if k not in D
"""

>>> d = {}
>>> d.setdefault(1, 2)

2
>>> d

{1: 2}
>>> d.setdefault(1, 3)

2
>>> d

{1: 2}


It has been suggested that get_or_set() would have been a better name for
that functionality...


Vito De Tullio 01-19-2013 08:16 AM

Re: Safely add a key to a dict only if it does not already exist?
 
Peter Otten wrote:

>>>> How can I add a key in a thread-safe manner?
>>> I'm not entirely sure, but have you investigated dict.setdefault() ?

>>
>> but how setdefault makes sense in this context? It's used to set a
>> default value when you try to retrieve an element from the dict, not when
>> you try to set a new one ...

>
> But it also sets the value if the key is not found:


yeah, sure, but with a fixed value :)

I mean: if the value is not important, why bother at all trying not to
override it with an if or a lock or other tecniques? doing

d['key'] = 'fixed_value'

multiple times in different threads is not a problem in my eyes...

--
ZeD


Mitya Sirenef 01-19-2013 08:18 AM

Re: Safely add a key to a dict only if it does not already exist?
 
On 01/19/2013 02:35 AM, Mitya Sirenef wrote:
> On 01/19/2013 02:27 AM, Vito De Tullio wrote:
>> Chris Rebert wrote:
>>
>>>> How can I add a key in a thread-safe manner?
>>> I'm not entirely sure, but have you investigated dict.setdefault() ?

>> but how setdefault makes sense in this context? It's used to set a
>> default
>> value when you try to retrieve an element from the dict, not when you
>> try to
>> set a new one ...
>>

>
> I guess setdefault with a sentinel default value, then set to your
> new value if d[k] is sentinel?
>
> - mitya
>
>


Er, that makes no sense.. just setdefault to desired value. -m



--
Lark's Tongue Guide to Python: http://lightbird.net/larks/


Vito De Tullio 01-19-2013 08:19 AM

Re: Safely add a key to a dict only if it does not already exist?
 
Peter Otten wrote:

uhhmm.. I think I misread the example

>>>> d = {}
>>>> d.setdefault(1, 2)

> 2
>>>> d

> {1: 2}
>>>> d.setdefault(1, 3)

> 2
>>>> d

> {1: 2}


yeah, sure it can be useful for the OP...

--
ZeD



All times are GMT. The time now is 02:46 PM.

Powered by vBulletin®. Copyright ©2000 - 2014, vBulletin Solutions, Inc.
SEO by vBSEO ©2010, Crawlability, Inc.