Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > Safely add a key to a dict only if it does not already exist?

Reply
Thread Tools

Safely add a key to a dict only if it does not already exist?

 
 
Steven D'Aprano
Guest
Posts: n/a
 
      01-19-2013
I wish to add a key to a dict only if it doesn't already exist, but do it
in a thread-safe manner.

The naive code is:

if key not in dict:
dict[key] = value


but of course there is a race condition there: it is possible that
another thread may have added the same key between the check and the
store.

How can I add a key in a thread-safe manner?

Any solutions must work with built-in dicts, I cannot use a custom dict
subclass.


--
Steven
 
Reply With Quote
 
 
 
 
Chris Rebert
Guest
Posts: n/a
 
      01-19-2013
On Friday, January 18, 2013, Steven D'Aprano wrote:

> I wish to add a key to a dict only if it doesn't already exist, but do it
> in a thread-safe manner.
>
> The naive code is:
>
> if key not in dict:
> dict[key] = value
>
>
> but of course there is a race condition there: it is possible that


another thread may have added the same key between the check and the
> store.
>
> How can I add a key in a thread-safe manner?
>


I'm not entirely sure, but have you investigated dict.setdefault() ?


--
Cheers,
Chris
--
http://rebertia.com

 
Reply With Quote
 
 
 
 
Lie Ryan
Guest
Posts: n/a
 
      01-19-2013
On 19/01/13 15:15, Chris Rebert wrote:
> On Friday, January 18, 2013, Steven D'Aprano wrote:
>
> I wish to add a key to a dict only if it doesn't already exist, but
> do it
> in a thread-safe manner.
>
> The naive code is:
>
> if key not in dict:
> dict[key] = value
>
>
> but of course there is a race condition there: it is possible that
>
> another thread may have added the same key between the check and the
> store.
>
> How can I add a key in a thread-safe manner?
>
>
> I'm not entirely sure, but have you investigated dict.setdefault() ?


dict.setdefault() was not atomic on older python version, they were made
atomic in Python 2.7.3 and Python 3.2.3.

See bug13521 in the issue tracker http://bugs.python.org/issue13521

PS: The bug tracker seems down at the moment, so pulled this from
Google's cache:
https://webcache.googleusercontent.c...&client=ubuntu

 
Reply With Quote
 
Vito De Tullio
Guest
Posts: n/a
 
      01-19-2013
Steven D'Aprano wrote:

> I wish to add a key to a dict only if it doesn't already exist, but do it
> in a thread-safe manner.
>
> The naive code is:
>
> if key not in dict:
> dict[key] = value
>
>
> but of course there is a race condition there: it is possible that
> another thread may have added the same key between the check and the
> store.
>
> How can I add a key in a thread-safe manner?


using locks?

import threading

lock = threading.Lock()
with lock:
if key not in dict:
dict[key] = value


--
ZeD

 
Reply With Quote
 
Vito De Tullio
Guest
Posts: n/a
 
      01-19-2013
Chris Rebert wrote:

>> How can I add a key in a thread-safe manner?

> I'm not entirely sure, but have you investigated dict.setdefault() ?


but how setdefault makes sense in this context? It's used to set a default
value when you try to retrieve an element from the dict, not when you try to
set a new one ...

--
ZeD

 
Reply With Quote
 
Mitya Sirenef
Guest
Posts: n/a
 
      01-19-2013
On 01/19/2013 02:27 AM, Vito De Tullio wrote:
> Chris Rebert wrote:
>
>>> How can I add a key in a thread-safe manner?

>> I'm not entirely sure, but have you investigated dict.setdefault() ?

> but how setdefault makes sense in this context? It's used to set a default
> value when you try to retrieve an element from the dict, not when you try to
> set a new one ...
>


I guess setdefault with a sentinel default value, then set to your
new value if d[k] is sentinel?

- mitya


--
Lark's Tongue Guide to Python: http://lightbird.net/larks/

 
Reply With Quote
 
Peter Otten
Guest
Posts: n/a
 
      01-19-2013
Vito De Tullio wrote:

> Chris Rebert wrote:
>
>>> How can I add a key in a thread-safe manner?

>> I'm not entirely sure, but have you investigated dict.setdefault() ?

>
> but how setdefault makes sense in this context? It's used to set a default
> value when you try to retrieve an element from the dict, not when you try
> to set a new one ...


But it also sets the value if the key is not found:

"""
setdefault(...)
D.setdefault(k[,d]) -> D.get(k,d), also set D[k]=d if k not in D
"""

>>> d = {}
>>> d.setdefault(1, 2)

2
>>> d

{1: 2}
>>> d.setdefault(1, 3)

2
>>> d

{1: 2}


It has been suggested that get_or_set() would have been a better name for
that functionality...

 
Reply With Quote
 
Vito De Tullio
Guest
Posts: n/a
 
      01-19-2013
Peter Otten wrote:

>>>> How can I add a key in a thread-safe manner?
>>> I'm not entirely sure, but have you investigated dict.setdefault() ?

>>
>> but how setdefault makes sense in this context? It's used to set a
>> default value when you try to retrieve an element from the dict, not when
>> you try to set a new one ...

>
> But it also sets the value if the key is not found:


yeah, sure, but with a fixed value

I mean: if the value is not important, why bother at all trying not to
override it with an if or a lock or other tecniques? doing

d['key'] = 'fixed_value'

multiple times in different threads is not a problem in my eyes...

--
ZeD

 
Reply With Quote
 
Mitya Sirenef
Guest
Posts: n/a
 
      01-19-2013
On 01/19/2013 02:35 AM, Mitya Sirenef wrote:
> On 01/19/2013 02:27 AM, Vito De Tullio wrote:
>> Chris Rebert wrote:
>>
>>>> How can I add a key in a thread-safe manner?
>>> I'm not entirely sure, but have you investigated dict.setdefault() ?

>> but how setdefault makes sense in this context? It's used to set a
>> default
>> value when you try to retrieve an element from the dict, not when you
>> try to
>> set a new one ...
>>

>
> I guess setdefault with a sentinel default value, then set to your
> new value if d[k] is sentinel?
>
> - mitya
>
>


Er, that makes no sense.. just setdefault to desired value. -m



--
Lark's Tongue Guide to Python: http://lightbird.net/larks/

 
Reply With Quote
 
Vito De Tullio
Guest
Posts: n/a
 
      01-19-2013
Peter Otten wrote:

uhhmm.. I think I misread the example

>>>> d = {}
>>>> d.setdefault(1, 2)

> 2
>>>> d

> {1: 2}
>>>> d.setdefault(1, 3)

> 2
>>>> d

> {1: 2}


yeah, sure it can be useful for the OP...

--
ZeD

 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Re: dict: retrieve the original key by key Christoph Groth Python 5 05-16-2011 12:47 AM
dict: retrieve the original key by key Christoph Groth Python 0 05-15-2011 08:28 AM
dict's as dict's key. Albert van der Horst Python 5 01-17-2010 05:26 PM
Inconsistency in dictionary behaviour: dict(dict) not calling __setitem__ Almad Python 8 12-14-2006 07:37 PM
how to safely extract dict values David Zaret Python 10 07-31-2006 11:34 AM



Advertisments