Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > updating dictionaries from/to dictionaries

Reply
Thread Tools

updating dictionaries from/to dictionaries

 
 
Brandon
Guest
Posts: n/a
 
      08-11-2008
Hi all,

I am not altogether experienced in Python, but I haven't been able to
find a good example of the syntax that I'm looking for in any tutorial
that I've seen. Hope somebody can point me in the right direction.

This should be pretty simple: I have two dictionaries, foo and bar.
I am certain that all keys in bar belong to foo as well, but I also
know that not all keys in foo exist in bar. All the keys in both foo
and bar are tuples (in the bigram form ('word1', 'word2)). I have to
prime foo so that each key has a value of 1. The values for the keys
in bar are variable integers. All I want to do is run a loop through
foo, match any of its keys that also exist in bar, and add those key's
values in bar to the preexisting value of 1 for the corresponding key
in foo. So in the end the key,value pairs in foo won't necessarily
be, for example, 'tuple1: 1', but also 'tuple2: 31' if tuple2 had a
value of 30 in bar.

I *think* the get method might work, but I'm not sure that it can work
on two dictionaries the way that I'm getting at. I thought that
converting the dictionaries to lists might work, but I can't see a way
yet to match the tuple key as x[0][0] in one list for all y in the
other list. There's just got to be a better way!

Thanks for any help,
Brandon
(trying hard to be Pythonic but isn't there yet)
 
Reply With Quote
 
 
 
 
Calvin Spealman
Guest
Posts: n/a
 
      08-11-2008
for k in foo:
foo[k] += bar.get(k, 0)

On Mon, Aug 11, 2008 at 3:27 AM, Brandon <(E-Mail Removed)> wrote:
> Hi all,
>
> I am not altogether experienced in Python, but I haven't been able to
> find a good example of the syntax that I'm looking for in any tutorial
> that I've seen. Hope somebody can point me in the right direction.
>
> This should be pretty simple: I have two dictionaries, foo and bar.
> I am certain that all keys in bar belong to foo as well, but I also
> know that not all keys in foo exist in bar. All the keys in both foo
> and bar are tuples (in the bigram form ('word1', 'word2)). I have to
> prime foo so that each key has a value of 1. The values for the keys
> in bar are variable integers. All I want to do is run a loop through
> foo, match any of its keys that also exist in bar, and add those key's
> values in bar to the preexisting value of 1 for the corresponding key
> in foo. So in the end the key,value pairs in foo won't necessarily
> be, for example, 'tuple1: 1', but also 'tuple2: 31' if tuple2 had a
> value of 30 in bar.
>
> I *think* the get method might work, but I'm not sure that it can work
> on two dictionaries the way that I'm getting at. I thought that
> converting the dictionaries to lists might work, but I can't see a way
> yet to match the tuple key as x[0][0] in one list for all y in the
> other list. There's just got to be a better way!
>
> Thanks for any help,
> Brandon
> (trying hard to be Pythonic but isn't there yet)
> --
> http://mail.python.org/mailman/listinfo/python-list
>




--
Read my blog! I depend on your acceptance of my opinion! I am interesting!
http://ironfroggy-code.blogspot.com/
 
Reply With Quote
 
 
 
 
John Machin
Guest
Posts: n/a
 
      08-11-2008
On Aug 11, 6:24*pm, "Calvin Spealman" <(E-Mail Removed)> wrote:
> for k in foo:
> * foo[k] += bar.get(k, 0)


An alternative:

for k in bar:
foo[k] += bar[k]

The OP asserts that foo keys are a superset of bar keys. If that
assertion is not true (i.e. there are keys in bar that are not in foo,
your code will silently ignore them whereas mine will cause an
exception to be raised (better behaviour IMHO). If the assertion is
true, mine runs faster (even when len(foo) == len(bar).
 
Reply With Quote
 
Steven D'Aprano
Guest
Posts: n/a
 
      08-11-2008
On Mon, 11 Aug 2008 00:27:46 -0700, Brandon wrote:

> This should be pretty simple: I have two dictionaries, foo and bar. I
> am certain that all keys in bar belong to foo as well, but I also know
> that not all keys in foo exist in bar. All the keys in both foo and bar
> are tuples (in the bigram form ('word1', 'word2)). I have to prime foo
> so that each key has a value of 1.


The old way:

foo = {}
for key in all_the_keys:
foo[key] = 1


The new way:

foo = dict.fromkeys(all_the_keys, 1)


> The values for the keys in bar are
> variable integers. All I want to do is run a loop through foo, match
> any of its keys that also exist in bar, and add those key's values in
> bar to the preexisting value of 1 for the corresponding key in foo. So
> in the end the key,value pairs in foo won't necessarily be, for example,
> 'tuple1: 1', but also 'tuple2: 31' if tuple2 had a value of 30 in bar.


Harder to say what you want to do than to just do it.

The long way:

for key in foo:
if bar.has_key(key):
foo[key] = foo[key] + bar[key]



Probably a better way:

for key, value in foo.iteritems():
foo[key] = value + bar.get(key, 0)



You should also investigate the update method of dictionaries. From an
interactive session, type:

help({}.update)

then the Enter key.



--
Steven
 
Reply With Quote
 
Brandon
Guest
Posts: n/a
 
      08-11-2008
"Harder to say what you want to do than to just do it."

The truly terrible thing is when you know that's the case even as
you're saying it. Thanks for the help, all!
 
Reply With Quote
 
John Machin
Guest
Posts: n/a
 
      08-11-2008
On Aug 12, 2:52 am, Steven D'Aprano <st...@REMOVE-THIS-
cybersource.com.au> wrote:
> On Mon, 11 Aug 2008 00:27:46 -0700, Brandon wrote:
> > This should be pretty simple: I have two dictionaries, foo and bar. I
> > am certain that all keys in bar belong to foo as well, but I also know
> > that not all keys in foo exist in bar. All the keys in both foo and bar
> > are tuples (in the bigram form ('word1', 'word2)). I have to prime foo
> > so that each key has a value of 1.

>

[snip]
> > The values for the keys in bar are
> > variable integers. All I want to do is run a loop through foo, match
> > any of its keys that also exist in bar, and add those key's values in
> > bar to the preexisting value of 1 for the corresponding key in foo. So
> > in the end the key,value pairs in foo won't necessarily be, for example,
> > 'tuple1: 1', but also 'tuple2: 31' if tuple2 had a value of 30 in bar.

>
> Harder to say what you want to do than to just do it.
>
> The long way:
>
> for key in foo:
> if bar.has_key(key):


dict.has_key(key) is nigh on obsolete since Python 2.2 introduced the
"key in dict" syntax.

> foo[key] = foo[key] + bar[key]


and foo[key] += bar[key] works in Python 2.1, maybe earlier.

>
> Probably a better way:
>
> for key, value in foo.iteritems():
> foo[key] = value + bar.get(key, 0)


Yeah, probably better than using has_key ...

>
> You should also investigate the update method of dictionaries. From an
> interactive session, type:
>
> help({}.update)
>
> then the Enter key.


I'm not sure what relevance dict.update has to the OP's problem.

Help is fine for when you need a reminder of the syntax of some method
you already know about. I'd suggest reading the manual of a modern
version of Python (http://docs.python.org/lib/typesmapping.html) to
get an overview of all the dict methods. The manual includes useful
information that isn't in help, like "a.has_key(k) Equivalent to k
in a, use that form in new code".
 
Reply With Quote
 
Brandon
Guest
Posts: n/a
 
      08-11-2008
I wasn't sure about the update method either, since AFAICT (not far)
the values would in fact update, not append as I needed them to. But
the iteritems and get combo definitely worked for me.

Thank you for the suggested link. I'm familiar with that page, but my
skill level isn't so far along yet that I can more or less intuitively
see how to combine methods, particularly in dictionaries. What would
be a dream for me is if somebody just had tons of use-case examples -
basically this post, condensed, for every potent combination of
dictionary methods. A guy can dream.
 
Reply With Quote
 
John Machin
Guest
Posts: n/a
 
      08-12-2008
On Aug 12, 9:14 am, Brandon <(E-Mail Removed)> wrote:
> I wasn't sure about the update method either, since AFAICT (not far)
> the values would in fact update, not append as I needed them to.


"append"? Don't you mean "add"???

> But
> the iteritems and get combo definitely worked for me.


Under some definition of "worked", yes, it would. What were your
selection criteria?

>
> Thank you for the suggested link. I'm familiar with that page, but my
> skill level isn't so far along yet that I can more or less intuitively
> see how to combine methods, particularly in dictionaries. What would
> be a dream for me is if somebody just had tons of use-case examples -
> basically this post, condensed, for every potent combination of
> dictionary methods. A guy can dream.


Nobody is going to write that, and if they did, what would you do?
Read it linearly, trying to find a match to your use-case? Forget
dreams. What you need to do is practice translating from your
requirements into Python, and it's not all that hard:

"run a loop through foo" -> for key in foo:
"match any of its keys that also exist in bar" -> if key in bar:
"add those key's values in bar to the preexisting value for the
corresponding key in foo" -> foo[key] += bar[key]

But you also need to examine your requirements:
(1) on a mechanical level, as I tried to point out in my first
response, if as you say all keys in bar are also in foo, you can
iterate over bar instead of and faster than iterating over foo.
(2) at a higher level, it looks like bar contains a key for every
possible bigram, and you are tallying actual counts in bar, and what
you want out for any bigram is (1 + number_of_occurrences) i.e.
Laplace adjustment. Are you sure you really need to do this two-dict
caper? Consider using only one dictionary (zot):

Initialise:
zot = {}

To tally:
if key in zot:
zot[key] += 1
else:
zot[key] = 1

Adjusted count (irrespective of whether bigram exists or not):
zot.get(key, 0) + 1

This method uses space proportional to the number of bigrams that
actually exist. You might also consider collections.defaultdict, but
such a dict may end up containing entries for keys that you ask about
(depending on how you ask), not just ones that exist.

HTH,
John
 
Reply With Quote
 
Brandon
Guest
Posts: n/a
 
      08-12-2008
John:

> "append"? Don't you mean "add"???


Yes, that is what I meant, my apologies.

> What you need to do is practice translating from your
> requirements into Python, and it's not all that hard:
>
> "run a loop through foo" -> for key in foo:
> "match any of its keys that also exist in bar" -> if key in bar:
> "add those key's values in bar to the preexisting value for the
> corresponding key in foo" -> foo[key] += bar[key]


Due to my current level of numbskullery, when I start to see things
like tuples as keys, the apparent ease of this evaporates in front of
my eyes! I know that I need more practice, though, and it will come.
>
> But you also need to examine your requirements:
> (1) on a mechanical level, as I tried to point out in my first
> response, if as you say all keys in bar are also in foo, you can
> iterate over bar instead of and faster than iterating over foo.
> (2) at a higher level, it looks like bar contains a key for every
> possible bigram, and you are tallying actual counts in bar, and what
> you want out for any bigram is (1 + number_of_occurrences) i.e.
> Laplace adjustment. Are you sure you really need to do this two-dict
> caper? Consider using only one dictionary (zot):
>
> Initialise:
> zot = {}
>
> To tally:
> if key in zot:
> zot[key] += 1
> else:
> zot[key] = 1
>
> Adjusted count (irrespective of whether bigram exists or not):
> zot.get(key, 0) + 1
>
> This method uses space proportional to the number of bigrams that
> actually exist. You might also consider collections.defaultdict, but
> such a dict may end up containing entries for keys that you ask about
> (depending on how you ask), not just ones that exist.


You are very correct about the Laplace adjustment. However, a more
precise statement of my overall problem would involve training and
testing which utilizes bigram probabilities derived in part from the
Laplace adjustment; as I understand the workflow that I should follow,
I can't allow myself to be constrained only to bigrams that actually
exist in training or my overall probability when I run through testing
will be thrown off to 0 as soon as a test bigram that doesn't exist in
training is encountered. Hence my desire to find all possible bigrams
in train (having taken steps to ensure proper set relations between
train and test). The best way I can currently see to do this is with
my current two-dictionary "caper", and by iterating over foo, not
bar

And yes, I know it seems silly to wish for that document with the use-
cases, but personally speaking, even if the thing is rather lengthy, I
would probably pick up better techniques for general knowledge by
reading through it and seeing the examples.

I actually think that there would be a good market (if only in
mindshare) for a thorough examination of the power of lists, nested
lists, and dictionaries (with glorious examples) - something that
might appeal to a lot of non-full time programmers who need to script
a lot but want to be efficient about it, yet don't want to deal with a
tutorial that unnecessarily covers all the aspects of Python. My
$0.027 (having gone up due to the commodities markets).

Thanks again for the input, I do appreciate it!

Brandon
 
Reply With Quote
 
John Machin
Guest
Posts: n/a
 
      08-12-2008
On Aug 12, 12:26 pm, Brandon <(E-Mail Removed)> wrote:

>
> You are very correct about the Laplace adjustment. However, a more
> precise statement of my overall problem would involve training and
> testing which utilizes bigram probabilities derived in part from the
> Laplace adjustment; as I understand the workflow that I should follow,
> I can't allow myself to be constrained only to bigrams that actually
> exist in training or my overall probability when I run through testing
> will be thrown off to 0 as soon as a test bigram that doesn't exist in
> training is encountered. Hence my desire to find all possible bigrams
> in train (having taken steps to ensure proper set relations between
> train and test).
> The best way I can currently see to do this is with
> my current two-dictionary "caper", and by iterating over foo, not
> bar


I can't grok large chunks of the above, especially these troublesome
test bigrams that don't exist in training but which you desire to find
in train(ing?).

However let's look at the mechanics: Are you now saying that your
original assertion "I am certain that all keys in bar belong to foo as
well" was not quite "precise"? If not, please explain why you think
you need to iterate (slowly) over foo in order to accomplish your
stated task.
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Updating JTable Atomically - block data changes while updating. news.rcn.com Java 2 12-10-2007 11:49 AM
Pickling dictionaries containing dictionaries: failing,recursion-style! lysdexia Python 6 12-02-2007 12:03 AM
TB1.5: Dictionaries =?windows-1252?Q?won=92t_install?= Hendrik Maryns Firefox 10 01-19-2006 04:36 PM
tools to create class dictionaries for documentation TJS ASP .Net 0 06-26-2005 08:49 PM
efficient updating of nested dictionaries omission9 Python 13 01-27-2004 02:45 AM



Advertisments