Velocity Reviews

Velocity Reviews (http://www.velocityreviews.com/forums/index.php)
-   Python (http://www.velocityreviews.com/forums/f43-python.html)
-   -   default behavior (http://www.velocityreviews.com/forums/t729519-default-behavior.html)

wheres pythonmonks 07-29-2010 06:12 PM

default behavior
 
Why is the default value of an int zero?

>>> x = int
>>> print x

<type 'int'>
>>> x()

0
>>>


How do I build an "int1" type that has a default value of 1?
[Hopefully no speed penalty.]
I am thinking about applications with collections.defaultdict.
What if I want to make a defaultdict of defaultdicts of lists? [I
guess my Perl background is showing -- I miss auto-vivification.]

W

Paul Rubin 07-29-2010 06:18 PM

Re: default behavior
 
wheres pythonmonks <wherespythonmonks@gmail.com> writes:
> How do I build an "int1" type that has a default value of 1?
> [Hopefully no speed penalty.]
> I am thinking about applications with collections.defaultdict.


You can supply an arbitary function to collections.defaultdict.
It doesn't have to be a class. E.g.

d = collections.defaultdict(lambda: 1)

will do what you are asking.

wheres pythonmonks 07-29-2010 06:35 PM

Re: default behavior
 
Thanks. I presume this will work for my nested example as well. Thanks again.

On Thu, Jul 29, 2010 at 2:18 PM, Paul Rubin <no.email@nospam.invalid> wrote:
> wheres pythonmonks <wherespythonmonks@gmail.com> writes:
>> How do I build an "int1" type that has a default value of 1?
>> [Hopefully no speed penalty.]
>> I am thinking about applications with collections.defaultdict.

>
> You can supply an arbitary function to collections.defaultdict.
> It doesn't have to be a class. *E.g.
>
> * *d = collections.defaultdict(lambda: 1)
>
> will do what you are asking.
> --
> http://mail.python.org/mailman/listinfo/python-list
>


John Nagle 07-29-2010 07:28 PM

Re: default behavior
 
On 7/29/2010 11:12 AM, wheres pythonmonks wrote:
> Why is the default value of an int zero?
>
>>>> x = int
>>>> print x

> <type 'int'>
>>>> x()

> 0
>>>>

>
> How do I build an "int1" type that has a default value of 1?



>>> class int1(object) :

.... def __init__(self) :
.... self.val = 1
.... def __call__(self) :
.... return(self.val)
....
>>> x = int1()
>>> x()

1

This isn't useful; you'd also have to define all the numeric operators
for this type. And then there are mixed-type conversion issues.

Inheriting from "int" is not too helpful, because you can't assign
to the value of the base class. "self=1" won't do what you want.

[Hopefully no speed penalty.]
In your dreams. Although all numbers in CPython are "boxed",
so there's more of a speed penalty with "int" itself than you
might expect. There are some C libraries for handling large
arrays if you really need to crunch numbers.

John Nagle


Peter Otten 07-30-2010 08:40 AM

Re: default behavior
 
wheres pythonmonks wrote:

> How do I build an "int1" type that has a default value of 1?
> [Hopefully no speed penalty.]
> I am thinking about applications with collections.defaultdict.


>>> from collections import defaultdict
>>> d = defaultdict(1 .conjugate)
>>> d["x"] += 2
>>> d["x"]

3

Isn't that beautiful? Almost like home;)

It is also fast:

$ python -m timeit -s"one = lambda: 1" "one()"
1000000 loops, best of 3: 0.213 usec per loop
$ python -m timeit -s"one = 1 .conjugate" "one()"
10000000 loops, best of 3: 0.0972 usec per loop

Micro-optimisation, the best excuse for ugly code...

Peter

Peter Otten 07-30-2010 10:21 AM

Re: default behavior
 
Duncan Booth wrote:

> Peter Otten <__peter__@web.de> wrote:
>
>>>>> from collections import defaultdict
>>>>> d = defaultdict(1 .conjugate)
>>>>> d["x"] += 2
>>>>> d["x"]

>> 3
>>
>> Isn't that beautiful? Almost like home;)
>>
>> It is also fast:
>>
>> $ python -m timeit -s"one = lambda: 1" "one()"
>> 1000000 loops, best of 3: 0.213 usec per loop
>> $ python -m timeit -s"one = 1 .conjugate" "one()"
>> 10000000 loops, best of 3: 0.0972 usec per loop
>>
>> Micro-optimisation, the best excuse for ugly code...
>>

>
> Nice one, but if you are going to micro-optimise why not save a few
> keystrokes while you're at it and use '1 .real' instead?


>>> 1 .real

1
>>> 1 .conjugate

<built-in method conjugate of int object at 0x1734298>
>>> 1 .conjugate()


real is a property, not a method. conjugate() was the first one that worked
that was not __special__. I think it has the added benefit that it's likely
to confuse the reader...

Peter

wheres pythonmonks 07-30-2010 11:59 AM

Re: default behavior
 
Instead of defaultdict for hash of lists, I have seen something like:


m={}; m.setdefault('key', []).append(1)

Would this be preferred in some circumstances?
Also, is there a way to upcast a defaultdict into a dict? I have also
heard some people use exceptions on dictionaries to catch key
existence, so passing in a defaultdict (I guess) could be hazardous to
health. Is this true?

W




On Fri, Jul 30, 2010 at 6:56 AM, Duncan Booth
<duncan.booth@invalid.invalid> wrote:
> Peter Otten <__peter__@web.de> wrote:
>> real is a property, not a method. conjugate() was the first one that
>> worked that was not __special__. I think it has the added benefit that
>> it's likely to confuse the reader...
>>

> Ah, silly me, I should have realised that.
>
> Yes, micro-optimisations that are also micro-obfuscations are always the
> best. :^)
>
> --
> Duncan Booth http://kupuguy.blogspot.com
> --
> http://mail.python.org/mailman/listinfo/python-list
>


Steven D'Aprano 07-30-2010 12:19 PM

Re: default behavior
 
On Fri, 30 Jul 2010 07:59:52 -0400, wheres pythonmonks wrote:

> Instead of defaultdict for hash of lists, I have seen something like:
>
>
> m={}; m.setdefault('key', []).append(1)
>
> Would this be preferred in some circumstances?


Sure, why not? Whichever you prefer.

setdefault() is a venerable old technique, dating back to Python 2.0, and
not a newcomer like defaultdict.


> Also, is there a way to upcast a defaultdict into a dict?


"Upcast"? Surely it is downcasting. Or side-casting. Or type-casting.
Whatever. *wink*

Whatever it is, the answer is Yes:

>>> from collections import defaultdict as dd
>>> x = dd(int)
>>> x[1] = 'a'
>>> x

defaultdict(<type 'int'>, {1: 'a'})
>>> dict(x)

{1: 'a'}



> I have also heard some people use
> exceptions on dictionaries to catch key existence, so passing in a
> defaultdict (I guess) could be hazardous to health. Is this true?


Yes, it is true that some people use exceptions on dicts to catch key
existence. The most common reason to do so is to catch the non-existence
of a key so you can add it:

try:
mydict[x] = mydict[x] + 1
except KeyError:
mydict[x] = 1


If mydict is a defaultdict with the appropriate factory, then the change
is perfectly safe because mydict[x] will not raise an exception when x is
missing, but merely return 0, so it will continue to work as expected and
all is good.

Of course, if you pass it an defaultdict with an *inappropriate* factory,
you'll get an error. So don't do that :) Seriously, you can't expect to
just randomly replace a variable with some arbitrarily different variable
and expect it to work. You need to know what the code is expecting, and
not break those expectations too badly.

And now you have at least three ways of setting missing values in a dict.
And those wacky Perl people say that Python's motto is "only one way to
do it" :)



--
Steven

Peter Otten 07-30-2010 12:33 PM

Re: default behavior
 
wheres pythonmonks wrote:

> Instead of defaultdict for hash of lists, I have seen something like:
>
>
> m={}; m.setdefault('key', []).append(1)
>
> Would this be preferred in some circumstances?


In some circumstances, sure. I just can't think of them at the moment.
Maybe if your code has to work in Python 2.4.

> Also, is there a way to upcast a defaultdict into a dict?


dict(some_defaultdict)

> I have also
> heard some people use exceptions on dictionaries to catch key
> existence, so passing in a defaultdict (I guess) could be hazardous to
> health. Is this true?


A problem could arise when you swap a "key in dict" test with a
"try...except KeyError". This would be an implementation detail for a dict
but affect the contents of a defaultdict:

>>> from collections import defaultdict
>>> def update(d):

.... for c in "abc":
.... try: d[c]
.... except KeyError: d[c] = c
....
>>> d = defaultdict(lambda:"-")
>>> update(d)
>>> d

defaultdict(<function <lambda> at 0x7fd4ce32a320>, {'a': '-', 'c': '-', 'b':
'-'})
>>> def update2(d):

.... for c in "abc":
.... if c not in d:
.... d[c] = c
....
>>> d = defaultdict(lambda:"-")
>>> update2(d)
>>> d

defaultdict(<function <lambda> at 0x7fd4ce32a6e0>, {'a': 'a', 'c': 'c', 'b':
'b'})

Peter


Steven D'Aprano 07-31-2010 03:47 AM

Re: default behavior
 
On Fri, 30 Jul 2010 08:34:52 -0400, wheres pythonmonks wrote:

> Sorry, doesn't the following make a copy?
>
>>>>> from collections import defaultdict as dd x = dd(int)
>>>>> x[1] = 'a'
>>>>> x

>> defaultdict(<type 'int'>, {1: 'a'})
>>>>> dict(x)

>> {1: 'a'}
>>
>>
>>

>
> I was hoping not to do that -- e.g., actually reuse the same underlying
> data.



It does re-use the same underlying data.

>>> from collections import defaultdict as dd
>>> x = dd(list)
>>> x[1].append(1)
>>> x

defaultdict(<type 'list'>, {1: [1]})
>>> y = dict(x)
>>> x[1].append(42)
>>> y

{1: [1, 42]}

Both the defaultdict and the dict are referring to the same underlying
key:value pairs. The data itself isn't duplicated. If they are mutable
items, a change to one will affect the other (because they are the same
item). An analogy for C programmers would be that creating dict y from
dict y merely copies the pointers to the keys and values, it doesn't copy
the data being pointed to.

(That's pretty much what the CPython implementation does. Other
implementations may do differently, so long as the visible behaviour
remains the same.)



> Maybe dict(x), where x is a defaultdict is smart? I agree that a
> defaultdict is safe to pass to most routines, but I guess I could
> imagine that a try/except block is used in a bit of code where on the
> key exception (when the value is absent) populates the value with a
> random number. In that application, a defaultdict would have no random
> values.


If you want a defaultdict with a random default value, it is easy to
provide:

>>> import random
>>> z = dd(random.random)
>>> z[2] += 0
>>> z

defaultdict(<built-in method random of Random object at 0xa01e4ac>, {2:
0.30707092626033605})


The point which I tried to make, but obviously failed, is that any piece
of code has certain expectations about the data it accepts. If take a
function that expects an int between -2 and 99, and instead decide to
pass a Decimal between 100 and 150, then you'll have problems: if you're
lucky, you'll get an exception, if you're unlucky, it will silently give
the wrong results. Changing a dict to a defaultdict is no different.

If you have code that *relies* on getting a KeyError for missing keys:

def who_is_missing(adict):
for person in ("Fred", "Barney", "Wilma", "Betty"):
try:
adict[person]
except KeyError:
print person, "is missing"

then changing adict to a defaultdict will cause the function to
misbehave. That's not unique to dicts and defaultdicts.



> Besides a slightly different favor, does the following have applications
> not covered by defaultdict?
>
> m.setdefault('key', []).append(1)


defaultdict calls a function of no arguments to provide a default value.
That means, in practice, it almost always uses the same default value for
any specific dict.

setdefault takes an argument when you call the function. So you can
provide anything you like at runtime.


> I think I am unclear on the difference between that and:
>
> m['key'] = m.get('key',[]).append(1)


Have you tried it? I guess you haven't, or you wouldn't have thought they
did the same thing.

Hint -- what does [].append(1) return?


--
Steven


All times are GMT. The time now is 07:37 AM.

Powered by vBulletin®. Copyright ©2000 - 2014, vBulletin Solutions, Inc.
SEO by vBSEO ©2010, Crawlability, Inc.