Velocity Reviews > List of lists surprising behaviour

# List of lists surprising behaviour

candide
Guest
Posts: n/a

 06-17-2010
Let's the following code :

>>> t=[[0]*2]*3
>>> t

[[0, 0], [0, 0], [0, 0]]
>>> t[0][0]=1
>>> t

[[1, 0], [1, 0], [1, 0]]

Rather surprising, isn't it ? So I suppose all the subarrays reférence
the same array :

>>> id(t[0]), id(t[1]), id(t[2])

(3077445996L, 3077445996L, 3077445996L)
>>>

So what is the right way to initialize to 0 a 2D array ? Is that way
correct :

>>> t=[[0 for _ in range(2)] for _ in range(3)]

It seems there is no more trouble now :

>>> t

[[0, 0], [0, 0], [0, 0]]
>>> t[0][0]=1
>>> t

[[1, 0], [0, 0], [0, 0]]
>>>

Correct ?

Lie Ryan
Guest
Posts: n/a

 06-17-2010
On 06/17/10 20:21, candide wrote:
> Let's the following code :
>
>>>> t=[[0]*2]*3
>>>> t

> [[0, 0], [0, 0], [0, 0]]
>>>> t[0][0]=1
>>>> t

> [[1, 0], [1, 0], [1, 0]]
>
> Rather surprising, isn't it ? So I suppose all the subarrays reférence
> the same array :
>
>>>> id(t[0]), id(t[1]), id(t[2])

> (3077445996L, 3077445996L, 3077445996L)
>>>>

Yep, you're right. They share the same subarray if you uses
multiplication to build the array.

> So what is the right way to initialize to 0 a 2D array ? Is that way
> correct :
>>>> t=[[0 for _ in range(2)] for _ in range(3)]

Right again. That's the way to go. Although if the elements are
immutable, you can create the innermost array by multiplication:

t=[[0]*2 for _ in range(3)]

Matteo Landi
Guest
Posts: n/a

 06-17-2010
Yes you are. List comprehension makes you create list of lists without
reference-sharing. You should also find a recipe about that on the
python cookbook.

On Thu, Jun 17, 2010 at 12:21 PM, candide <(E-Mail Removed)> wrote:
> Let's the following code :
>
>>>> t=[[0]*2]*3
>>>> t

> [[0, 0], [0, 0], [0, 0]]
>>>> t[0][0]=1
>>>> t

> [[1, 0], [1, 0], [1, 0]]
>
> Rather surprising, isn't it ? So I suppose all the subarrays reférence the
> same array :
>
>>>> id(t[0]), id(t[1]), id(t[2])

> (3077445996L, 3077445996L, 3077445996L)
>>>>

>
>
> So what is the right way to initialize to 0 a 2D array ? Is that way correct
> *:
>
>
>>>> t=[[0 for _ in range(2)] for _ in range(3)]

>
> It seems there is no more trouble now :
>
>>>> t

> [[0, 0], [0, 0], [0, 0]]
>>>> t[0][0]=1
>>>> t

> [[1, 0], [0, 0], [0, 0]]
>>>>

>
> Correct ?
> --
> http://mail.python.org/mailman/listinfo/python-list
>

--
Matteo Landi
http://www.matteolandi.net/

Boris Borcic
Guest
Posts: n/a

 06-17-2010
candide wrote:
>
> So what is the right way to initialize to 0 a 2D array ? Is that way
> correct :
>
>
> >>> t=[[0 for _ in range(2)] for _ in range(3)]

That's overkill You can skip the inner loop by using a list display, eg

t=[[0,0] for _ in range(3)]

>
> It seems there is no more trouble now :
>
> >>> t

> [[0, 0], [0, 0], [0, 0]]
> >>> t[0][0]=1
> >>> t

> [[1, 0], [0, 0], [0, 0]]
> >>>

>
> Correct ?

J Kenneth King
Guest
Posts: n/a

 06-17-2010
candide <(E-Mail Removed)> writes:

> Let's the following code :
>
>>>> t=[[0]*2]*3
>>>> t

> [[0, 0], [0, 0], [0, 0]]
>>>> t[0][0]=1
>>>> t

> [[1, 0], [1, 0], [1, 0]]
>
> Rather surprising, isn't it ?

Not at all, actually.

I'd be surprised if the multiplication operator was aware of object
constructors. Even arrays are "objects" in Python. Should the
multiplication operator know how to instantiate three arrays from a
single array instance? What about an instance of a user-defined class?

> So I suppose all the subarrays refĂ©rence
> the same array :
>
>>>> id(t[0]), id(t[1]), id(t[2])

> (3077445996L, 3077445996L, 3077445996L)
>>>>

>

As they should.

>
> So what is the right way to initialize to 0 a 2D array ? Is that way
> correct :
>
>
>>>> t=[[0 for _ in range(2)] for _ in range(3)]

>
> It seems there is no more trouble now :
>
>>>> t

> [[0, 0], [0, 0], [0, 0]]
>>>> t[0][0]=1
>>>> t

> [[1, 0], [0, 0], [0, 0]]
>>>>

>
> Correct ?

>>> 2d_zero_vector = lambda len: [[0, 0] for _ in range(len)]
>>> t = 2d_zero_vector(3)
>>> print t

[[0, 0], [0, 0], [0, 0]]
>>> t[0][0] = 1
>>> print t

[[1, 0], [0, 0], [0, 0], [0, 0]]

(Of course, if you're doing matrix math you'll probably want to work
with numpy which has a function for doing just this)

bart.c
Guest
Posts: n/a

 06-17-2010

"J Kenneth King" <(E-Mail Removed)> wrote in message
news:(E-Mail Removed)...
> candide <(E-Mail Removed)> writes:
>
>> Let's the following code :
>>
>>>>> t=[[0]*2]*3
>>>>> t

>> [[0, 0], [0, 0], [0, 0]]
>>>>> t[0][0]=1
>>>>> t

>> [[1, 0], [1, 0], [1, 0]]
>>
>> Rather surprising, isn't it ?

>
> Not at all, actually.

The code is clearly trying to set only t[0][0] to 1, not t[1][0] and t[2][0]
as well.

This behaviour is quite scary actually, especially when t[0]=42 *does* work
as expected, while t[0][0]=42 is apparently duplicated. It appears
inconsistent.

> I'd be surprised if the multiplication operator was aware of object
> constructors. Even arrays are "objects" in Python. Should the
> multiplication operator know how to instantiate three arrays from a
> single array instance? What about an instance of a user-defined class?

Multiplication operators shouldn't need to be directly aware of any such
thing; it should just request that an object be duplicated without worrying

I don't know how Python does things, but an object should either specify a
special way of duplicating itself, or lend itself to some standard way of
doing so. (So for a list, it's just a question of copying the data in the
list, then recursively duplicating each new element..)

--
Bartc

Benjamin Kaplan
Guest
Posts: n/a

 06-17-2010
On Thu, Jun 17, 2010 at 4:20 PM, bart.c <(E-Mail Removed)> wrote:
>
> "J Kenneth King" <(E-Mail Removed)> wrote in message
> news:(E-Mail Removed)...
>>
>> candide <(E-Mail Removed)> writes:
>>
>>> Let's the following code :
>>>
>>>>>> t=[[0]*2]*3
>>>>>> t
>>>
>>> [[0, 0], [0, 0], [0, 0]]
>>>>>>
>>>>>> t[0][0]=1
>>>>>> t
>>>
>>> [[1, 0], [1, 0], [1, 0]]
>>>
>>> Rather surprising, isn't it ?

>>
>> Not at all, actually.

>
> The code is clearly trying to set only t[0][0] to 1, not t[1][0] and t[2][0]
> as well.
>
> This behaviour is quite scary actually, especially when t[0]=42 *does* work
> as expected, while t[0][0]=42 is apparently duplicated. It appears
> inconsistent.
>
>> I'd be surprised if the multiplication operator was aware of object
>> constructors. *Even arrays are "objects" in Python. *Should the
>> multiplication operator know how to instantiate three arrays from a
>> single array instance? *What about an instance of a user-defined class?

>
> Multiplication operators shouldn't need to be directly aware of any such
> thing; it should just request that an object be duplicated without worrying
>
> I don't know how Python does things, but an object should either specify a
> special way of duplicating itself, or lend itself to some standard way of
> doing so. (So for a list, it's just a question of copying the data in the
> list, then recursively duplicating each new element..)
>
> --
> Bartc

It's the recursively duplicating each element that's the problem. How
do you know when to stop?

rantingrick
Guest
Posts: n/a

 06-18-2010
On Jun 17, 6:44*pm, Benjamin Kaplan <(E-Mail Removed)> wrote:

> It's the recursively duplicating each element that's the problem. How
> do you know when to stop?

Thats easy, stack overflow!

Lie Ryan
Guest
Posts: n/a

 06-18-2010
On 06/18/10 09:20, bart.c wrote:
>
> "J Kenneth King" <(E-Mail Removed)> wrote in message
> news:(E-Mail Removed)...
>> candide <(E-Mail Removed)> writes:
>>
>>> Let's the following code :
>>>
>>>>>> t=[[0]*2]*3
>>>>>> t
>>> [[0, 0], [0, 0], [0, 0]]
>>>>>> t[0][0]=1
>>>>>> t
>>> [[1, 0], [1, 0], [1, 0]]
>>>
>>> Rather surprising, isn't it ?

>>
>> Not at all, actually.

>
> The code is clearly trying to set only t[0][0] to 1, not t[1][0] and
> t[2][0]
> as well.
>
> This behaviour is quite scary actually, especially when t[0]=42 *does* work
> as expected, while t[0][0]=42 is apparently duplicated. It appears
> inconsistent.

I agree, the behavior is often quite inconvenient, but I disagree that
it is inconsistent. List multiplication behavior is consistent with the
tenet: "objects are never copied unless explicitly requested"

Peeking further:
t = [[0] * 2] * 3
print id(t[0]) == id(t[1]) # True
print id(t[0][0]) == id(t[1][0]) # True

so, it is consistent (though it is quite inconvenient).

>> I'd be surprised if the multiplication operator was aware of object
>> constructors. Even arrays are "objects" in Python. Should the
>> multiplication operator know how to instantiate three arrays from a
>> single array instance? What about an instance of a user-defined class?

>
> Multiplication operators shouldn't need to be directly aware of any such
> thing; it should just request that an object be duplicated without
> worrying about how it's done.
>
> I don't know how Python does things, but an object should either specify
> a special way of duplicating itself, or lend itself to some standard way
> of doing so. (So for a list, it's just a question of copying the data in
> the list, then recursively duplicating each new element..)

That is inconsistent with the tenet. Moreover, the implicit copying
makes it quite difficult to reason about the program's behavior. How
would you propose this list should be copied:

class O(object):
def __init__(self):
self.attr = 0
self.attr += 1

b = [[O()] * 2] * 3

Steven D'Aprano
Guest
Posts: n/a

 06-18-2010
On Fri, 18 Jun 2010 00:20:30 +0100, bart.c wrote:

> The code is clearly trying to set only t[0][0] to 1, not t[1][0] and
> t[2][0] as well.

Trying to guess the motivation of the person writing code is tricky, but
in this case, that's a reasonable assumption. I can't think of any reason
why somebody would explicitly *want* that behaviour:

# Does it make sense to talk of anonymous aliases?
list_of_aliases = [[0]*2]*3
list_of_aliases[0][0] = 1
assert list_of_aliases[0][1] == 1

so it is a safe guess that anyone writing [[0]*2]*3 has probably made a
mistake.

However, I've certainly done something like this:

a = [0]*2
my_instance.items = a
a[0] = 1
assert my_instance.items[0] = 1

This is the same fundamental behaviour with the same cause: Python does
not copy objects unless you explicitly tell it to.

I cheerfully accept that the behaviour of [[0]*2]*3 is a Gotcha, but it
follows logically from Python's object model and assignment rules. If you
are surprised by it, it just goes to show that your understanding of
Python has at least one hole in it.

> This behaviour is quite scary actually, especially when t[0]=42 *does*
> work as expected, while t[0][0]=42 is apparently duplicated. It appears
> inconsistent.

Emphasis on the word "appears". It actually displays a deep consistency
with the language fundamentals.

If you're ever interviewing somebody for a position as Python developer,
this is a quick test to distinguish those who know the language from
those who know the language *well*.

>> I'd be surprised if the multiplication operator was aware of object
>> constructors. Even arrays are "objects" in Python. Should the
>> multiplication operator know how to instantiate three arrays from a
>> single array instance? What about an instance of a user-defined class?

>
> Multiplication operators shouldn't need to be directly aware of any such
> thing; it should just request that an object be duplicated without
> worrying about how it's done.

The multiplication operator is not a duplicator (copier). It is a
*repetition* operator: repeat the object N times, not make N copies.

> I don't know how Python does things,

Well there you go

> but an object should either specify
> a special way of duplicating itself, or lend itself to some standard way
> of doing so.

import copy
copy.copy(obj)

Dicts have a copy() method as a shortcut, and for lists you can use
slicing:

L = [1,2,3]
Lcopy = L[:]

> (So for a list, it's just a question of copying the data in
> the list, then recursively duplicating each new element..)

There's nothing "just" about that. Consider:

L = [1, 2]
L.append(L)

How would you copy that?

x = copy.deepcopy(L)

--
Steven