«

Peter Hansen wrote:

> Josiah Carlson wrote:
>
>> Checking the memory usage of Python before and after the creation of 1
>> million integer list, I get a /very/ consistant ~16 bytes per. That
>> would be 12 bytes for the object, and 4 bytes for each pointer in the
>> list.
>
> Please post the expression you are using, for comparison.

By the way, in case it wasn't clear, we have a disagreement (and perhaps
a misunderstanding) only about the meaning of the original code posted.
You believe it is allocating many integers. I believe it allocates
only a single one. Perhaps one of us is wrong. I hadn't read the code
in great detail, and still haven't, and perhaps that's my mistake.
Depending on your reply, I'll actually go back and look at it closely
and perhaps execute it myself and see the result (in terms of the list
produced, not in terms of the memory consumption). Maybe you will
do the same and we'll identify which of us is mistaken...

-Peter

>> You can't just store the integer. How would you differentiate between
>> an integer in a list and a pointer? Answer: you must use
>> PyIntObjects. Use the source.
>
>
> Python does not recognize anything called "pointers" at the language
> level, only internally.
>
> What I was saying is that the only PyIntObject created was one with
> the ob_ival of 1. Then a list containing one pointer to it was
> created. Then it was replicated "c" times. Only one integer.

Yeah, I was thinking of the case in generating a sequence of integers
via range. In that case, range(N) produces 12-byte intobjects and 4
byte pointers. In the case of c*[1), indeed you are right, one object
gets created, and some c pointers to that object are generated for the list.

Seems I got off topic and we are both right.

- Josiah

On Mon, 12 Apr 2004 07:22:26 -0400, Peter Hansen <> wrote:

>Peter Hansen wrote:
>
>> Josiah Carlson wrote:
>>
>>> Checking the memory usage of Python before and after the creation of 1
>>> million integer list, I get a /very/ consistant ~16 bytes per. That
>>> would be 12 bytes for the object, and 4 bytes for each pointer in the
>>> list.
>>
>> Please post the expression you are using, for comparison.
>
>By the way, in case it wasn't clear, we have a disagreement (and perhaps
>a misunderstanding) only about the meaning of the original code posted.
>You believe it is allocating many integers. I believe it allocates
>only a single one. Perhaps one of us is wrong. I hadn't read the code
>in great detail, and still haven't, and perhaps that's my mistake.
>Depending on your reply, I'll actually go back and look at it closely
>and perhaps execute it myself and see the result (in terms of the list
>produced, not in terms of the memory consumption). Maybe you will
>do the same and we'll identify which of us is mistaken...
>
>>> dict([(id(x),x) for x in [1]*100])
{7946208: 1}
>>> dict([(id(x),x) for x in range(5)])
{7939088: 0, 7946208: 1, 7943184: 4, 7945200: 2, 7944192: 3}
>>> dict([(id(x),x) for x in range(5)*10])
{7939088: 0, 7946208: 1, 7943184: 4, 7945200: 2, 7944192: 3}

>>> for v in -6,-5,0,1,99,100:
... print dict([(id(x),x) for x in [v for i in xrange(5)]])
...
{8053508: -6}
{7936720: -5}
{7939088: 0}
{7946208: 1}
{8042496: 99}
{8166456: 100}
>>> for v in -6,-5,0,1,99,100:
... print dict([(id(x),x) for x in [v*1 for i in xrange(5)]])
...
{8166384: -6, 8166444: -6, 8166492: -6, 8166360: -6, 8166468: -6}
{7936720: -5}
{7939088: 0}
{7946208: 1}
{8042496: 99}
{8166360: 100, 8166468: 100, 8166480: 100, 8166348: 100, 8166492: 100}

Regards,
Bengt Richter

....

Lupe wrote:

> I just would like to know if there is any limit to a list or tuple.

Yes. Since the universe only contains something like
10**85 particles of matter, the upper limit to a list
is roughly 2**(10**85) bits.

Sorry, I couldn't resist

You can store as many things in a list as you have
memory for. I suppose there is probably some
fundamental limit due to 32 bit addressing issues, but
are you sure you are really storing so many items you
are hitting that limit? I'd guess you aren't.

For example, the other day I was playing around with
Python, and just for a lark I created a list with
300,000,000 instances of None. It took about five
minutes, but it worked

Going back to your original problem:

> Each time the function is called, it compares an
> element of a list (which is a list too) to other
> elements, which all amounts to millions of
> comparisons.

Are you sure there isn't a more efficient algorithm for
what you are trying to do?

> The program is working for short lists but not for
> longer ones.

What do you mean it is working? It doesn't finish? It
is too slow? Or it gives the wrong answer?

If it is giving a wrong answer, there is a bug in your
code. If it is running too slowly, chances are there is
a better algorithm that will fix it.

--
Steven D'Aprano

Lupe wrote:

> I just would like to know if there is any limit to a list or tuple.

Yes. Since the universe only contains something like
10**85 particles of matter, the upper limit to a list
is roughly 2**(10**85) bits.

Sorry, I couldn't resist

You can store as many things in a list as you have
memory for. I suppose there is probably some
fundamental limit due to 32 bit addressing issues, but
are you sure you are really storing so many items you
are hitting that limit? I'd guess you aren't.

For example, the other day I was playing around with
Python, and just for a lark I created a list with
300,000,000 instances of None. It took about five
minutes, but it worked

Going back to your original problem:

> Each time the function is called, it compares an
> element of a list (which is a list too) to other
> elements, which all amounts to millions of
> comparisons.

Are you sure there isn't a more efficient algorithm for
what you are trying to do?

> The program is working for short lists but not for
> longer ones.

What do you mean it is working? It doesn't finish? It
is too slow? Or it gives the wrong answer?

If it is giving a wrong answer, there is a bug in your
code. If it is running too slowly, chances are there is
a better algorithm that will fix it.

--
Steven D'Aprano