«

Ok, we've added list comprehensions to the language, and seen that
they were good. We've added generator expressions to the language, and
seen that they were good as well.

I'm left a bit confused, though - when would I use a list comp instead
of a generator expression if I'm going to require 2.4 anyway?

Thanks,
<mike
--
Mike Meyer <> http://www.mired.org/home/mwm/
Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information.

Mike Meyer wrote:
> Ok, we've added list comprehensions to the language, and seen that
> they were good. We've added generator expressions to the language, and
> seen that they were good as well.
>
> I'm left a bit confused, though - when would I use a list comp instead
> of a generator expression if I'm going to require 2.4 anyway?

Never. If you really need a list

list(x*x for x in xrange(10))

Sadly, we can't remove list comprehensions until 3.0.

--
Robert Kern


"In the fields of hell where the grass grows high
Are the graves of dreams allowed to die."
-- Richard Harter

Robert Kern wrote:
> Mike Meyer wrote:
>
>> Ok, we've added list comprehensions to the language, and seen that
>> they were good. We've added generator expressions to the language, and
>> seen that they were good as well.
>>
>> I'm left a bit confused, though - when would I use a list comp instead
>> of a generator expression if I'm going to require 2.4 anyway?
>
> Never. If you really need a list
>
> list(x*x for x in xrange(10))

Not quite true. If you discovered the unlikely scenario that the
construction of a list from the generator expression was an efficiency
bottleneck, you might choose a list comprehension -- they're slightly
faster when you really do want a list:

$ python -m timeit "list(x*x for x in xrange(10))"
100000 loops, best of 3: 6.54 usec per loop

$ python -m timeit "[x*x for x in xrange(10)]"
100000 loops, best of 3: 5.08 usec per loop

STeVe

Steven Bethard wrote:
> Robert Kern wrote:
>
>> Mike Meyer wrote:
>>
>>> Ok, we've added list comprehensions to the language, and seen that
>>> they were good. We've added generator expressions to the language, and
>>> seen that they were good as well.
>>>
>>> I'm left a bit confused, though - when would I use a list comp instead
>>> of a generator expression if I'm going to require 2.4 anyway?
>>
>> Never. If you really need a list
>>
>> list(x*x for x in xrange(10))
>
> Not quite true. If you discovered the unlikely scenario that the
> construction of a list from the generator expression was an efficiency
> bottleneck, you might choose a list comprehension -- they're slightly
> faster when you really do want a list:
>
> $ python -m timeit "list(x*x for x in xrange(10))"
> 100000 loops, best of 3: 6.54 usec per loop
>
> $ python -m timeit "[x*x for x in xrange(10)]"
> 100000 loops, best of 3: 5.08 usec per loop

Okay, okay, *almost* never.

However, I don't expect that speed relationship to hold past Python 2.4.

--
Robert Kern


"In the fields of hell where the grass grows high
Are the graves of dreams allowed to die."
-- Richard Harter

Robert Kern wrote:

> Mike Meyer wrote:
>
>> Ok, we've added list comprehensions to the language, and seen that
>> they were good. We've added generator expressions to the language, and
>> seen that they were good as well.
>>
>> I'm left a bit confused, though - when would I use a list comp instead
>> of a generator expression if I'm going to require 2.4 anyway?

If you want a list right away you'd use a list comprehension.
X =[i for i in something() if somethingelse()]
random.shuffle(X)
print x[23]

On the other hand it's generator expressions which should be used
only when the code can be written in as a pipe. For example a filter
of a -otherwise- very long list:

make_fractal_with_seed (x for x in range(100000000) if
fibonacci_prime (x))


>
>
> Never. If you really need a list
>
> list(x*x for x in xrange(10))
>
> Sadly, we can't remove list comprehensions until 3.0.
>

Why???
Then we should also remove:
x=[] to x=list()
x=[1,2,3] to x=list(1,2,3)

I think "list" is useful only:
1) to subclass it
2) to convert a list/tuple/string to a list, which is
done extremely fast.

But for iterators I find the list comprehension syntax nicer.


jfj

I jfj wrote:

>
> make_fractal_with_seed (x for x in range(100000000) if fibonacci_prime
> (x))
>

and this is stupendus.
At least range should be xrange.


jfj

Robert Kern wrote:

> jfj wrote:
>> 2) to convert a list/tuple/string to a list, which is
>> done extremely fast.
>
>
> Add "any iteratable". Genexps are iterables.
>

The thing is that when you want to convert a tuple to a list
you know already the size of it and you can avoid using append()
and expanding the list gradually. For iterables you can't avoid
appending items until StopIteration so using list() doesn't have
any advantage. The OP was about genexps vs list comprehensions
but this is about list() vs. list comprehensions.

> Possibly. I find them too similar with little enough to choose between them, hence the OP's question.

One solution is to forget about list(). If you want a list use [].
Unless you want to convert a tuple...

I think a better question would be "What do *generator expressions* do
that list comprehensions don't?". And always use list comprehensions
unless you want the extra bit.


jfj

jfj wrote:
> Robert Kern wrote:
>
>> Mike Meyer wrote:
>>
>>> Ok, we've added list comprehensions to the language, and seen that
>>> they were good. We've added generator expressions to the language, and
>>> seen that they were good as well.
>>>
>>> I'm left a bit confused, though - when would I use a list comp instead
>>> of a generator expression if I'm going to require 2.4 anyway?
>
> If you want a list right away you'd use a list comprehension.
> X =[i for i in something() if somethingelse()]
> random.shuffle(X)
> print x[23]
>
> On the other hand it's generator expressions which should be used
> only when the code can be written in as a pipe. For example a filter
> of a -otherwise- very long list:
>
> make_fractal_with_seed (x for x in range(100000000) if fibonacci_prime
> (x))
>
>> Never. If you really need a list
>>
>> list(x*x for x in xrange(10))
>>
>> Sadly, we can't remove list comprehensions until 3.0.
>>
>
> Why???
> Then we should also remove:
> x=[] to x=list()
> x=[1,2,3] to x=list(1,2,3)

Well, that last one doesn't work. Removing the empty list literal would
be inconsistent.

> I think "list" is useful only:
> 1) to subclass it
> 2) to convert a list/tuple/string to a list, which is
> done extremely fast.

Add "any iteratable". Genexps are iterables.

> But for iterators I find the list comprehension syntax nicer.

Possibly. I find them too similar with little enough to choose between
them, hence the OP's question.

--
Robert Kern


"In the fields of hell where the grass grows high
Are the graves of dreams allowed to die."
-- Richard Harter

jfj wrote:
> Robert Kern wrote:
>
>> jfj wrote:
>>
>>> 2) to convert a list/tuple/string to a list, which is
>>> done extremely fast.
>>
>> Add "any iteratable". Genexps are iterables.
>
> The thing is that when you want to convert a tuple to a list
> you know already the size of it and you can avoid using append()
> and expanding the list gradually. For iterables you can't avoid
> appending items until StopIteration so using list() doesn't have
> any advantage.

But no real disadvantage either.

> The OP was about genexps vs list comprehensions
> but this is about list() vs. list comprehensions.

Yes, and list(genexp) replaces list comprehensions rather handily.
There's very little reason to prefer list comprehensions over the
list(genexp) construction. If anything, it's pretty much a wash. That
was the OP's question:

[mike]
> when would I use a list comp instead of a generator expression if
> I'm going to require 2.4 anyway?

>> Possibly. I find them too similar with little enough to choose between
>> them, hence the OP's question.
>
> One solution is to forget about list(). If you want a list use [].
> Unless you want to convert a tuple...
>
> I think a better question would be "What do *generator expressions* do
> that list comprehensions don't?". And always use list comprehensions
> unless you want the extra bit.

<shrug> What we have now are two very similar constructs, one of which
is more general and subsumes nearly all the uses of the other. The
presence of both is confusing, as evidenced by the fact that the OP
asked his question. In the Great Breaking of Backwards Compatibility,
list comprehensions should go away.

--
Robert Kern


"In the fields of hell where the grass grows high
Are the graves of dreams allowed to die."
-- Richard Harter

Mike Meyer wrote:

> jfj <> writes:
>
>
>>I think a better question would be "What do *generator expressions* do
>>that list comprehensions don't?". And always use list comprehensions
>>unless you want the extra bit.
>
>
> As the OP, I can say why I didn't ask those questions.

Sorry. I was referring to the subject line

>
> Generator expressions don't build the entire list in memory before you
> have to deal with it. This makes it possible to deal with expressions
> that are to long to fit in memory.
>
> Which means that the real rule should be always use generator
> expressions, unless you *know* the expression will always fit in
> memory.
>

Consider this code which I also included the first reply:

x = [i for in in something()]
random.shuffle (x)
x.sort ()

Shuffle and sort are two examples where need *the entire list* to
work. Similarily for a dictionary where the values are small lists.
In this example using a generator buys you *nothing* because you
will immediately build a list.

So there are cases where we need the list as the product of an algorithm
and a generator is not good enough. In fact, in my experience with
python so far I'd say that those cases are the most common case.

That is the one question.
The other question is "why not list(generator) instead of[list
comprehension]?"

I guess that lists are *so important* that having a primary language
feature for building them is worth it. On the other hand "list()" is
not a primary operator of the python language. It is merely a builtin
function.


jfj