Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > shouldn't list comprehension be faster than for loops?

Reply
Thread Tools

shouldn't list comprehension be faster than for loops?

 
 
Carlos Grohmann
Guest
Posts: n/a
 
      12-17-2009
Hello all

I am testing my code with list comprehensions against for loops.

the loop:

dipList=[float(val[1]) for val in datalist]
dip1=[]
for dp in dipList:
if dp == 90:
dip1.append(dp - 0.01)
else:
dip1.append(dp)

listcomp:

dipList=[float(val[1]) for val in datalist]
dip1=[(dp, dp-0.01)[dp==90.0] for dp in dipList]


Tenting the time spent by each approach (using time.clock()), with a
file with about 100,000 entries, I get 0.03s for the loop and 0.05s
for the listcomp.

thoughts?

TIA
Carlos

 
Reply With Quote
 
 
 
 
Alf P. Steinbach
Guest
Posts: n/a
 
      12-17-2009
* Carlos Grohmann:
> Hello all
>
> I am testing my code with list comprehensions against for loops.
>
> the loop:
>
> dipList=[float(val[1]) for val in datalist]
> dip1=[]
> for dp in dipList:
> if dp == 90:
> dip1.append(dp - 0.01)
> else:
> dip1.append(dp)
>
> listcomp:
>
> dipList=[float(val[1]) for val in datalist]
> dip1=[(dp, dp-0.01)[dp==90.0] for dp in dipList]
>
>
> Tenting the time spent by each approach (using time.clock()), with a
> file with about 100,000 entries, I get 0.03s for the loop and 0.05s
> for the listcomp.
>
> thoughts?


In the list comprehension you're constructing n tuples that you're not
constructing in the loop.

Have you tried this with

dip1 = [dp - 0.01 if dp == 90 else dp for dp in dipList]

?


Cheers & hth.,

- Alf
 
Reply With Quote
 
 
 
 
Tobias Weber
Guest
Posts: n/a
 
      12-17-2009
In article
<(E-Mail Removed)>,
Carlos Grohmann <(E-Mail Removed)> wrote:

> thoughts?


Well, for the loop you use an if statement, for the list you create a
tuple, so your benchmark is invalid. Try again.

Also, I wouldn't worry about speed and use what looks better in writing.

--
Tobias Weber
 
Reply With Quote
 
Carlos Grohmann
Guest
Posts: n/a
 
      12-17-2009

> Have you tried this with
>
> * *dip1 = [dp - 0.01 if dp == 90 else dp for dp in dipList]
>


Yes that is better! many thanks!



 
Reply With Quote
 
sturlamolden
Guest
Posts: n/a
 
      12-18-2009
On 17 Des, 18:37, Carlos Grohmann <(E-Mail Removed)> wrote:

> Tenting the time spent by each approach (using time.clock()), with a
> file with about 100,000 entries, I get 0.03s for the loop and 0.05s
> for the listcomp.
>
> thoughts?


Anything else being equal, list comprehensions will be the faster
becuase they incur fewer name and attribute lookups. It will be the
same as the difference between a for loop and a call to map. A list
comprehension is basically an enhancement of map.





 
Reply With Quote
 
sturlamolden
Guest
Posts: n/a
 
      12-18-2009
On 17 Des, 18:42, "Alf P. Steinbach" <(E-Mail Removed)> wrote:

> Have you tried this with
>
> * *dip1 = [dp - 0.01 if dp == 90 else dp for dp in dipList]


And for comparison with map:

map(lambda dp: dp - 0.01 if dp == 90 else dp, dipList)

 
Reply With Quote
 
Carl Banks
Guest
Posts: n/a
 
      12-18-2009
On Dec 17, 9:37*am, Carlos Grohmann <(E-Mail Removed)> wrote:
> Tenting the time spent by each approach (using time.clock()), with a
> file with about 100,000 entries, I get 0.03s for the loop and 0.05s
> for the listcomp.
>
> thoughts?


You shouldn't trust your intuition in things like this. Some features
were added to Python to make writing easier, not to make it run
faster. This time your intuition was correct. Next time, who knows?


Carl Banks
 
Reply With Quote
 
sturlamolden
Guest
Posts: n/a
 
      12-18-2009
On 17 Des, 18:37, Carlos Grohmann <(E-Mail Removed)> wrote:

> Tenting the time spent by each approach (using time.clock()), with a
> file with about 100,000 entries, I get 0.03s for the loop and 0.05s
> for the listcomp.
>
> thoughts?


Let me ask a retoric question:

- How much do you really value 20 ms of CPU time?






 
Reply With Quote
 
Ryan Kelly
Guest
Posts: n/a
 
      12-19-2009

> > Tenting the time spent by each approach (using time.clock()), with a
> > file with about 100,000 entries, I get 0.03s for the loop and 0.05s
> > for the listcomp.

>
> Anything else being equal, list comprehensions will be the faster
> becuase they incur fewer name and attribute lookups. It will be the
> same as the difference between a for loop and a call to map. A list
> comprehension is basically an enhancement of map.


Not so. If you use the "dis" module to peek at the bytecode generated
for a list comprehension, you'll see it's very similar to that generated
for an explicit for-loop. The byte-code for a call to map is very
different.

Basically: both a for-loop and a list-comp do the looping in python
bytecode, while a call to map will do the actual looping in C.

>>> def comper():

.... return [i*2 for i in xrange(10)]
....
>>>
>>> dis.dis(comper)

2 0 BUILD_LIST 0
3 DUP_TOP
4 STORE_FAST 0 (_[1])
7 LOAD_GLOBAL 0 (xrange)
10 LOAD_CONST 1 (10)
13 CALL_FUNCTION 1
16 GET_ITER
>> 17 FOR_ITER 17 (to 37)

20 STORE_FAST 1 (i)
23 LOAD_FAST 0 (_[1])
26 LOAD_FAST 1 (i)
29 LOAD_CONST 2 (2)
32 BINARY_MULTIPLY
33 LIST_APPEND
34 JUMP_ABSOLUTE 17
>> 37 DELETE_FAST 0 (_[1])

40 RETURN_VALUE
>>>
>>>
>>>
>>> def maper():

.... return map(lambda i: i*2,xrange(10))
....
>>> dis.dis(maper)

2 0 LOAD_GLOBAL 0 (map)
3 LOAD_CONST 1 (<code object ...)
6 MAKE_FUNCTION 0
9 LOAD_GLOBAL 1 (xrange)
12 LOAD_CONST 2 (10)
15 CALL_FUNCTION 1
18 CALL_FUNCTION 2
21 RETURN_VALUE
>>>




Cheers,

Ryan

--
Ryan Kelly
http://www.rfk.id.au | This message is digitally signed. Please visit
http://www.velocityreviews.com/forums/(E-Mail Removed) | http://www.rfk.id.au/ramblings/gpg/ for details


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)

iEYEABECAAYFAkssLDsACgkQfI5S64uP50q7kQCgmPy124Aj+1 6usI6xPxGDrTQI
aN0AnikJyb6vCaeE995RckkVpbw5LuUh
=awFd
-----END PGP SIGNATURE-----

 
Reply With Quote
 
Gregory Ewing
Guest
Posts: n/a
 
      12-19-2009
Ryan Kelly wrote:
>Someone else wrote:
>>It will be the
>>same as the difference between a for loop and a call to map.

>
> Not so. If you use the "dis" module to peek at the bytecode generated
> for a list comprehension, you'll see it's very similar to that generated
> for an explicit for-loop.


The usual advice is that if you have a built-in function that
does what you want done for each element, then using map() is
probably the fastest way.

However, if you need to create a Python function to pass to
map(), the list comprehension may well be faster, because it
avoids the cost of a Python function call per element.

--
Greg
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Re: faster than list.extend() Chris Rebert Python 2 11-17-2009 02:56 AM
List comprehension in if clause of another list comprehension Vedran Furac( Python 4 12-19-2008 01:35 PM
Appending a list's elements to another list using a list comprehension Debajit Adhikary Python 17 10-18-2007 06:45 PM
List comprehension returning subclassed list type? Shane Geiger Python 4 03-25-2007 09:34 AM
Is Set faster than List ck Java 7 03-09-2007 11:25 AM



Advertisments