Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > benchmark

Reply
Thread Tools

benchmark

 
 
Jack
Guest
Posts: n/a
 
      08-07-2008
I know one benchmark doesn't mean much but it's still disappointing to see
Python as one of the slowest languages in the test:

http://blog.dhananjaynene.com/2008/0...-jruby-groovy/


 
Reply With Quote
 
 
 
 
Jake Anderson
Guest
Posts: n/a
 
      08-07-2008
Jack wrote:
> I know one benchmark doesn't mean much but it's still disappointing to see
> Python as one of the slowest languages in the test:
>
> http://blog.dhananjaynene.com/2008/0...-jruby-groovy/
>
>
> --
> http://mail.python.org/mailman/listinfo/python-list
>

Something to note though, The python version is ~ half the length of the
rest of them ;->
 
Reply With Quote
 
 
 
 
bearophileHUGS@lycos.com
Guest
Posts: n/a
 
      08-07-2008
On Aug 7, 2:05 am, "Jack" <(E-Mail Removed)> wrote:
> I know one benchmark doesn't mean much but it's still disappointing to see
> Python as one of the slowest languages in the test:
>
> http://blog.dhananjaynene.com/2008/0...rison-c-java-p...


That Python code is bad, it contains range() instead of xrange, the
big loop is in the main code instead of inside a function, uses ==
None, etc. That person can try this (with Psyco), I have changed very
little, the code is essentially the same:


import time, psyco
from psyco.classes import __metaclass__

class Person:
def __init__(self, count):
self.count = count
self.prev = None
self.next = None
def shout(self, shout, deadif):
if shout < deadif:
return shout + 1
self.prev.next = self.next
self.next.prev = self.prev
return 1

class Chain:
def __init__(self, size):
self.first = None
last = None
for i in xrange(size):
current = Person(i)
if self.first is None:
self.first = current
if last is not None:
last.next = current
current.prev = last
last = current
self.first.prev = last
last.next = self.first
def kill(self, nth):
current = self.first
shout = 1
while current.next != current:
shout = current.shout(shout, nth)
current = current.next
self.first = current
return current


def main():
ITER = 100000
start = time.time()
for i in xrange(ITER):
chain = Chain(40)
chain.kill(3)
end = time.time()
print 'Time per iteration = %s microseconds ' % ((end - start) *
1000000 / ITER)

psyco.full()
main()

us = microseconds
On my PC (that seems similar to his one) this version needs about 38.9
us/iter instead of 189.

On my PC the Java version takes 1.17 us, while the C++ version (with
MinGW 4.2.1) takes 9.8 us.
A raw D translation needs 14.34 us, while a cleaned up (that uses
structs, no getters/setters) needs 4.67 us.
I don't know why my C++ is so much slow (doing the same things to the C
++ version doesn't change its running time much).

Bye,
bearophile
 
Reply With Quote
 
Stefan Behnel
Guest
Posts: n/a
 
      08-07-2008
Jack wrote:
> I know one benchmark doesn't mean much but it's still disappointing to see
> Python as one of the slowest languages in the test:
>
> http://blog.dhananjaynene.com/2008/0...-jruby-groovy/


Just ignore that. If the code had been designed for Python from the start, it
would have performed a lot better.

Currently it looks like syntax-adapted Java code to me.

Stefan
 
Reply With Quote
 
Steven D'Aprano
Guest
Posts: n/a
 
      08-07-2008
On Thu, 07 Aug 2008 07:49:45 +0200, Stefan Behnel wrote:

> Jack wrote:
>> I know one benchmark doesn't mean much but it's still disappointing to
>> see Python as one of the slowest languages in the test:
>>
>> http://blog.dhananjaynene.com/2008/0...arison-c-java-

python-ruby-jython-jruby-groovy/
>
> Just ignore that. If the code had been designed for Python from the
> start, it would have performed a lot better.



I recommend folks copy and paste his code into an interactive session,
and watch the thousands of <__main__.Person object at 0xb7f18e2c> that
flash onto the screen. You want to know why it's so slow? That's part of
the reason.

Doing so on my computer gives a final result of:

"Time per iteration = 502.890818119 microseconds"

When I make a single, two character modification to the program
(inserting "t=" at the beginning of the line "chain.kill(3)"), I get this
instead:

"Time per iteration = 391.469910145 microseconds"

In other words, about 20% of the time he measures is the time taken to
print junk to the screen.


--
Steven
 
Reply With Quote
 
jlist
Guest
Posts: n/a
 
      08-07-2008
I think what makes more sense is to compare the code one most
typically writes. In my case, I always use range() and never use psyco.
But I guess for most of my work with Python performance hasn't been
a issue. I haven't got to write any large systems with Python yet, where
performance starts to matter.

> That Python code is bad, it contains range() instead of xrange, the
> big loop is in the main code instead of inside a function, uses ==
> None, etc. That person can try this (with Psyco), I have changed very
> little, the code is essentially the same:



 
Reply With Quote
 
alex23
Guest
Posts: n/a
 
      08-07-2008
Steven D'Aprano wrote:
> In other words, about 20% of the time he measures is the time taken to
> print junk to the screen.


Which makes his claim that "all the console outputs have been removed
so that the benchmarking activity is not interfered with by the IO
overheads" somewhat confusing...he didn't notice the output? Wrote it
off as a weird Python side-effect?

I find his reluctance to entertain more idiomatic implementations
particularly telling. It's seems he's less interested in actual
performance comparisons and more into showing that writing static lang
style code in dynamic langs is bad, which isn't really anything new to
anyone anywhere, I would've thought.

All up, this reminds me of last years benchmarking fiasco that
demonstrated Storm's ORM handling being -incredibly-faster than
SQLAlchemy's SQL expression handling, something that just didn't seem
to be born out by user experience. Eventually, Mike Beyer reverse
engineered the benchmarks to discover that, surprise surprise, the
tests -weren't equal-; in one test SQLA was issuing a commit per
insert, while Storm was performing -no commits whatsoever-.

Benchmarks, IMO, are like statistics. You can tweak them to prove
pretty much any position you already take.
 
Reply With Quote
 
Steven D'Aprano
Guest
Posts: n/a
 
      08-07-2008
On Thu, 07 Aug 2008 00:44:14 -0700, alex23 wrote:

> Steven D'Aprano wrote:
>> In other words, about 20% of the time he measures is the time taken to
>> print junk to the screen.

>
> Which makes his claim that "all the console outputs have been removed so
> that the benchmarking activity is not interfered with by the IO
> overheads" somewhat confusing...he didn't notice the output? Wrote it
> off as a weird Python side-effect?


Wait... I've just remembered, and a quick test confirms... Python only
prints bare objects if you are running in a interactive shell. Otherwise
output of bare objects is suppressed unless you explicitly call print.

Okay, I guess he is forgiven. False alarm, my bad.


--
Steven
 
Reply With Quote
 
Angel Gutierrez
Guest
Posts: n/a
 
      08-07-2008
Steven D'Aprano wrote:

> On Thu, 07 Aug 2008 00:44:14 -0700, alex23 wrote:
>
>> Steven D'Aprano wrote:
>>> In other words, about 20% of the time he measures is the time taken to
>>> print junk to the screen.

>>
>> Which makes his claim that "all the console outputs have been removed so
>> that the benchmarking activity is not interfered with by the IO
>> overheads" somewhat confusing...he didn't notice the output? Wrote it
>> off as a weird Python side-effect?

>
> Wait... I've just remembered, and a quick test confirms... Python only
> prints bare objects if you are running in a interactive shell. Otherwise
> output of bare objects is suppressed unless you explicitly call print.
>
> Okay, I guess he is forgiven. False alarm, my bad.
>
>

Well.. there must be somthing because this is what I got in a normal script
execution:

[angel@jaulat test]$ python iter.py
Time per iteration = 357.467989922 microseconds
[angel@jaulat test]$ vim iter.py
[angel@jaulat test]$ python iter2.py
Time per iteration = 320.306909084 microseconds
[angel@jaulat test]$ vim iter2.py
[angel@jaulat test]$ python iter2.py
Time per iteration = 312.917997837 microseconds

iter.py - Original script
iter2.py - xrange instead of range
iter2.py (2nd) - 't=' added

--
Angel
 
Reply With Quote
 
M8R-n7vorv@mailinator.com
Guest
Posts: n/a
 
      08-07-2008
On Aug 7, 6:38*am, (E-Mail Removed) wrote:
> On Aug 7, 2:05 am, "Jack" <(E-Mail Removed)> wrote:
>
> > I know one benchmark doesn't mean much but it's still disappointing to see
> > Python as one of the slowest languages in the test:

>
> >http://blog.dhananjaynene.com/2008/0...rison-c-java-p...

>
> That Python code is bad, it contains range() instead of xrange, the
> big loop is in the main code instead of inside a function, uses ==
> None, etc. That person can try this (with Psyco), I have changed very
> little, the code is essentially the same:
>


Yes, this was pointed out in the comments. I had updated the code to
use
xrange and is and is not instead of range, == and !=, which is how
the
benchmark got updated to 192 microseconds. Moving the main loop into
a main function resulted in no discernible difference.

Testing with psyco resulted in a time of 33 microseconds per
iteration.

> On my PC the Java version takes 1.17 us, while the C++ version (with
> MinGW 4.2.1) takes 9.8 us.
> A raw D translation needs 14.34 us, while a cleaned up (that uses
> structs, no getters/setters) needs 4.67 us.
> I don't know why my C++ is so much slow (doing the same things to the C
> ++ version doesn't change its running time much).
>
> Bye,
> bearophile


Wonder what optimisation level you are using. I to the best of my
recollection used -O3

Cheers,
Dhananjay
http://blog.dhananjaynene.com

 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
IE7 Benchmark jim_bowery@hotmail.com ASP .Net 1 10-08-2005 02:44 PM
Benchmark Roedy Green Java 0 06-10-2004 02:02 AM
processor benchmark on 2621 router sideshowblah Cisco 1 04-17-2004 10:10 PM
Looking for multithreaded applications for a Benchmark to a disributed JVM Assaf Harel Java 0 01-28-2004 01:16 AM
ASP.NET performance benchmark ke zhang ASP .Net 1 11-07-2003 03:02 PM



Advertisments