Velocity Reviews

Velocity Reviews (http://www.velocityreviews.com/forums/index.php)
-   Python (http://www.velocityreviews.com/forums/f43-python.html)
-   -   Generic Python Benchmark suite? (http://www.velocityreviews.com/forums/t712116-generic-python-benchmark-suite.html)

Anand Vaidya 01-18-2010 09:58 AM

Generic Python Benchmark suite?
 
Is there a generic python benchmark suite in active development? I am
looking forward to comparing some code on various python
implementations (primarily CPython 2.x, CPython 3.x, UnladenSwallow,
Psyco).

I am happy with something that gives me a relative number eg: ULS is
30% faster than CPy2.x etc

I found pybench which is probably not maintained actively.

What do you suggest?

PS: I think a benchmark should cover file / network, database I/O,
data structures (dict, list etc), object creation/manipulation,
numbers, measure looping inefficiencies, effects of caching (memcache
etc) at the minimum

Stefan Behnel 01-18-2010 10:30 AM

Re: Generic Python Benchmark suite?
 
Anand Vaidya, 18.01.2010 10:58:
> Is there a generic python benchmark suite in active development?
> [...]
> PS: I think a benchmark should cover file / network, database I/O,
> data structures (dict, list etc), object creation/manipulation,
> numbers, measure looping inefficiencies, effects of caching (memcache
> etc) at the minimum


That doesn't sound generic at all. Maybe you should prefer an application
based benchmark instead.

Stefan

Dotan Cohen 01-18-2010 03:42 PM

Re: Generic Python Benchmark suite?
 
> What do you suggest?
>


$ man time


--
Dotan Cohen

http://what-is-what.com
http://gibberish.co.il

Antoine Pitrou 01-18-2010 05:42 PM

Re: Generic Python Benchmark suite?
 
Le Mon, 18 Jan 2010 11:30:16 +0100, Stefan Behnel a √©crit¬*:
> Anand Vaidya, 18.01.2010 10:58:
>> Is there a generic python benchmark suite in active development? [...]
>> PS: I think a benchmark should cover file / network, database I/O,
>> data structures (dict, list etc), object creation/manipulation,
>> numbers, measure looping inefficiencies, effects of caching (memcache
>> etc) at the minimum

>
> That doesn't sound generic at all. Maybe you should prefer an
> application based benchmark instead.


Actually, it sounds much more generic that most of the benchmarks we
usually rely on :-)


Antoine Pitrou 01-18-2010 05:47 PM

Re: Generic Python Benchmark suite?
 
Le Mon, 18 Jan 2010 01:58:42 -0800, Anand Vaidya a √©crit¬*:
> Is there a generic python benchmark suite in active development? I am
> looking forward to comparing some code on various python implementations
> (primarily CPython 2.x, CPython 3.x, UnladenSwallow, Psyco).
>
> I am happy with something that gives me a relative number eg: ULS is 30%
> faster than CPy2.x etc
>
> I found pybench which is probably not maintained actively.


pybench is not abandoned. However, it only gets an addition now and then.
More importantly, it's a set of low-level microbenchmarks designed to
stress the execution cost of certain primitives. It won't give you the
answer to any high-level questions.

In the SVN sandbox (*) you'll find a couple of other benchmarks:
- stringbench, stressing string operations
- iobench, stressing common file I/O operations
- ccbench, trying to measure interpreter efficiency in the face of multi-
threaded workloads

(*) http://svn.python.org/view/sandbox/trunk/

Regards

Antoine.



Terry Reedy 01-18-2010 09:42 PM

Re: Generic Python Benchmark suite?
 
On 1/18/2010 4:58 AM, Anand Vaidya wrote:
> Is there a generic python benchmark suite in active development? I am
> looking forward to comparing some code on various python
> implementations (primarily CPython 2.x, CPython 3.x, UnladenSwallow,
> Psyco).


You might find this interesting if you have not seen it before:
http://code.google.com/p/unladen-swa...iki/Benchmarks


Anand Vaidya 01-19-2010 05:05 AM

Re: Generic Python Benchmark suite?
 
On Jan 19, 5:42*am, Terry Reedy <tjre...@udel.edu> wrote:
> On 1/18/2010 4:58 AM, Anand Vaidya wrote:
>
> > Is there a generic python benchmark suite in active development? I am
> > looking forward to comparing some code on various python
> > implementations (primarily CPython 2.x, CPython 3.x, UnladenSwallow,
> > Psyco).

>
> You might find this interesting if you have not seen it before:http://code.google.com/p/unladen-swa...iki/Benchmarks


@Antoine, Terry,

Thanks for the suggestions.

I will investigate those. I just ran the pybench, doesn't run on 3.x,
2to3 fails. I was hoping to locate something like Phoronix Test Suite
(that is a suite of microbenchmarks for Linux OS)

Regards
Anand

Antoine Pitrou 01-19-2010 12:04 PM

Re: Generic Python Benchmark suite?
 
Le Mon, 18 Jan 2010 21:05:26 -0800, Anand Vaidya a √©crit¬*:
> @Antoine, Terry,
>
> Thanks for the suggestions.
>
> I will investigate those. I just ran the pybench, doesn't run on 3.x,
> 2to3 fails.


You just have to use the pybench version that is bundled with 3.x (in the
Tools directory).




All times are GMT. The time now is 09:34 AM.

Powered by vBulletin®. Copyright ©2000 - 2014, vBulletin Solutions, Inc.
SEO by vBSEO ©2010, Crawlability, Inc.