Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Ruby > python/ruby benchmark.

Reply
Thread Tools

python/ruby benchmark.

 
 
Phil Tomson
Guest
Posts: n/a
 
      06-10-2005
In article <(E-Mail Removed)>,
Jim Freeze <(E-Mail Removed)> wrote:
>* gabriele renzi <(E-Mail Removed)> [2005-06-10 09:35:28 +0900]:
>
>> I agree, It would be nice if ruby was faster. Luckily,
>> people are working on this.

>
>Don't go away thinking that X is faster than Ruby. Do
>your own tests. We tested Perl against Ruby in a real
>world problem (netlist parsing) and Ruby was faster
>than Perl by 30%. When we did a C implementation
>(which could now be done automatically with optimize)
>it was 100% faster (ie, took half the time.)


Are you saying that the Ruby netlist parser was 100% faster than the C
implementation or the other way around?

BTW: I recall someone at a PDX.rb meeting who said he had benchmarked
Ruby vs. OO-style Perl and Ruby was quite a bit faster (don't recall the
percentage). As I recall the explanation was that doing OO in perl
requires more levels of indirection and thus more function calls.

Phil
 
Reply With Quote
 
 
 
 
Jim Freeze
Guest
Posts: n/a
 
      06-10-2005
* Phil Tomson <(E-Mail Removed)> [2005-06-10 13:55:28 +0900]:

> In article <(E-Mail Removed)>,
> Jim Freeze <(E-Mail Removed)> wrote:
> >* gabriele renzi <(E-Mail Removed)> [2005-06-10 09:35:28 +0900]:
> >
> >> I agree, It would be nice if ruby was faster. Luckily,
> >> people are working on this.

> >
> >Don't go away thinking that X is faster than Ruby. Do
> >your own tests. We tested Perl against Ruby in a real
> >world problem (netlist parsing) and Ruby was faster
> >than Perl by 30%. When we did a C implementation
> >(which could now be done automatically with optimize)
> >it was 100% faster (ie, took half the time.)

>
> Are you saying that the Ruby netlist parser was 100% faster than the C
> implementation or the other way around?


The C implementation was Ruby C (my bad). So, the Ruby C
implementation ended up twice as fast as the Perl
implementation. Bad comparison you say. Well, in this
environment, we found over four netlist parsers written
in Perl. These parsers were intermingled with the netlist
munging code, so it was difficult to do abstraction and
pull out the netlist parser in Perl and optimize it.

But you say, well all that could and should have been done in
Perl. My response is, well, you had 10 years, what were you
waiting for?

The pragmatic nature of Ruby lent itself to writing better code,
more modular code, and thus the ability to optimize a particular
module muuuchh more than Perl.

--
Jim Freeze
Theory and practice are the same, in theory. -- Ryan Davis


 
Reply With Quote
 
 
 
 
Isaac Gouy
Guest
Posts: n/a
 
      06-10-2005
Vincent Foley wrote:
> As Gabriele mentionned, they implement a lot of stuff that is done in C
> in Ruby.


Specifically?

> Also, the tests must use the exact same algorithm, so this
> disadvantages Ruby is some situations. I say the test should compare
> same algorithms, but also the one best suited to the language.


Do this disadvantage Ruby more than PHP?
http://shootout.alioth.debian.org/gr...p&sort=fullcpu

Or more than Tcl, or more than GNU Smalltalk?
http://shootout.alioth.debian.org/gr...t&sort=fullcpu

 
Reply With Quote
 
Austin Ziegler
Guest
Posts: n/a
 
      06-10-2005
On 6/10/05, Isaac Gouy <(E-Mail Removed)> wrote:
> > Also, the tests must use the exact same algorithm, so this
> > disadvantages Ruby is some situations. I say the test should compare
> > same algorithms, but also the one best suited to the language.

> Do this disadvantage Ruby more than PHP?
> Or more than Tcl, or more than GNU Smalltalk?


When the algorithm, as entered, won't complete, yes. When compared
against a language that has tail recursion optimisation, or a possible
situation where optimisation of the compiler deals with recursion
intelligently? Yes.

The definition of Ackermann numbers is, I presume, well known. Because
some platforms can't modify their stack level (Windows), and because
the stack level isn't modified in the runs for the benchmark, there's
going to be an issue with the deep recursion. In this case, it would
be appropriate to reduce the level of recursion. But given the rules
mentioned, this isn't acceptable.

What's the *purpose* of something like the Ackermann test? Is it to
see how quickly the environment winds and unwinds the stack? How much
state it carries around?

I personally think that most of the shootout items are bogus in just
about every way. Don't get me wrong -- I'm not at all suggesting that
Ruby couldn't be faster. I'm just saying that benchmarks, like
statistics, are lies.

-austin
--=20
Austin Ziegler * http://www.velocityreviews.com/forums/(E-Mail Removed)
* Alternate: (E-Mail Removed)


 
Reply With Quote
 
Isaac Gouy
Guest
Posts: n/a
 
      06-10-2005
Austin Ziegler wrote:
> On 6/10/05, Isaac Gouy <(E-Mail Removed)> wrote:
> > > Also, the tests must use the exact same algorithm, so this
> > > disadvantages Ruby is some situations. I say the test should compare
> > > same algorithms, but also the one best suited to the language.

> > Do this disadvantage Ruby more than PHP?
> > Or more than Tcl, or more than GNU Smalltalk?

>
> When the algorithm, as entered, won't complete, yes. When compared
> against a language that has tail recursion optimisation, or a possible
> situation where optimisation of the compiler deals with recursion
> intelligently? Yes.


Ruby completes ackermann for N=7
http://shootout.alioth.debian.org/gr...llcpu#cputable

Which is how there can be a comparison between Ruby and Tcl on
ackermann
http://shootout.alioth.debian.org/gr...=fullcpu#ratio


> The definition of Ackermann numbers is, I presume, well known. Because
> some platforms can't modify their stack level (Windows), and because
> the stack level isn't modified in the runs for the benchmark, there's
> going to be an issue with the deep recursion.


We use Debian Linux, so tell us how to modify the stack level for Ruby.


-snip-
> I personally think that most of the shootout items are bogus in just
> about every way. Don't get me wrong -- I'm not at all suggesting that
> Ruby couldn't be faster. I'm just saying that benchmarks, like
> statistics, are lies.


Sounds like me in editorial mode.
http://shootout.alioth.debian.org/gr...d%20Benchmarks

 
Reply With Quote
 
Trey Campbell
Guest
Posts: n/a
 
      06-10-2005
On Jun 10, 2005, at 1:02 PM, Austin Ziegler wrote:
>
> I personally think that most of the shootout items are bogus in just
> about every way. Don't get me wrong -- I'm not at all suggesting that
> Ruby couldn't be faster. I'm just saying that benchmarks, like
> statistics, are lies.
>
> -austin
> --
> Austin Ziegler * (E-Mail Removed)
> * Alternate: (E-Mail Removed)
>
>


The most succinct way that I've heard it put is that there are "lies,
damn lies, and benchmarks".

Trey Campbell
(E-Mail Removed)




 
Reply With Quote
 
Austin Ziegler
Guest
Posts: n/a
 
      06-10-2005
On 6/10/05, Isaac Gouy <(E-Mail Removed)> wrote:
> Austin Ziegler wrote:
> > The definition of Ackermann numbers is, I presume, well known. Because
> > some platforms can't modify their stack level (Windows), and because
> > the stack level isn't modified in the runs for the benchmark, there's
> > going to be an issue with the deep recursion.

> We use Debian Linux, so tell us how to modify the stack level for Ruby.


I'm sorry you do. However, yours should be one of the ones that should
be able to modify the depth of the stack with ulimit.

If I cared enough to get more than annoyed at this damned shootout
with its inane implementation rules, I'd do more. But as it is, the
best I can tell people is that the Alioth shootout is bullshit.

-austin
--=20
Austin Ziegler * (E-Mail Removed)
* Alternate: (E-Mail Removed)


 
Reply With Quote
 
Isaac Gouy
Guest
Posts: n/a
 
      06-10-2005


Trey Campbell wrote:
> On Jun 10, 2005, at 1:02 PM, Austin Ziegler wrote:
> >
> > I personally think that most of the shootout items are bogus in just
> > about every way. Don't get me wrong -- I'm not at all suggesting that
> > Ruby couldn't be faster. I'm just saying that benchmarks, like
> > statistics, are lies.
> >
> > -austin
> > --
> > Austin Ziegler * (E-Mail Removed)
> > * Alternate: (E-Mail Removed)
> >
> >

>
> The most succinct way that I've heard it put is that there are "lies,
> damn lies, and benchmarks".
>
> Trey Campbell
> (E-Mail Removed)


There isn't with benchmarks per-se; there can be a problem with how one
chooses to interpret benchmarks.

 
Reply With Quote
 
Austin Ziegler
Guest
Posts: n/a
 
      06-10-2005
On 6/10/05, Isaac Gouy <(E-Mail Removed)> wrote:
> Trey Campbell wrote:
>> On Jun 10, 2005, at 1:02 PM, Austin Ziegler wrote:
>>> I personally think that most of the shootout items are bogus in
>>> just about every way. Don't get me wrong -- I'm not at all
>>> suggesting that Ruby couldn't be faster. I'm just saying that
>>> benchmarks, like statistics, are lies.

>> The most succinct way that I've heard it put is that there are
>> "lies, damn lies, and benchmarks".

> There isn't with benchmarks per-se; there can be a problem with
> how one chooses to interpret benchmarks.


No, it's the benchmarks themselves that are the problem. Remember
that NVidia got caught detecting a benchmark in their drivers and
optimizing output for the benchmark.

Similarly, this thread was started because someone came across your
language shootout benchmark page and assumed that Ruby is
dramatically slower in Real Life Examples because it's slower in the
benchmarks. Which, I can assure you, it most assuredly NOT.

You get it correct when you indicate that "the best benchmark is
your program." That's the only one that matters. It also lets you
measure benchmarks that can't be categorised in CPU seconds or
memory use or FLOPS or other nonsense like that.

It's hype, but the idea that Rails makes one 10x more productive is
part of the value of Ruby. Unless your program is of a special case
-- and a lot of these special cases are handled already -- then Ruby
programs will be developed faster and more easily maintained than
their non-Ruby counterparts. Enough so that execution speed is
something of a wash.

At any rate, I consider the Alioth shootout to be harmful to all
languages involved. There is no useful value provided by it,
especially as it does not permit language-appropriate modifications
to the algorithms in use.

-austin
--=20
Austin Ziegler * (E-Mail Removed)
* Alternate: (E-Mail Removed)


 
Reply With Quote
 
Isaac Gouy
Guest
Posts: n/a
 
      06-10-2005


Austin Ziegler wrote:
> On 6/10/05, Isaac Gouy <(E-Mail Removed)> wrote:
> > There isn't with benchmarks per-se; there can be a problem with
> > how one chooses to interpret benchmarks.

>
> No, it's the benchmarks themselves that are the problem. Remember
> that NVidia got caught detecting a benchmark in their drivers and
> optimizing output for the benchmark.
>
> Similarly, this thread was started because someone came across your
> language shootout benchmark page and assumed that Ruby is
> dramatically slower in Real Life Examples because it's slower in the
> benchmarks.


As I said - There isn't a problem with benchmarks per-se; there can be
a problem with how one chooses to interpret benchmarks.


> Which, I can assure you, it most assuredly NOT.


Not "dramatically slower" than what other language, for what specific
"Real Life Examples"?


> It's hype, but the idea that Rails makes one 10x more productive is
> part of the value of Ruby. Unless your program is of a special case
> -- and a lot of these special cases are handled already -- then Ruby
> programs will be developed faster and more easily maintained than
> their non-Ruby counterparts. Enough so that execution speed is
> something of a wash.


Faster to develop-in than what other languages - Smalltalk? Lisp?
Mathematica?


> At any rate, I consider the Alioth shootout to be harmful to all
> languages involved. There is no useful value provided by it,
> especially as it does not permit language-appropriate modifications
> to the algorithms in use.


What's a language-appropriate modification?
Have you ever asked for one to be accepted in the Alioth shootout?

 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off




Advertisments