«

On 6/12/05, Isaac Gouy <> wrote:
> Austin Ziegler wrote:
> -snip-
> > Never *once* have I needed to implement an Ackermann function. Not
> > once. In my entire career. I look at the crap that is on alioth and
> > there's very little that represents common use. There's some neat
> > things -- the new DNA transformation ones -- but exactly how many
> > people will actually be using that in their work?
> Never once needed to implement a recursive function?

Not an Ackermann recursive. Only simple recursive. Most of the time, I
haven't even needed that. There's an important point there. Indeed, I
have sometimes gone in and changed recursive into iterative because it
was too expensive to implement as recursion.

But what are you really testing with this? If you want to test stack
winding and unwinding speeds, then you're probably testing the wrong
thing here. If you're testing something else, it's not clear. And
that's the damnable thing about the whole alioth shootout -- you
refuse to take an editorial stance anywhere (well, sort of) on the
interpretation of the numbers, leaving them to stand on their own --
which leads to people making stupid assumptions about them. You state
that you refuse to take an editorial stance, but you end up doing so
-- both by what tests you accept for the shootout and by what
implementations you'll accept.

I stand by what I said, though: the alioth shootout is a waste of
everyone's time. Frankly, I think it would be better if it were just
taken down.

> Some details are always specific to a problem domain and application,
> but the same representations and approaches are used across problem
> domains and across applications - yes the details are from DNA sequence
> analysis, but the programs process ascii strings and hashtables.

Perhaps. But more often than not, the approaches aren't perfectly portable.

-austin
--=20
Austin Ziegler *
* Alternate:

Austin Ziegler wrote:
> Huh. They didn't do it with benchmarks.

They did. I was there; you weren't. But don't let facts get in the way.

Steve

On 6/12/05, Steven Jenkins <> wrote:
> Austin Ziegler wrote:
> > Huh. They didn't do it with benchmarks.
> They did. I was there; you weren't. But don't let facts get in the way.

Bully for you. Were you there when they mixed unit systems, too? Did
they do *that* with benchmarks?

Sorry, but I don't buy the argument that they did this with
benchmarks. Measurements, perhaps, of the performance of their own
programs (or even prototypes), but *not* with benchmarks. In other
words, I think you're full of it and I'm calling you on it. Provide
some evidence that the decisions were made with benchmarks -- "A
standard by which something can be measured or judged."

Tell me that they did prototyping and I'll believe you. Tell me that
they measured *performance* and I'll believe you. Tell me that
benchmarks were involved in the process and I won't believe you unless
you provide some evidence of this claim.

-austin
--=20
Austin Ziegler *
* Alternate:

Austin Ziegler wrote:
> On 6/12/05, Isaac Gouy <> wrote:
> > Austin Ziegler wrote:
> > -snip-
> > > Never *once* have I needed to implement an Ackermann function. Not
> > > once. In my entire career. I look at the crap that is on alioth and
> > > there's very little that represents common use. There's some neat
> > > things -- the new DNA transformation ones -- but exactly how many
> > > people will actually be using that in their work?
> > Never once needed to implement a recursive function?
>
> Not an Ackermann recursive. Only simple recursive. Most of the time, I
> haven't even needed that. There's an important point there. Indeed, I
> have sometimes gone in and changed recursive into iterative because it
> was too expensive to implement as recursion.

And in some other language implementations that change is unecessary -
perhaps that's the point.

-snip-
> And that's the damnable thing about the whole alioth shootout -- you
> refuse to take an editorial stance anywhere (well, sort of) on the
> interpretation of the numbers, leaving them to stand on their own --
> which leads to people making stupid assumptions about them.
-snip-

Alioth Shootout - a Rorschach test for programmers.

Austin Ziegler wrote:
> On 6/12/05, Steven Jenkins <> wrote:
>>They did. I was there; you weren't. But don't let facts get in the way.
>
> Bully for you. Were you there when they mixed unit systems, too? Did
> they do *that* with benchmarks?

Yes, I was there. It's irrelevant to the discussion, but I was there.

> [badgering elided]
>
> Tell me that they did prototyping and I'll believe you. Tell me that
> they measured *performance* and I'll believe you. Tell me that
> benchmarks were involved in the process and I won't believe you unless
> you provide some evidence of this claim.

I've said what I have to say. Believe it or don't, as you see fit. We're
done.

Steve

Hello Michael,

MC> On 6/12/05, Alexandru Popescu <> wrote:


>> Also, I agree with the fact the optimization comes later in the
>> dev cycles. But we need to have the
>> doors open to optimize. Moreover, I read that in Ruby you can
>> always go and code the hot spots in C.
>> What if you don't have these resources? Which are the costs to
>> bring such a resource and make him
>> productive?

MC> How does this issue relate to *ruby*? The same charge could be levied
MC> at python, even J2EE. "What happens if you don't have the resources
MC> to speed up something slow?"

If a language is a few times faster then the risk factor that this
happens is just lower. This is a management descision.

There are many things to keep in balance when you start a project.
Development time is just one of them and often not that important.

I can just speak for the application domain where i worked for years:
This is products for the mass market. Here you don't have something
like "your customer". In this market you must look a lot about "your
competitor" and this makes it different. Spending a few month more on
development is just okay if the (peak !!) speed is well and
competitive.

This is a domain where Ruby is unfortunately not useable at the
moment (missing a good GUI toolkit is the second reason). And i would
like to see it possible to use it there too.


--
Best regards, emailto: scholz at scriptolutions dot com
Lothar Scholz http://www.ruby-ide.com
CTO Scriptolutions Ruby, PHP, Python IDE 's

In article <>,
Austin Ziegler <> wrote:
>On 6/12/05, Steven Jenkins <> wrote:
>> Austin Ziegler wrote:
>> What's more amazing is that these fools why buy into this "crap" have
>> made fundamental contributions to the theory and practice of digital
>> communications, image processing, and numerical analysis, practically
>> invented systems engineering, found volcanoes on Io and landslides on
>> Venus, communicated with spacecraft beyond the edge of the solar system,
>> put three rovers on the surface of Mars, and a whole bunch of other
>> stuff that will go into history books.
>
>Huh. They didn't do it with benchmarks. (And I could very easily point
>out that those same people have screwed up massively -- forgetting to
>convert between English units and Metric units?) Look closely at the
>people who espouse benchmarks. They're mostly marketers or fools who
>can't tell the difference.
>
>There are *real* measures to deal with; they're not benchmarks. They
>aren't and never have been.
>
>For a first person shooter, the real measure is "is the game fun?" The
>answer will be different for everyone, but there are some objective
>things that will break the "fun" factor for just about everyone.
>Frames Per Second. Load Time. These things should be as fast as they
>possibly can. Gigaflops never enters the question here. Nor does
>specmark or anything else like that.
>
>For image manipulations, they need to be quick. But not once does the
>Ackermann function ever enter the question.

No, but as you've said, you need quick response from a game in order for
it to be engaging. It comes down to frames per second (as you note).

>
>Never *once* have I needed to implement an Ackermann function. Not
>once. In my entire career.

No one is saying that you would need to. The Ackermann benchmark can be
a measure of how well your language handles recursion (of course it can
be written iteratively as well). Sure it may not be something that you'd
ever use, but if someone is perusing the benchmark results and sees that
Ruby falls way behind gawk in this particular benchmark, they're likely
to draw some conclusions. Now you and I might think that the conclusions
they draw are ot fair or accurate, but that doesn't matter. If I've
never encountered Ruby before and all I know of it is from the alioth
benchmarks, the conclusions I come to will not be positive.

>I look at the crap that is on alioth and
>there's very little that represents common use. There's some neat
>things -- the new DNA transformation ones -- but exactly how many
>people will actually be using that in their work?
>I won't. Wouldn't have my entire career.

Not everyone is doing web development. Some people work in interesting
areas like Bioinformatics. It's great that Ruby is finding a niche in web
development with Rails, but I really hope that doesn't translate into
Ruby only aspiring to be another PHP (ie. being perceived as a language
that is only useful for web development - I'm afraid that's already
starting to happen).

The fact is that it's in various scientific areas where you'll find the
most compute-intensive applications (the other area would be gaming), so
including DNA transformation benchmarks can be very informative.

>Measures that mattered to me
>at my last job were "how many bills can I generate in an hour?" At my
>current job "what's the average backup speed throughput for this?"
>
>If we're not getting the performance we need, we fix the damn problem.

Good; we can all agree on this

>We don't rely on "benchmarks" -- we rely on real world measurements of
>our real problems. Not on pseudo-crap like gigaflops or specmark or
>the speed of an airborne swallow. Actually, strike that. The last is
>useful.
>

Again, benchmarks don't tell the whole story, but unfortuneately many
will judge Ruby based on these benchmarks - that's just reality. What
are you going to do, buy banner ads on the alioth site which read "Don't
trust these benchmarks!". I don't think that'll work.

Since benchmarks tend to be a uniform measure (we can't all trot out
our 'real world' code to be translated into all the popular languages and
then tested and timed in each one - and if we did that would just become
the next benchmark, wouldn't it?). A good set of benchmarks needs to be
easily implemented in all of the languages being compared and they also
need to exercise a lot of different features that would show up in real
world programs. I'm going to give the alioth folks the benefit of the
doubt here and assume that they want to develop a 'good' set of
benchmarks that can be used to measure relative performance of various
languages. We can either dismiss their efforts with various
pejoratives (which will make people outside of our community wonder
what we're trying to hide) or help out.



Phil

On 6/13/05, Lothar Scholz <> wrote:
> Hello Michael,
>> On 6/12/05, Alexandru Popescu <> wrote:
>>> Also, I agree with the fact the optimization comes later in the
>>> dev cycles. But we need to have the doors open to optimize.
>>> Moreover, I read that in Ruby you can always go and code the hot
>>> spots in C. What if you don't have these resources? Which are
>>> the costs to bring such a resource and make him productive?
>> How does this issue relate to *ruby*? The same charge could be
>> levied at python, even J2EE. "What happens if you don't have the
>> resources to speed up something slow?"
> If a language is a few times faster then the risk factor that this
> happens is just lower. This is a management descision.

Sometimes yes, sometimes no. More often than I would like, but less
often, I imagine, than you think.

> There are many things to keep in balance when you start a project.
> Development time is just one of them and often not that important.

Actually, in my experience -- both with cusomisable largeware (a
billing system for ISPs) and with consumer-oriented software --
development time is one of the top items of concern. Not *the* top,
but definitely a component in the top (which is usually support
costs, a combination of tech support time, QA time, and developer
time, the cost of each rising).

> This is a domain where Ruby is unfortunately not useable at the
> moment (missing a good GUI toolkit is the second reason). And i
> would like to see it possible to use it there too.

I'd argue that Ruby's speed is secondary to the lack of a good
cross-platform GUI kit. I, too, would like to see Ruby perform
faster than it does. But I don't think that satisfying the alioth
shootout is going to make that happen. The problem *is* known, and
the solutions are at hand. If moving slowly.

-austin
--=20
Austin Ziegler *
* Alternate:

you wrote:
> If we keep telling ourselves that Ruby is 'fast
> enough' for our application (and it may well be) are we going to be
> sitting still while other languages improve performance?

luckily there is work on this.
what i don't get it why everyone
talks about it rather than actually working on it

Alex

On 6/15/05, James Britt <> wrote:
> Austin Ziegler wrote:
> > Right. I *personally* wouldn't consider what Steven has described in
> > his fascinating story as a benchmark. He may have *benchmarked* some
> > performance with his simulation, and used that as a baseline for
> > further performance tests, but the marketers have taken over the
> > word by and large.
> > The alioth "benchmarks" don't do what Steven did;
> > things like MIPS, TPS, SpecMARK, FLOPS, and other benchmark values
> > are pure marketing speak -- just like the alioth shootout.
> So, do we call that "benchmarketing"?

You know, it's a good thing that I didn't have a drink in my hands
when I read this.

Yeah. Benchmarketing. I like that.

-austin
--=20
Austin Ziegler *
* Alternate: