Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Ruby > Confirm my Performance Test Against Java?

Reply
Thread Tools

Confirm my Performance Test Against Java?

 
 
Ben Christensen
Guest
Posts: n/a
 
      08-20-2009
Edging away from the data driven performance analysis, I'd like to
explore some of my more subjective and opinionated thoughts on the
matter since several of your answers have touched on them. I'd very much
appreciate your rebuttals to my following comments (if you feel like
spending the time reading and responding).

I am obviously biased by my long use of the Java platform, and very
likely by my many years of focusing on processing large amounts of data,
writing search engines and other such applications very sensitive to
performance - and thus I have profiled and optimized virtually every
aspect of the stack and code - and to great gains for the user
experience. On the other hand, I've also connected to mainframes where
no amount of code optimization on my end could make any difference in
how things performed as I depended on the result from an external
resource, and I hid that from the user as much as possible with
asynchronous user interface magic as opposed to worrying about code
optimization, language choice or even hardware.



## Fast Enough ##

I often hear that Ruby is "fast enough" or that the performance
difference is not important since IO is generally the real source of
performance issues.

I understand that in certain cases - though when I see potential
performance degradation as multiples (2x, 3x, etc) as opposed to small
percentages, it makes me question the decision to use Ruby much more
than if we were talking percentages of 10-20% - such as 115ms vs 100ms
on a page response.

For example, a java environment my team has built provides SOAP/REST
webservices for product catalog search - and responsiveness is a very
significant measurement criteria of the success of the product and
system. Therefore, our average server side response time is something we
watch very closely.

It's difficult for me to accept the use of a language or platform which
means I'm taking a significant hit in performance - similar to the first
5-8 years of Java.

Even Java is still improving ... Java 5 to Java 6 was a very noticeable
improvement in performance at the JVM level (I've had 30-80% performance
increases from 5 to 6). Same hardware, same code - noticeable
improvement in performance and thus responsiveness on webservices,
webapps and shorter data processing times.

In the late 90s it was worth it to me to use Java and take the
performance hit - as the benefits were so strong over C for so many
things.

However, I'm still struggling to see the strong reasons to adopt Ruby
when I'm penalized performance-wise and the answer becomes "it's good
enough" or "network IO is usually the slowest aspect, so it's negligible
what Ruby adds" or "CPU is cheap".

Yes, "CPU is cheap", but that applies to Java as well. 6 months ago we
upgraded our hardware (systems were only 18 months old when we upgraded)
and shaved off another 30% just by taking advantage of new CPU
architectures and bus speeds that had changed in the previous year.

Perhaps in a straight-forward CRUD app where all that's being done is
retrieving/storing data in a database where IO truly is the bottleneck
that no amount of optimization can improve - then it doesn't matter and
"good enough" rings true where IO takes 100ms and the Ruby/Java portion
is only 10-20ms on top of the IO.



## C Extensions to Ruby ##

This feels akin to saying in the late 90s that to make Java perform
well, use JNI to use C. To me it defeats the whole purpose of the "Ruby
is simple and pleasant" paradigm. If I have to optimize it with C, then
it's no longer simple or a joy to use.



## Maintainability, Speed of Development and 'Enjoyment' ##

I hear "speed of development", "maintainability", and "enjoyment of
coding" as the reasons to move to Ruby - and to accept the negatives in
performance, tools, libraries etc.

I'm still not sold on these reasons - as I truly enjoy working in the
Java ecosystem of tools, projects, libraries etc - despite what may or
may not be "crufty" or verbose in some aspects of the language itself.

Nor am I convinced yet that managing a codebase over 5-10 years and 40+
developers is any easier with Ruby - Java's static typing and now its
generics (the polar oppostite approach of 'duck typing' in Ruby) are
actually very nice for readability, navigation of code, refactoring and
other such things on such large codebases when so much of it is from
other coders, teams, or just plain old and forgotten about.

Putting aside these more subjective decision points that I do not yet
have the experience to weigh in on with Ruby - if the performance impact
affects the end user, then that is in my opinion an objective point of
concern. Google and Amazon have both publicized how slow downs of
100-300ms on a user interface affects user experience and how much those
users utilize their sites. I certainly recognize that fact while
operating a hosted search engine. Speed of search directly impacts how
much someone will choose to use it. Slower performance equals increased
friction to use.

Also, no amount of "throwing hardware at it" will make Ruby faster than
throwing the same hardware at Java - which is ultimately I think the
biggest issue I have with the concept. If I throw the fastest hardware I
can at something, I want my user to get the biggest bang for the buck -
not just make up for me using a slower language.

As for "brevity" equalling "maintainability" and "less bugs" - I tend to
disagree when the pursuit results in code such as this example given:

puts "The number of tokens is: %d." % File.open(ARGV[0]){|f|
f.inject(0){|a,l| a+l.split.length } } ,
"It took #{(Time.now - start) * 1000} ms"

I find it intellectually stimulating and admirable at the power of what
is accomplished in such a short statement.

Understanding it however takes time and thought - and a certain level of
skill.

Perhaps your experiences are different - but most development teams have
a lot of more junior and intermediate developers than senior - and the
more verbose, easy to read, simple to understand code is far more
preferable - even for myself when I must review, debug and profile the
code of dozens of others - as opposed to something that looks like an
academic puzzle to unravel.




## Concluding Thoughts ##

I recognize that many of my concerns are similar to the C versus Java
argument of 10 years past.

Moving to Java from C had a variety of very significant benefits
however, amongst them being: garbage collection and vastly simplified
memory management, no need to worry about pointers, "write once, run
[mostly] everywhere" (as long as you were talking server-side and not
desktop where Java is miserable) and APIs designed better to address the
networked world of the then new (to the common public anyways) internet.

In moving from Java to Ruby I don't see the benefits as strongly
motivating - and therefore am much less willing to accept performance
penalties.

In short, after waiting through 10+ years of maturation to get the Java
performance now enjoyed, it feels somewhat odd to step back
significantly in performance, tool maturity and available libraries.

What types of applications and codebases do you feel truly are served
best by Ruby - and therefore not in need of the highest performance
given to the end-user?

--
Posted via http://www.ruby-forum.com/.

 
Reply With Quote
 
 
 
 
pharrington
Guest
Posts: n/a
 
      08-20-2009
> pharrington, in your response you stated:
>
> "as the code that happens is neither the most "elegant" *nor* fastest
> Ruby can do."
>
> Can you please provide me a re-write of the Ruby code I used that is
> elegant and fast so I can learn from you?


Hi, I was gone for the day, but numerous people in the thread already
did both, so

> @Mike
>
> Thank you for providing the Gist link to a file.
> (http://gist.github.com/170476)
>
> However, the changes don't improve the performance when I take into
> account what was removed and I had in there on purpose. Take note of
> item #2 below.
>
> 1) Object structure
>
> The modified code removed all of the class/object structure, which I
> purposefully had in there to simulate this being an object within a
> larger project.


Sticking methods in a class really doesn't simulate an object in a
larger project at all; its just methods in a class. The general
concept of ***larger project*** isn't really something you can factor
out; its just how the code ends up needing to be structured for the
task at hand.

> That being said, converting the lines of code we're discussing for
> performance into a script means nothing to this discussion - but I
> purposefully am writing the code in an OO style with classes as opposed
> to scripts.


Again, coders code to solve the task at hand. When you say "Yes, this
test is representative of some of the types of applications
and necessary data processing I have current applications doing and am
needing in some future ones" we look at what the code *does*, not
guess at a vague idea of a large project which defining a class is
apparently supposed to imply. The code you posted counts words in a
file. Nothing more; nothing less.


> I was also purposefully making the Java and Ruby versions as similar to
> each other so as to allow a performance comparison to be done with as
> little difference as possible in approaching the code.


If you want to write Java code, then why use something that isn't
Java? This is like taking a C program, trying to emulate as closely as
possible, line-for-line the C code in Erlang (using mutable data and
everything) and then dismissing Erlang because it's worse at being C
than C. Different languages express solutions to different problems in
different ways; that's the whole point.

I guess you just wanted to know whether or not the Ruby interpreter is
generally slower than compiled Java bytecode? Of course it is (I
assumed this was common knowledge (to the point of being a cliche
even) but :\). If anyone told you otherwise, I'm sorry you were
blatantly lied to. BUT Ruby lets us *produce* faster and more
accurately, giving us plenty of spare time to optimize the code (even
porting specific parts to C if needed) after we've easily made it
correct.

> 2) Counting versus Using the Tokens
>
> In the modified code, it is now just counting the tokens:
>
> * * num += l.split.length
>
> Obviously that is faster than what I had in the original code. Again
> however, I'm doing this on purpose.
>
> Counting the number of tokens in an of itself is not all that I was
> doing in the original code or in the Java version. To simulate more
> closely what actually occurs in a functional system I am:
>
> - assigning the array of tokens to a variable
> - iterating the tokens to do something with each of them
>
> In this case I'm just assigning each token to another variable and then
> performing the count.
>
> In a real world use I'd perform some function on the text, put it
> somewhere, whatever.


In the real world, the "do something with each of them" is the real
juicy part that we want to compare. What is the something? Does the
real world program just end up counting tokens? Then we realize this,
count tokens, and be on our merry way. Is the real world program
taking each word in a text file, comparing relationships against a
lexical database, then based off whatever relationships in context and
calculations constructing a sort of hash to classify a given text
document? String token = tk.nextToken(); numTokens++ does not begin to
describe or "simulate" this, so what is the point of the benchmark?

> This change accounts for the difference in time from "7965.289 ms" to
> "4821.399 ms" when I run the original code and the modified code.
>
> So yes, the modified code is "faster", but it's not doing the same thing
> as the original and therefore not a valid comparison.


The input is the same. The output is the same. The person running your
code does not care if its object-oriented, procedural, a script, is
functional, etc; he only cares if he gets the expected output in a
reasonable amount of time when he gives his input. Thus the coder only
cares if she can code fast enough to give the client the features he
wants, and if she can does this in a way thats easy to keep up with
his increasing feature demands while keeping the code stable and
fast.





But I dunno, maybe I'm still completely missing the point?
 
Reply With Quote
 
 
 
 
pharrington
Guest
Posts: n/a
 
      08-20-2009
On Aug 20, 1:16*am, Ben Christensen <(E-Mail Removed)> wrote:
> @Matthew K. Williams
>
> -- 1. How *often* are you going to be processing these files? *If they
> are
> -- batch style jobs, then does absolute speed matter over
> maintainability?
>
> The particular application I'm looking at in the future has a virtually
> continuous feed of incoming data from multiple concurrent sources.
>
> Thus I'm looking at what language the processing code would be in. My
> default go to is Java - but I want to consider Ruby and not blindly just
> use what I'm accustomed to before establishing what will likely be in
> existence for the next 3-5 years.


OK, speed and excellent concurrency handling (non-JRuby Ruby strikes
out across the board on the second aspect unfortunately) really is
your priority then. May I then ask what made you consider Ruby in the
first place? Something along the lines of Scala (runs on the JVM,
*relatively* mature, *fantastic* concurrency support, works well with
functional and imperative/object oriented styles) or Erlang (SMP
almost for free, but *almost* strictly immutable and functional coding
(which is much more of a learning curve than a negative)) would make a
ton more sense here.
 
Reply With Quote
 
Josh Cheek
Guest
Posts: n/a
 
      08-20-2009
[Note: parts of this message were removed to make it a legal post.]

>
> puts "The number of tokens is: %d." % File.open(ARGV[0]){|f|
> f.inject(0){|a,l| a+l.split.length } } ,
> "It took #{(Time.now - start) * 1000} ms"
> I find it intellectually stimulating and admirable at the power of what
> is accomplished in such a short statement.
>
> Understanding it however takes time and thought - and a certain level of
> skill.
>
> Perhaps your experiences are different - but most development teams have
> a lot of more junior and intermediate developers than senior - and the
> more verbose, easy to read, simple to understand code is far more
> preferable - even for myself when I must review, debug and profile the
> code of dozens of others - as opposed to something that looks like an
> academic puzzle to unravel.
>


Hello, Ben,
I apologize if my solution turns you off to Ruby (because of the joy, and
excitement you'll miss Ruby is supportive of many paradigms, and so I
adapt my code to my preference. I also write my Java like this, as much as
the language allows. Certainly you can write code which is much more
straightforward than mine. Due to the way I read my code later, I like to
get as much accomplished in as little room as possible (I'm perfectly happy
to let it run off the end of the screen), and then supply a comment telling
me what it does, and if it is esoteric, explaining how it does this. This
allows me very quickly quantify chunks of code, and narrow my attention to
only the relevant portions. If those portions are complex, my comment helps
me quickly figure it out. If your junior developers read code differently,
then perhaps a less terse style would be more fitting. Ruby also adapts
itself very well to legibility, if you choose to write it that way (in
Rails, people often remark that the code documents itself).

Choosing the right tool for the job is a relevant cliche here, and it sounds
like your job, being so performance oriented, requires a tool well suited to
meet these performance needs. If that need is so pressing that you've had to
replace Java with C, then Ruby is probably not the tool you need. You can do
things in Ruby that will make your head spin (these simple examples do not
even hint at the power Ruby grants you), but that doesn't make it the right
tool for this job. If Java is better suited to your needs for this project,
then that would certainly be the responsible choice.

However, I'd encourage you to keep evaluating the language, even if it is
not the best choice for this particular project, because it can take a
little time to figure out 'the Ruby way'. And as Pharrington pointed out
"Different languages express solutions to different problems in
different ways". I think that once you play with it to the point of comfort,
then you should see how Ruby addresses various patterns, (consider
http://www.amazon.com/Design-Pattern.../dp/0321490452),
you may begin to feel that same love some of us here do. And then, I suspect
that instead of seeing how well Ruby can pretend to be Java, you'll find
yourself wondering if Java can't be a bit more like Ruby. Perhaps at that
time, a solution like JRuby would look very desirable. Also, great strides
are being made in regards to speed, which would significantly alleviate the
most relevant objection.

Anyway, regardless of your choice, thank you for taking the time to educate
yourself about Ruby.

 
Reply With Quote
 
David Masover
Guest
Posts: n/a
 
      08-20-2009
First, I should say that I'm going to present arguments for Ruby here, whether
or not I think it's the best choice. Without knowing what you need, I really
can't say.

On Thursday 20 August 2009 12:25:58 am Ben Christensen wrote:
> Yes, "CPU is cheap", but that applies to Java as well.


But if you are in a position to be able to throw more hardware at the problem,
it does really become a question of CPU time vs programmer time. That is, if
Ruby really does cost 3x the CPU of Java, you can calculate in real dollars
how much it will cost to use.

Speaking for myself, I certainly feel Ruby makes me more than three times as
productive as Java, and programmer time is much more expensive than CPU time.

Obviously, there are cases where spending programmer time makes sense. For
example, performance-critical desktop apps (CAD, games, etc) cannot use the
"throw more CPU at it" argument, because you're now forcing your clients to
upgrade their hardware to use your product. A large enough organization might
want to optimize as much as they can -- if rewriting it in C saves a thousand
machines and takes a developer a year, it's probably worth it, unless you can
get a thousand machines cheaper than a developer.

On the other hand, there's a case to be made that you should "write one to
throw away" -- if you can do it in Ruby in a few days, and rewrite it in Java
in a few weeks, the rewrite can take lessons learned from that ruby sketch.

It would also give you time to evaluate the "fast enough" argument. If it
turns out that you have plenty of extra capacity, and the Ruby version runs
fast enough, it may not be worth rewriting. If it turns out that Ruby is too
slow (even after trying Ruby 1.9 and JRuby), you've only lost a few days.

> Perhaps in a straight-forward CRUD app where all that's being done is
> retrieving/storing data in a database where IO truly is the bottleneck
> that no amount of optimization can improve - then it doesn't matter and
> "good enough" rings true where IO takes 100ms and the Ruby/Java portion
> is only 10-20ms on top of the IO.


That depends...

Response time is only part of the story. What you really want to benchmark is
requests per second, and that's not always as simple as multiplying response
time.

> ## C Extensions to Ruby ##
>
> This feels akin to saying in the late 90s that to make Java perform
> well, use JNI to use C. To me it defeats the whole purpose of the "Ruby
> is simple and pleasant" paradigm. If I have to optimize it with C, then
> it's no longer simple or a joy to use.


It's a bit like C -- it's going to be fast enough most of the time, but
there's always the possibility you'll find some small part that can be
rewritten in assembly to squeeze some extra performance out of it.

Most of what I said could be summarized as: The speed of a nonworking program
is irrelevant, and premature optimization is the root of all evil. (I don't
remember who I'm quoting, but it was someone important.)

My preference would be, if I can write 97% of the program in Ruby, and 3% in
C, is that really going to be less pleasant than writing 100% of the program
in Java?

> Nor am I convinced yet that managing a codebase over 5-10 years and 40+
> developers is any easier with Ruby - Java's static typing and now its
> generics (the polar oppostite approach of 'duck typing' in Ruby) are
> actually very nice for readability, navigation of code, refactoring and
> other such things on such large codebases when so much of it is from
> other coders, teams, or just plain old and forgotten about.


I suspect most of that is due to the tools, more than the type system itself.

The most compelling argument I've heard is: Type checks are just a special
case of unit tests. You need unit tests anyway, and good unit tests will
already more than cover what type checks were meant to save you from.

I suppose I'm curious -- when was the last time the type system saved you?
That is, when was the last time you tried to pass an object of the wrong type
to a method, and gotten a type error of some sort, and realized you needed to
do something other than a simple typecast on that object?

> Also, no amount of "throwing hardware at it" will make Ruby faster than
> throwing the same hardware at Java - which is ultimately I think the
> biggest issue I have with the concept.


Indeed -- but again, you're paying for it with increased developer time.

And throwing the same hardware at Java that you would need for Ruby just gives
you a bunch of unused capacity -- you'd be buying less hardware. So it is
pretty much a straight trade.

> If I throw the fastest hardware I
> can at something, I want my user to get the biggest bang for the buck -


Which is pretty much going to give you benchmark candy. If your site is
slowing down, that's a bug. Once the speed of the site is acceptable, and
you're set up to handle spikes appropriately, more performance doesn't really
buy you anything other than "because you can".

> As for "brevity" equalling "maintainability" and "less bugs" - I tend to
> disagree when the pursuit results in code such as this example given:
>
> puts "The number of tokens is: %d." % File.open(ARGV[0]){|f|
> f.inject(0){|a,l| a+l.split.length } } ,
> "It took #{(Time.now - start) * 1000} ms"


Probably true for that -- though, to be fair, I wouldn't have put it that way.

However, there was a study done, at one point, which showed that the ratio of
bugs to lines of code was constant across languages. So while I wouldn't say
it makes sense to make things unreadably brief, Ruby is usually both more
readable and shorter. 100 lines of code is generally easier to read and debug
than a thousand.

> Perhaps your experiences are different - but most development teams have
> a lot of more junior and intermediate developers than senior - and the
> more verbose, easy to read, simple to understand code is far more
> preferable - even for myself when I must review, debug and profile the
> code of dozens of others - as opposed to something that looks like an
> academic puzzle to unravel.


I think that particular code sample was misleading -- certainly, you can play
Perl Golf in any language. But you have coding conventions in Java, and you
would in Ruby.

> What types of applications and codebases do you feel truly are served
> best by Ruby - and therefore not in need of the highest performance
> given to the end-user?


I feel the kind that is served best is any sort of web app, or small scripts
for system administration -- particularly anything that needs to be flexible
and constantly maintained and improved, for which the developer controls the
hardware.

Were there better deployment tools (and Shoes seems to be an effort in that
direction), I'd also say any sort of desktop app that doesn't need the highest
performance possible. Frankly, that's most of them -- an instant messaging
client, for instance, doesn't need to be blazingly fast, just fast enough.

But, it's not always about whether the end-user needs the highest performance.
It's about whether it's possible to throw CPUs at the problem, or whether the
CPU is the bottleneck at all.

 
Reply With Quote
 
Brian Candler
Guest
Posts: n/a
 
      08-20-2009
Ben Christensen wrote:
> ## Fast Enough ##
>
> I often hear that Ruby is "fast enough" or that the performance
> difference is not important since IO is generally the real source of
> performance issues.
>
> I understand that in certain cases - though when I see potential
> performance degradation as multiples (2x, 3x, etc) as opposed to small
> percentages, it makes me question the decision to use Ruby much more
> than if we were talking percentages of 10-20% - such as 115ms vs 100ms
> on a page response.
>
> For example, a java environment my team has built provides SOAP/REST
> webservices for product catalog search - and responsiveness is a very
> significant measurement criteria of the success of the product and
> system. Therefore, our average server side response time is something we
> watch very closely.


And in that case, your response time may be dominated by the time to
process the SOAP request and format the SOAP response, so evaluating the
performance of those libraries is important.

I still don't buy the "fastest to execute must be best" reasoning. In
reality, there will be a threshold of acceptability - e.g. 90% of
requests must be returned within 150ms - in which case you're free to
choose a platform which meets that goal, and/or apply money to the
hardware platform as required.

However, if your competitor using Ruby gets a product to market in one
third of the time, then it doesn't matter if your Java solution performs
50% better - you won't have any customers.

> ## Maintainability, Speed of Development and 'Enjoyment' ##
>
> I hear "speed of development", "maintainability", and "enjoyment of
> coding" as the reasons to move to Ruby - and to accept the negatives in
> performance, tools, libraries etc.
>
> I'm still not sold on these reasons - as I truly enjoy working in the
> Java ecosystem of tools, projects, libraries etc


Then you have no need to ask anything more here. You're sold on Java,
you're productive with Java, you enjoy Java, so use Java.

But computing is not static. There are probably still people who use
Algol and Fortran daily, and they are Turing-complete of course, but
newer languages make programming easier. You found the same when you
moved from C to Java, so you can see the benefits of keeping abreast of
new developments. It's always good to stretch yourself out of your
comfort zone to experience how people are using different languages
effectively. Perhaps you should try something more radically different
for a C/Java programmer, like Erlang.

> Nor am I convinced yet that managing a codebase over 5-10 years and 40+
> developers is any easier with Ruby - Java's static typing and now its
> generics (the polar oppostite approach of 'duck typing' in Ruby) are
> actually very nice for readability, navigation of code, refactoring and
> other such things on such large codebases


Again, if you find this a benefit, then go with Java. Most people here
find the opposite, but then, you're on a ruby users' mailing list so
what do you expect

That is: most of us hugely value Ruby's speed of development (like
writing the same thing in 1/10th of the number of lines of code), and if
there's a reduced run-time penalty, that's a minor issue.

You won't really get a taste for what we mean until you start writing
Ruby (real Ruby, not Java ported line-by-line to Ruby). Perhaps you
could start with using jruby to wire up your Java objects. You'll get
the raw performance of the underlying Java objects, but using jruby as
the integration glue.

> As for "brevity" equalling "maintainability" and "less bugs" - I tend to
> disagree when the pursuit results in code such as this example given:
>
> puts "The number of tokens is: %d." % File.open(ARGV[0]){|f|
> f.inject(0){|a,l| a+l.split.length } } ,
> "It took #{(Time.now - start) * 1000} ms"
>
> I find it intellectually stimulating and admirable at the power of what
> is accomplished in such a short statement.
>
> Understanding it however takes time and thought - and a certain level of
> skill.


I agree with you, this is an unnecessary use of inject, and people who
push this sort of code at newcomers are not doing any favours. I would
write this simply as:

tokens = 0
File.foreach(ARGV[0]) do |line|
tokens += line.split.length
end
puts "The number of tokens is: #{tokens}"

Regards,

Brian.
--
Posted via http://www.ruby-forum.com/.

 
Reply With Quote
 
David Masover
Guest
Posts: n/a
 
      08-20-2009
On Wednesday 19 August 2009 01:45:22 pm Josh Cheek wrote:
> My previous version would probably be better like this:
>
> start = Time.now
> puts "Starting to read file ..."
> puts "The number of tokens is: %d." % File.open(ARGV[0]){|f|
> f.inject(0){|a,l| a+l.split.length } } ,
> "It took #{(Time.now - start) * 1000} ms"


Just for fun, here's a verbose, somewhat less magical version:

start = Time.now
filename = ARGV.first

puts 'Starting to read file...'

count = 0
File.open filename do |file|
file.each_line do |line|
count += line.split.length
end
end

puts "The number of tokens is: #{count}."
duration = Time.now - start
puts "It took #{duration*1000} ms"


That is intended to be somewhat self-documenting, so a bit more verbose than I
might normally do. It does more or less the same thing, in more or less the
same way. It also seems to be following roughly the pattern you did in Java,
and I find it _much_ more readable.

Of course, it's a short (benchmark) example, so it's difficult to show Ruby
really shining here, unless you want to play golf. But even the readable
version is also far less verbose than the equivalent Java.

 
Reply With Quote
 
Rick DeNatale
Guest
Posts: n/a
 
      08-20-2009
On Wed, Aug 19, 2009 at 10:41 AM, <(E-Mail Removed)> wrote:
> On Wed, Aug 19, 2009 at 9:31 AM, Ben Christensen
> <(E-Mail Removed)> wrote:
>> I'm evaluating Ruby for use in a variety of systems that are planned by
>> default to be Java.
>>
>> I've started down a path of doing various performance tests to see what
>> kind of impact will occur by using Ruby and in my first test the numbers
>> are very poor - so poor that I have to question if I'm doing something
>> wrong.

>
> Is this test case in any way representative of the tasks you will
> actually be performing?
>
> Test file 1:
>
>> java FileReadParse

> Starting to read file...
> The number of tokens is: 1954
> It took 16 ms
>
>> ruby -v file_read_parse.rb

> ruby 1.8.6 (2007-09-24 patchlevel 111) [i386-linux]
> Starting to read file ...
> The number of tokens is: 1954
> It took 4.951 ms
>
> Test file 2:
>
>> java FileReadParse

> Starting to read file...
> The number of tokens is: 479623
> It took 337 ms
>
>> ruby file_read_parse.rb

> Starting to read file ...
> The number of tokens is: 479623
> It took 2526.455 ms
>
>> ruby file_read_parse-2.rb

> Starting to read file ...
> It took 588.065 ms
> The number of tokens is: 479623


One of the things this 'benchmark' skips over is the time it takes to
initialize the two environments.

There's no measurement of the time between hitting enter on the
command line and the point where the

start = Time.now

line (or it's equivalent in the Java program) gets executed.

It might be interesting to execute those benchmarks using something
like the linux time command to measure the differences in "womb to
tomb" execution times.

I suspect, but I may be totally wrong, that Java takes a while to
'warm up' a more complex runtime environment, and that Ruby gets going
faster. The JVM has evolved in an environment where it has tended to
be used for long-running processes.

Depending on the task this may be important.

This can also affect deployment architecture. JRuby tends to
encourage multi-threading in a single process to amortize the start-up
time. Most of us who are using MRI for, say rails, are pretty happy
with having multiple server processes which can be brought up as
needed (and terminated when not) under something like Passenger
(a.k.a. mod ruby) particularly using the "Enterprise" version of Ruby
which allows for sharing vm code between the processes.


--
Rick DeNatale

Blog: http://talklikeaduck.denhaven2.com/
Twitter: http://twitter.com/RickDeNatale
WWR: http://www.workingwithrails.com/pers...-rick-denatale
LinkedIn: http://www.linkedin.com/in/rickdenatale

 
Reply With Quote
 
Ben Christensen
Guest
Posts: n/a
 
      08-20-2009
Thank you very much for the excellent answers, and your well reasoned
responses to what could have easily been dismissed as someone "not
getting it" or attempting to start a flame war.

I have quoted various aspects of the responses and added my response or
further comments.



-- May I then ask what made you consider Ruby in the first place?

The reason I'm considering it is because I don't want to blindly choose
Java just because it's the default.

As for why Ruby and not Erlang, Scala, Groovy etc -- the honest answer
is because Ruby is getting so much attention these days, to the point of
religious fervor amongst many I speak to that I need to take an
objective look at it and what it does well.

I greatly dislike religious wars amongst technology - for example Linux,
Mac and Windows - and thus want to understand the objective
benefits/drawbacks as opposed to personal taste.



-- "seeing how well Ruby can pretend to be Java"

My intention is not to see how Ruby can pretend to be Java.

I'm using Java as the point of comparison for a few reasons:

- it's what I have the most expertise in
- it's generally the "default" choice in the types of projects and
development teams I lead
- the language has a very wide range of understanding in the development
world and is therefore a good point of reference to discuss from
- I need a valid point of reference for objective performance
comparisons

That being said, I am trying to figure out what the "Ruby way" is -
which so far is far from clear to me.

I appreciate the reference to the Design Patterns in Ruby book. That is
very much the type of recommendation that will probably help me out, so
thank you.

http://www.amazon.com/Design-Pattern.../dp/0321490452



-- You won't really get a taste for what we mean until you start writing
-- Ruby (real Ruby, not Java ported line-by-line to Ruby).

What example opensource projects can you refer me to which espouse the
"real Ruby" style of doing things?

I'd prefer non-Rails projects, as I understand the completely different
approach of webapp dev with Rails.

I'm looking specifically at Ruby.

I keep getting told that I must understand the "Ruby way" - so I'd
appreciate instruction on how to accomplish the "Ruby way" considering I
am apparently boxed in as a "Java/C style programmer" ... despite
disliking C




-- But if you are in a position to be able to throw more hardware at the
problem,
-- it does really become a question of CPU time vs programmer time. That
-- is, if Ruby really does cost 3x the CPU of Java, you can calculate in
real
-- dollars how much it will cost to use.

If scalability was the only issue, this would be a valid response.

For example, if both Java and Ruby both performed single threaded
transactions at 150ms each, and both scaled to 10 concurrent threads
equally well, but Java continues to scale to 30 concurrent threads and
Ruby does not, then that's a scenario where I can add 3 machines to
scale Ruby horizontally and truly argue that the cost of the hardware is
more than made up for by lower developer costs.

But, "per request" performance does not get improved by this type of
solution.

Adding faster hardware does not make Ruby catch up to Java - since Java
also improves with faster hardware.

This is why the "add hardware" answer doesn't win me over on the
performance issue, because performance and scalability are two
completely different problems. I haven't even begun to test scalability
with Ruby yet.



-- Response time is only part of the story. What you really want to
benchmark is
-- requests per second, and that's not always as simple as multiplying
response time.

That's correct ... but supports my point. Requests per second is the
throughput, or scalability - not performance.

That is something I can throw hardware at - performance is not.


-- Which is pretty much going to give you benchmark candy. If your site
is
-- slowing down, that's a bug. Once the speed of the site is acceptable,
-- and you're set up to handle spikes appropriately, more performance
doesn't
-- really buy you anything other than "because you can".

I disagree. If I can cut 30% of the transaction time off of a search
engine request - that is valuable.

It provides a better use experience to the user and (according to Google
and Amazon) increases their usage of the system.

Performance of response (not talking about scalability here but actual
performance) is more than just "bragging rights" or "benchmark candy".
The speed at which an application responds to an end users request
impacts the overall usability of an application.

It is for this same reason that things such as network compression,
network optimization (CDNs, Akamai route acceleration etc) and client
side caching also all play a role.

In the presentation layer however, I tend to think the performance
degradation of using Ruby is far less of an issue than backend services,
since IO does play such a huge role - which is more or less what
Thoughtworks has come to conclude from their use of it based on their
reported experiences.



-- premature optimization is the root of all evil

I 100% agree. Martin Fowler comes to mind or someone similar.



-- My preference would be, if I can write 97% of the program in Ruby,
and
-- 3% in C, is that really going to be less pleasant than writing 100%
of the
-- program in Java?

An interesting observation and one I must consider.



-- when was the last time the type system saved you?

This is a valid and interesting question.

I would suggest that it's not that it is "saving" anything - cause there
is nothing to save once the application is running, because the code
can't be compiled if things aren't type-safe.

It's the toolset as you stated that you suspect.

The readability of code to know exactly what types a given argument,
variable or array contain.

The IDE telling me as I type when errors are occuring, what objects
relate to what, navigating through code, etc.

I've attempted RubyMine, Aptana and Netbeans. They are attempting this
dynamic interpretation but are far from accomplishing it.

For example, code completion in these tools to suggest the available API
methods is almost useless, as they offer virtually every method
available under the sun, as they are not interpreting what actual type
the variable is. Therefore they'll show me 15 different versions of a
method with the same name, all for different object types from the Ruby
API.

Similarly, looking at an array or collection in Ruby does not tell me
what it is, especially if things are being passed around through
methods, across class boundaries etc. Instead of the method signature
telling me "Collection<ZebraAnimal>" I just see a variable.

Thus, I must now depend on a team of developers properly documenting
everything, using very descriptive naming conventions (and properly
refactoring all of that when changes occur), and wrapping everything in
unit tests.

Now, all of those are "ideal" cases - ones I believe in and stress
continually. I have hundreds and hundreds of unit tests and automated
build servers etc - but in the "real world", getting teams to comment
every method, properly name (and refactor) variable names and cover
everything in unit tests just doesn't happen - unless it's a small team
of very competent people who all believe in the same paradigm and treat
their code as art. I wish that's how all dev teams were, but it's not a
reality.

Perhaps if it's a personal project where I know the code and can ensure
all is covered it's a different story.



-- 100 lines of code is generally easier to read and
-- debug than a thousand.

I'll give you that - but I have yet to see anything that proves to me
that a competent developer using both Ruby and Java (or C# for that
matter) would have 10x as much written code than they would in Ruby.

The "cruft" so often referred to are things that I don't even consider
or think of. Boilerplate code ... clutter and sometimes annoying ...
fades into the background and tools remove the pain of it. And with the
advent of annotations, many of these arguments disappear when Java code
is written correctly with modern patterns and frameworks.



-- I think that particular code sample was misleading -- certainly, you
can
-- play Perl Golf in any language. But you have coding conventions in
Java, and
-- you would in Ruby.

You surely can, and I'm trying to understand what the coding conventions
are in Ruby. The book link offered is something I'm going to go look at.
Amazon referred another book called "The Ruby Way" which may also
provide me good insights. Any experience with that one?




-- ... best served by Ruby ...
-- small scripts for system administration

I completely agree here.

-- any sort of web app ... that needs to be flexible and constantly
maintained and improved,
-- for which the developer controls the hardware.

I'm leaning more and more towards this. In fact, I'm trying to figure
out how to rip Java out of my webapps completely and leave that to the
backend webservices and let the presentation layer be as free from
"code" as possible. Java developers typically aren't exactly the best at
client facing solutions (don't attack me on this if you disagree ...
this is ofcourse not a definitive rule, it's just that I find it more
challenging to hire good web developers who are 'Java' skilled as
opposed to PHP, Ruby, Javascript, CSS, etc).

For example, if I can accomplish a dynamic front-end purely driven by
client side Javascript using AJAX techniques with a REST style
webservices backend, I will try to pursue that.

The middle ground seems to be pursuing Ruby or something else that is
still server-side, but better suited to the always changing pace of
webapp dev and more creative, script driven coding style better suited
to web developers and designers.

-- desktop app that doesn't need the highest performance possible

What you say makes sense here, but I am so far removed from desktop apps
that I'm useless in weighing in on this.



-- It's about whether it's possible to throw CPUs at the problem, or
-- whether the CPU is the bottleneck at all.

Understood and I agree.




--
Posted via http://www.ruby-forum.com/.

 
Reply With Quote
 
brabuhr@gmail.com
Guest
Posts: n/a
 
      08-20-2009
On Thu, Aug 20, 2009 at 11:22 AM, Ben
Christensen<(E-Mail Removed)> wrote:
> -- My preference would be, if I can write 97% of the program in Ruby,
> and
> -- 3% in C, is that really going to be less pleasant than writing 100%
> of the
> -- program in Java?
>
> An interesting observation and one I must consider.


Or, perhaps in your case: 9x% in Ruby, y% in Java.


Example 1:

require 'java'
java_import 'FileReadParse'

FileReadParse.new.do_stuff


Example 2:

require 'java'

java_import 'java.util.StringTokenizer'

File.open("/tmp/file_test.txt") do |file|
file.each_line do |line|
tokens = StringTokenizer.new(line)
tokens.each do |token|
#do_stuff_with(token)
end
end
end

(Though in the token counting case, example 2 is slower than pure
ruby: "tokens = StringTokenizer.new(line)" takes more time than
"tokens = line.split".)


Example 3:

require 'java'

java_import 'TokenProcessor'

token_processor = TokenProcessor.new

File.open("/tmp/file_test.txt") do |file|
file.each_line do |line|
line.split.each do |token|
token_processor.process(token)
end
end
end


There could also be room in your toolbox for Ruby to help in the
testing of your Java code:
http://jtestr.codehaus.org/
http://wiki.github.com/aslakhellesoy...jruby-and-java

 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
C Test Incorrectly Uses printf() - Please Confirm Martin O'Brien C Programming 147 08-21-2010 05:31 AM
M$ against Blu-ray, M$ for Blu-ray, M$ against Blu-ray, M$ forBlu-ray, ...... Blig Merk DVD Video 66 04-27-2008 04:46 AM
Performance of datatable.select() func against database query orthe xquery Mukesh ASP .Net 2 06-14-2007 07:03 AM
poor performance against Oracle DB on specific machine yoram.ayalon@structuredweb.com ASP .Net 2 02-26-2007 12:17 AM
test test test test test test test Computer Support 2 07-02-2003 06:02 PM



Advertisments