Confirm my Performance Test Against Java?
 

I'm evaluating Ruby for use in a variety of systems that are planned by
default to be Java.

I've started down a path of doing various performance tests to see what
kind of impact will occur by using Ruby and in my first test the numbers
are very poor - so poor that I have to question if I'm doing something
wrong.

I've tried it on both Linux and Mac OSX and get similar performance
numbers on each - differences being hardware, but the ratio between the
results about the same.

Please take a look at my blog post on my test results and view the
source code and let me know if I'm doing something completely wrong with
the Ruby code or execution - or if these are accurate numbers.

http://benjchristensen.com/2009/08/1...y-performance/

NOTE: This is not an attempt to start a flame war. This is a legitimate
effort to take a good look at Ruby and let the numbers speak for
themselves in making decisions for what types of applications I can
choose to use Ruby for without sacrificing the performance of a mature
platform such as Java.

Thank you.

Ben
--
Posted via http://www.ruby-forum.com/.

Re: Confirm my Performance Test Against Java?
 

On Aug 19, 9:31*am, Ben Christensen <benjchristen...@gmail.com> wrote:
> I'm evaluating Ruby for use in a variety of systems that are planned by
> default to be Java.
>
> I've started down a path of doing various performance tests to see what
> kind of impact will occur by using Ruby and in my first test the numbers
> are very poor - so poor that I have to question if I'm doing something
> wrong.
>
> I've tried it on both Linux and Mac OSX and get similar performance
> numbers on each - differences being hardware, but the ratio between the
> results about the same.
>
> Please take a look at my blog post on my test results and view the
> source code and let me know if I'm doing something completely wrong with
> the Ruby code or execution - or if these are accurate numbers.
>
> http://benjchristensen.com/2009/08/1...ns-on-ruby-per...
>
> NOTE: This is not an attempt to start a flame war. This is a legitimate
> effort to take a good look at Ruby and let the numbers speak for
> themselves in making decisions for what types of applications I can
> choose to use Ruby for without sacrificing the performance of a mature
> platform such as Java.
>
> Thank you.
>
> Ben
> --
> Posted viahttp://www.ruby-forum.com/.

Well.... without having put a ton of thought into this... yes, Ruby
(*especially* 1.8 MRI) is slow. No one's going to argue that the Ruby
interpreter is one of the quicker kids around. If performance is the
#1 priority of whatever you'll be developing, Ruby doesn't fit your
needs, and no one will tell you it does. That's what Java (for the
most part) and C are still hanging around for.

What sort of software is in needed of being developed here?

Ask yourself: is it critical that my code always performs as fast as
possible? Or is the greater concern speed of development and project
maintainability?

Also as to the benchmark... can you post your /tmp/file_test.txt?
Posting some benchmarky code isn't very useful if no one can replicate
your results. Reading the whole file into memory may be faster than
reading it line-by-line (but obviously the wrong thing to do if the
file's enormous, which.... 8 secs to read??? i'd better be moved to
tears by the size it.) And not entirely sure what it is you're trying
to benchmark here? Vagggguuee benchmarks are fairly useless, as the
code your timing is never going to be anywhere close to the actual
code you'll write. Are you trying to just compare file reading times?
Benchmark that, and only that. Is there something specific string
manipulation-wise you want to measure? Then... measure that. Until
your code starts getting at least halfway specific, just doing a line-
by-line Java-Ruby conversion doesn't tell anything, as the code that
happens is neither the most "elegant" *nor* fastest Ruby can do.

Re: Confirm my Performance Test Against Java?
 

On Wed, Aug 19, 2009 at 9:31 AM, Ben Christensen
<benjchristensen@gmail.com> wrote:
> I'm evaluating Ruby for use in a variety of systems that are planned by
> default to be Java.
>
> I've started down a path of doing various performance tests to see what
> kind of impact will occur by using Ruby and in my first test the numbers
> are very poor - so poor that I have to question if I'm doing something
> wrong.

Is this test case in any way representative of the tasks you will
actually be performing?

Test file 1:

> uname -a
Linux linux116.ctc.com 2.6.18-92.1.22.el5 #1 SMP Tue Dec 16 12:03:43
EST 2008 i686 i686 i386 GNU/Linux

> java -version
java version "1.6.0_0"
IcedTea6 1.3.1 (6b12-Fedora-EPEL-5) Runtime Environment (build 1.6.0_0-b12)
OpenJDK Server VM (build 1.6.0_0-b12, mixed mode)

> java FileReadParse
Starting to read file...
The number of tokens is: 1954
It took 16 ms

> ruby -v file_read_parse.rb
ruby 1.8.6 (2007-09-24 patchlevel 111) [i386-linux]
Starting to read file ...
The number of tokens is: 1954
It took 4.951 ms

Test file 2:

> java FileReadParse
Starting to read file...
The number of tokens is: 479623
It took 337 ms

> ruby file_read_parse.rb
Starting to read file ...
The number of tokens is: 479623
It took 2526.455 ms

> ruby file_read_parse-2.rb
Starting to read file ...
It took 588.065 ms
The number of tokens is: 479623

> cat file_read_parse-2.rb
puts "Starting to read file ..."
start = Time.now

tokens = File.new("/tmp/file_test.txt").read.scan(/[^\s]+/)
count = tokens.size

stop = Time.now
puts "It took #{(stop - start) * 1000} ms"
puts "The number of tokens is: #{count}"

Re: Confirm my Performance Test Against Java?
 

brabuhr@gmail.com wrote:
> On Wed, Aug 19, 2009 at 9:31 AM, Ben Christensen
> <benjchristensen@gmail.com> wrote:
>> I'm evaluating Ruby for use in a variety of systems that are planned by
>> default to be Java.
>>
>> I've started down a path of doing various performance tests to see what
>> kind of impact will occur by using Ruby and in my first test the numbers
>> are very poor - so poor that I have to question if I'm doing something
>> wrong.
>
> Is this test case in any way representative of the tasks you will
> actually be performing?

If it is, then you should just do
$ time wc approach.txt
6836 78325 484114 approach.txt

real 0m0.041s
user 0m0.046s
sys 0m0.015s

Re: Confirm my Performance Test Against Java?
 

[Note: parts of this message were removed to make it a legal post.]

On Wed, Aug 19, 2009 at 9:31 AM, Ben Christensen
<benjchristensen@gmail.com>wrote:

> I'm evaluating Ruby for use in a variety of systems that are planned by
> default to be Java.
>
> I've started down a path of doing various performance tests to see what
> kind of impact will occur by using Ruby and in my first test the numbers
> are very poor - so poor that I have to question if I'm doing something
> wrong.
>
> I've tried it on both Linux and Mac OSX and get similar performance
> numbers on each - differences being hardware, but the ratio between the
> results about the same.
>
> Please take a look at my blog post on my test results and view the
> source code and let me know if I'm doing something completely wrong with
> the Ruby code or execution - or if these are accurate numbers.
>
>
> http://benjchristensen.com/2009/08/1...y-performance/
>
> NOTE: This is not an attempt to start a flame war. This is a legitimate
> effort to take a good look at Ruby and let the numbers speak for
> themselves in making decisions for what types of applications I can
> choose to use Ruby for without sacrificing the performance of a mature
> platform such as Java.
>

Hi Ben,

The point everyone keeps bringing up--whether this benchmark is indicative
of what you will actually be doing with Ruby, and whether it is "fast
enough"--is worth considering for any project, but the fact remains that for
many things, Java is going to execute faster than Ruby. You can certainly
optimize Ruby code (and yes, writing Ruby extensions in C is actually pretty
easy), but that's not why many of us love Ruby. We love it because it allows
you to turn FileReadParse.java into this: http://gist.github.com/170466.
Now, in the spirit of good fun:

$ ruby file_read_parse_2.rb file_read_parse_2.rb
Starting to read file ...
The number of tokens is: 39.
It took 0.189 ms

$ ruby file_read_parse_2.rb FileReadParse.java
Starting to read file ...
The number of tokens is: 159.
It took 0.215 ms

See? :-)

Good luck with Ruby, and don't be afraid to ask more questions!
Mike


> Thank you.
>
> Ben
> --
> Posted via http://www.ruby-forum.com/.
>
>

Re: Confirm my Performance Test Against Java?
 

[Note: parts of this message were removed to make it a legal post.]

Argh! That gist should be http://gist.github.com/170476. Sigh...

On Wed, Aug 19, 2009 at 1:17 PM, Mike Sassak <msassak@gmail.com> wrote:

> On Wed, Aug 19, 2009 at 9:31 AM, Ben Christensen <
> benjchristensen@gmail.com> wrote:
>
>> I'm evaluating Ruby for use in a variety of systems that are planned by
>> default to be Java.
>>
>> I've started down a path of doing various performance tests to see what
>> kind of impact will occur by using Ruby and in my first test the numbers
>> are very poor - so poor that I have to question if I'm doing something
>> wrong.
>>
>> I've tried it on both Linux and Mac OSX and get similar performance
>> numbers on each - differences being hardware, but the ratio between the
>> results about the same.
>>
>> Please take a look at my blog post on my test results and view the
>> source code and let me know if I'm doing something completely wrong with
>> the Ruby code or execution - or if these are accurate numbers.
>>
>>
>> http://benjchristensen.com/2009/08/1...y-performance/
>>
>> NOTE: This is not an attempt to start a flame war. This is a legitimate
>> effort to take a good look at Ruby and let the numbers speak for
>> themselves in making decisions for what types of applications I can
>> choose to use Ruby for without sacrificing the performance of a mature
>> platform such as Java.
>>
>
> Hi Ben,
>
> The point everyone keeps bringing up--whether this benchmark is indicative
> of what you will actually be doing with Ruby, and whether it is "fast
> enough"--is worth considering for any project, but the fact remains that for
> many things, Java is going to execute faster than Ruby. You can certainly
> optimize Ruby code (and yes, writing Ruby extensions in C is actually pretty
> easy), but that's not why many of us love Ruby. We love it because it allows
> you to turn FileReadParse.java into this: http://gist.github.com/170466.
> Now, in the spirit of good fun:
>
> $ ruby file_read_parse_2.rb file_read_parse_2.rb
> Starting to read file ...
> The number of tokens is: 39.
> It took 0.189 ms
>
> $ ruby file_read_parse_2.rb FileReadParse.java
> Starting to read file ...
> The number of tokens is: 159.
> It took 0.215 ms
>
> See? :-)
>
> Good luck with Ruby, and don't be afraid to ask more questions!
> Mike
>
>
>> Thank you.
>>
>> Ben
>> --
>> Posted via http://www.ruby-forum.com/.
>>
>>
>

Re: Confirm my Performance Test Against Java?
 

Mike Sassak wrote:
> Argh! That gist should be http://gist.github.com/170476. Sigh...

And you can even, with another ounce of ruby-love, rewrite that as:

num = 0
ARGF.each do |l|
num += l.split.length
end

Then it also works with stdin or multiple filenames on the cmdline.

I'll leave it to others to #inject... ;)

--
vjoel : Joel VanderWerf : path berkeley edu : 510 665 3407

Re: Confirm my Performance Test Against Java?
 

On Wed, Aug 19, 2009 at 11:07 AM, Reid Thompson<reid.thompson@ateb.com> wro=
te:
> brabuhr@gmail.com wrote:
>>
>> On Wed, Aug 19, 2009 at 9:31 AM, Ben Christensen
>> <benjchristensen@gmail.com> wrote:
>>>
>>> I'm evaluating Ruby for use in a variety of systems that are planned by
>>> default to be Java.
>>>
>>> I've started down a path of doing various performance tests to see what
>>> kind of impact will occur by using Ruby and in my first test the number=
s
>>> are very poor - so poor that I have to question if I'm doing something
>>> wrong.
>>
>> Is this test case in any way representative of the tasks you will
>> actually be performing?
>
> If it is, then you should just do
> $ time wc approach.txt
> =A06836 =A078325 484114 approach.txt

:-)

I got a little crazy; first the numbers (slower hardware this time):

> uname -a
Linux eXist 2.6.28-14-generic #47-Ubuntu SMP Sat Jul 25 00:28:35 UTC
2009 i686 GNU/Linux

> java -version
java version "1.6.0_0"
OpenJDK Runtime Environment (IcedTea6 1.4.1) (6b14-1.4.1-0ubuntu11)
OpenJDK Client VM (build 14.0-b08, mixed mode, sharing)

> java FileReadParse
Starting to read file...
The number of tokens is: 479623
It took 596 ms

> /opt/matzruby/trunk/bin/ruby -v -rubygems file_read_parse.rb
ruby 1.9.2dev (2009-08-14 trunk 24539) [i686-linux]
Starting to read file ...
The number of tokens is: 479623
It took 1751.92544 ms

> /opt/matzruby/trunk/bin/ruby -v -rubygems file_read_parse-3.rb
ruby 1.9.2dev (2009-08-14 trunk 24539) [i686-linux]
ffi_c.so: warning: method redefined; discarding old inspect
struct.rb:26: warning: method redefined; discarding old offset
variadic.rb:15: warning: method redefined; discarding old call
library.rb:78: warning: method redefined; discarding old fopen
library.rb:78: warning: method redefined; discarding old fgetc
Starting to read file ...
It took 4565.077896 ms
The number of tokens is: 479623

> jruby -v -rubygems file_read_parse.rb
jruby 1.3.0 (ruby 1.8.6p287) (2009-06-03 5dc2e22) (OpenJDK Client VM
1.6.0_0) [i386-java]
Starting to read file ...
The number of tokens is: 479623
It took 2316.0 ms

> jruby -v -rubygems file_read_parse-3.rb
jruby 1.3.0 (ruby 1.8.6p287) (2009-06-03 5dc2e22) (OpenJDK Client VM
1.6.0_0) [i386-java]
Starting to read file ...
It took 3117.0 ms
The number of tokens is: 479623

And the code:

> cat file_read_parse-3.rb
require 'ffi'

module LibC
extend FFI::Library

# FILE *fopen(const char *path, const char *mode);
attach_function :fopen, [ :string, :string ], :pointer

# int fgetc(FILE *stream);
attach_function :fgetc, [ :pointer ], :int
end

puts "Starting to read file ..."
start =3D Time.now

file =3D LibC.fopen("/tmp/file_test.txt", "r")
count =3D 0; in_word =3D false
while (c =3D LibC.fgetc(file)) !=3D -1
if 32 < c and c < 127
unless in_word
count +=3D 1
in_word =3D true
end
else
in_word =3D false
end
end

stop =3D Time.now
puts "It took #{(stop - start) * 1000} ms"
puts "The number of tokens is: #{count}"

Re: Confirm my Performance Test Against Java?
 

[Note: parts of this message were removed to make it a legal post.]

On Wed, Aug 19, 2009 at 1:26 PM, Joel VanderWerf <vjoel@path.berkeley.edu>wrote:

> Mike Sassak wrote:
>
>> Argh! That gist should be http://gist.github.com/170476. Sigh...
>>
>
> And you can even, with another ounce of ruby-love, rewrite that as:
>
> num = 0
> ARGF.each do |l|
> num += l.split.length
> end
>
> Then it also works with stdin or multiple filenames on the cmdline.
>
> I'll leave it to others to #inject... ;)
>

Ha! I wrote it with inject initially, but then thought, "Nah... I don't want
to blow *too* many minds." :-)

Re: Confirm my Performance Test Against Java?
 

Thanks everyone for your responses.

Yes, this test is representative of some of the types of applications
and necessary data processing I have current applications doing and am
needing in some future ones.

The file I'm using is 49MB in size unzipped - too large for me to upload
right now as I'm on a mobile cell network.

To provide context on the file, it contains data such as this:

Western Digital Caviar Special Edition Hard Drive - 80GB - 7200rpm -
Ultra ATA - IDE/EIDE - Internal
Kingston 256MB SDRAM Memory Module - 256MB (1 x 256MB) - 133MHz PC133 -
SDRAM - 144-pin
512Mo (1 x 512Mo) - 133MHz PC133 - SDRAM - 168 broches

It's stats are:

wc /tmp/file_test.txt
1778983 7764115 51084191 /tmp/file_test.txt

This is not a test of "file reading". The test is related to the
performance of iterating over large lists of data and performing
processing on them - such as indexing for searching, cleansing,
normalizing etc.

This is a very small representation of the level of complexity and size
of data I would in reality be dealing with.

It seems however that the answer is that this is not what Ruby is well
suited for. Am I correct in that determination?

I will however be continuing my ongoing tests with SOAP/REST webservices
and more CRUD focused webapps, where I expect to see Ruby shine.
--
Posted via http://www.ruby-forum.com/.