«

On Sep 22, 12:23 am, William James <w_a_x_...@yahoo.com> wrote:
> Ruby Maniac wrote:
> > I am new to Ruby and I could use some expert advice as to how I can
> > make this code run faster.
>
> > def scramble(fname)
> > f = File.new(fname, "rb")
> > _fname = fname + ".scrambled"
> > begin
> > File.exist?(_fname) if File.delete(_fname)
> > rescue
> > end
> > ff = File.new(_fname, "wb+")
> > for l in f
> > l.each_byte{|c| ff.write((c | 0x80).chr) }
> > end
> > f.close()
> > ff.close()
> > end
>
> About 2.8 times as fast.
>
> def scramble( fname )
> _fname = fname + ".scrambled"
> mask = 0x8080808080808080
> data = File.open( fname, "rb" ){|f| f.read }
> pad = (8 - (data.size % ) % 8
> data << "p" * pad
> File.open( _fname, "wb"){|ff|
> array = data.unpack('Q*')
> array.map!{|x| x | mask }
> ff.write( array.pack('Q*')[0 ... -pad] )
> }
> end- Hide quoted text -
>
> - Show quoted text -

Okay but wake me up when you can make the original Ruby code I
submitted at the outset of this post run 120 times faster because that
is the difference between Ruby and optimized Python (when Psyco is
used to optimize the Python).

Whether or not Rubynauts feel like Python is perversion or not it is
amazing easy to achieve machine code speeds from Python as opposed to
the difficulty in getting Ruby code to run even 5x faster once
optimized.

From: "Ruby Maniac" <>
>
> Okay but wake me up when you can make the original Ruby code I
> submitted at the outset of this post run 120 times faster because that
> is the difference between Ruby and optimized Python (when Psyco is
> used to optimize the Python).
>
> Whether or not Rubynauts feel like Python is perversion or not it is
> amazing easy to achieve machine code speeds from Python as opposed to
> the difficulty in getting Ruby code to run even 5x faster once
> optimized.

I'm only just now reading about Psyco. But the speed gains
described on the introduction page[1] seem about what one would
expect if similar techniques were used with Ruby:

"For common code, expect at least a 2x speed-up, more typically 4x.
But where Psyco shines is when running algorithmical code ..."

"The difference with the traditional approach to JIT compilers is
that Psyco writes several version of the same blocks (a block is a
bit of a function), which are optimized by being specialized to
some kinds of variables (a "kind" can mean a type, but it is more
general)."

So it could compile specialized versions of (portions of) a method
that would be called if the arguments are Floats or Fixnums.

I would imagine such a technique could work with Ruby, provided
one agrees to disallow various Numeric methods from being
redefined.

For example, Ruby allows things like this:

class Fixnum
alias_method ld_times, :*
def *(other)
warn("i'm thinking: #{self} * #{other}")
old_times(other)
end
end

>> x = 6; y = 7
>> x * y
i'm thinking: 6 * 7
=> 42


I'd suppose if one is willing to give up the ability to modify
a certain set of Fixnum, Float, and perhaps String methods, one
could make the equivalent of Psyco for Ruby.

Indeed, there are a couple of already existing projects in
this general category:

http://rubyforge.org/projects/ruby2c/
http://rubyforge.org/projects/ruby2cext/



[1] http://psyco.sourceforge.net/introduction.html


Regards,

Bill

Ruby Maniac wrote:
> On Sep 22, 12:23 am, William James <w_a_x_...@yahoo.com> wrote:
>
>> Ruby Maniac wrote:
>>
>>> I am new to Ruby and I could use some expert advice as to how I can
>>> make this code run faster.
>>>
>>> def scramble(fname)
>>> f = File.new(fname, "rb")
>>> _fname = fname + ".scrambled"
>>> begin
>>> File.exist?(_fname) if File.delete(_fname)
>>> rescue
>>> end
>>> ff = File.new(_fname, "wb+")
>>> for l in f
>>> l.each_byte{|c| ff.write((c | 0x80).chr) }
>>> end
>>> f.close()
>>> ff.close()
>>> end
>>>
>> About 2.8 times as fast.
>>
>> def scramble( fname )
>> _fname = fname + ".scrambled"
>> mask = 0x8080808080808080
>> data = File.open( fname, "rb" ){|f| f.read }
>> pad = (8 - (data.size % ) % 8
>> data << "p" * pad
>> File.open( _fname, "wb"){|ff|
>> array = data.unpack('Q*')
>> array.map!{|x| x | mask }
>> ff.write( array.pack('Q*')[0 ... -pad] )
>> }
>> end- Hide quoted text -
>>
>> - Show quoted text -
>>
>
> Okay but wake me up when you can make the original Ruby code I
> submitted at the outset of this post run 120 times faster because that
> is the difference between Ruby and optimized Python (when Psyco is
> used to optimize the Python).
>
> Whether or not Rubynauts feel like Python is perversion or not it is
> amazing easy to achieve machine code speeds from Python as opposed to
> the difficulty in getting Ruby code to run even 5x faster once
> optimized.
>
>
>
>
I'm still confused about why someone wouldn't just have a C extension
for a Ruby module instead of going through all this claimed hard work.

We know the facts: a VM solution is better than an interpreted language;
a compiled solution is better than a VM.

Yes, we get it - Python, with the effort you put in, is faster. But,
it's simply a case of matching problems to languages. I don't think
anyone on this group is saying that Ruby is the answer for everything.
There are known issues and limitations to Ruby - some of these will be
resolved and some will not.

As I mentioned before (I think in this thread), Ruby serves its purpose
in many situations. You would not write the firmware of a mobile phone
in Ruby - nor would you in Python - nor would you in Java. You would
most likely write it in C/C++ and where needed, a bit of assembly.

We can discuss this endlessly. Why not write our web applications in
assembly? It is shown that Java takes more resources than C++ and I saw
a paper that showed that on average, C++ /may/ be quite a bit slower
than C and people claim assembly is about 10x faster than C in extreme
cases. So, let's just write everything in assembly language and get a
speedup of 2000 times or some absurd number like that.

Match languages to the problems and the resources. You can, with a C
extension, easily get the performance you talk about. The point is that
for anything complex, it is very likely that Ruby will let you get to
the point where you know where the bottlenecks are much quicker.

Cheers,
Mohit.
9/24/2007 | 11:08 AM.

On Sep 23, 7:55 pm, "Bill Kelly" <bi...@cts.com> wrote:
> From: "Ruby Maniac" <raych...@hotmail.com>
>
>
>
> > Okay but wake me up when you can make the original Ruby code I
> > submitted at the outset of this post run 120 times faster because that
> > is the difference between Ruby and optimized Python (when Psyco is
> > used to optimize the Python).
>
> > Whether or not Rubynauts feel like Python is perversion or not it is
> > amazing easy to achieve machine code speeds from Python as opposed to
> > the difficulty in getting Ruby code to run even 5x faster once
> > optimized.
>
> I'm only just now reading about Psyco. But the speed gains
> described on the introduction page[1] seem about what one would
> expect if similar techniques were used with Ruby:
>
> "For common code, expect at least a 2x speed-up, more typically 4x.
> But where Psyco shines is when running algorithmical code ..."
>
> "The difference with the traditional approach to JIT compilers is
> that Psyco writes several version of the same blocks (a block is a
> bit of a function), which are optimized by being specialized to
> some kinds of variables (a "kind" can mean a type, but it is more
> general)."
>
> So it could compile specialized versions of (portions of) a method
> that would be called if the arguments are Floats or Fixnums.
>
> I would imagine such a technique could work with Ruby, provided
> one agrees to disallow various Numeric methods from being
> redefined.
>
> For example, Ruby allows things like this:
>
> class Fixnum
> alias_method ld_times, :*
> def *(other)
> warn("i'm thinking: #{self} * #{other}")
> old_times(other)
> end
> end
>
> >> x = 6; y = 7
> >> x * y
>
> i'm thinking: 6 * 7
> => 42
>
> I'd suppose if one is willing to give up the ability to modify
> a certain set of Fixnum, Float, and perhaps String methods, one
> could make the equivalent of Psyco for Ruby.
>
> Indeed, there are a couple of already existing projects in
> this general category:
>
> http://rubyforge.org/projects/ruby2c/
> http://rubyforge.org/projects/ruby2cext/
>
> [1]http://psyco.sourceforge.net/introduction.html
>
> Regards,
>
> Bill

ruby2c and ruby2cext are hacks that convert Ruby code into C code.

Pysco is a JIT that works transparently at runtime.

If you want to code C/C++ then code C/C++ and be done with it.

If you want to code Ruby then live with the benefits or detriments of
using Ruby including the less than stellar performance.

I have coded both Ruby and Python and for me some aspects of Ruby code
is bass-ackwards and less than intuitive even though Ruby is supposed
to be an easy to understand langauge that is intuitive, blah-blah-
blah. For me Python seems far more intuitive than Ruby could ever
be.

Maybe I am just not cool enough to think Ruby is god's gift to
programming but when I write code I want nothing less than performance
relative to the amount of time I wish to spend writing code.

If I wanted blazing performance I would write Aseembly Language (been
there and done that back in the day when 8 bits was all there was) but
one must commit a lot of time to hand-crafting Assembly code and most
of the time I just don't have that much time to spend writing code.

Once upon a time I used to code everything in C but that was well
before C++ existed as anything other than an idea. And 'yes' I was
coding some rather intense C code, back in the day, using C++ like
constructs.

Python uses some of the same syntax as Ruby. Python is dynamic like
Ruby. Python source can be compiled into byte-codes and rolled-up
into an .EXE for easy delivery to end users. I can code Windows
services usign Python if I wanted to but I cannot code Windows
services using Ruby whether I wanted to or not.

When I write code that I wish to distribute I would rather not have to
ship source code. I realize shipping source code is supposed to be
"cool" but I think it is more "cool" to withhold my source code so my
customers might be more tempted to buy my object-code than hack-up my
source code. Also I think it is more "cool" to cause those who wish
to reverse engineer my code have to spend more time doing so rather
than simply reading the source code such as is the case with Ruby.

Also if someone could code a JIT for Ruby like Psyco then someone
would have done this already and we wouldn't have to talk about ruby2c
or ruby2ext.

The bottom line is this: "Ruby coders will always have to spend more
time making their Ruby code run faster than Python coders will have to
spend making their Python code run faster and Python code will always
run faster than Ruby code even after YARV has been released to save
the day until such time as someone codes a JIT for Ruby." <-- Wake me
up when this statement proves to be false because I will be happy when
the day comes when Ruby code runs faster than Python code and I don't
have to ship Ruby source code.

--nextPart2117252.tOTnPM945E
Content-Type: text/plain;
charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline

Quoth Ruby Maniac:
> On Sep 23, 7:55 pm, "Bill Kelly" <bi...@cts.com> wrote:
> > From: "Ruby Maniac" <raych...@hotmail.com>
> >
> >
> >
> > > Okay but wake me up when you can make the original Ruby code I
> > > submitted at the outset of this post run 120 times faster because that
> > > is the difference between Ruby and optimized Python (when Psyco is
> > > used to optimize the Python).
> >
> > > Whether or not Rubynauts feel like Python is perversion or not it is
> > > amazing easy to achieve machine code speeds from Python as opposed to
> > > the difficulty in getting Ruby code to run even 5x faster once
> > > optimized.
> >
> > I'm only just now reading about Psyco. But the speed gains
> > described on the introduction page[1] seem about what one would
> > expect if similar techniques were used with Ruby:
> >
> > "For common code, expect at least a 2x speed-up, more typically 4x.
> > But where Psyco shines is when running algorithmical code ..."
> >
> > "The difference with the traditional approach to JIT compilers is
> > that Psyco writes several version of the same blocks (a block is a
> > bit of a function), which are optimized by being specialized to
> > some kinds of variables (a "kind" can mean a type, but it is more
> > general)."
> >
> > So it could compile specialized versions of (portions of) a method
> > that would be called if the arguments are Floats or Fixnums.
> >
> > I would imagine such a technique could work with Ruby, provided
> > one agrees to disallow various Numeric methods from being
> > redefined.
> >
> > For example, Ruby allows things like this:
> >
> > class Fixnum
> > alias_method ld_times, :*
> > def *(other)
> > warn("i'm thinking: #{self} * #{other}")
> > old_times(other)
> > end
> > end
> >
> > >> x =3D 6; y =3D 7
> > >> x * y
> >
> > i'm thinking: 6 * 7
> > =3D> 42
> >
> > I'd suppose if one is willing to give up the ability to modify
> > a certain set of Fixnum, Float, and perhaps String methods, one
> > could make the equivalent of Psyco for Ruby. =20
> >
> > Indeed, there are a couple of already existing projects in
> > this general category:
> >
> > http://rubyforge.org/projects/ruby2c/
> > http://rubyforge.org/projects/ruby2cext/
> >
> > [1]http://psyco.sourceforge.net/introduction.html
> >
> > Regards,
> >
> > Bill
>=20
> ruby2c and ruby2cext are hacks that convert Ruby code into C code.
>=20
> Pysco is a JIT that works transparently at runtime.
>=20
> If you want to code C/C++ then code C/C++ and be done with it.
>=20
> If you want to code Ruby then live with the benefits or detriments of
> using Ruby including the less than stellar performance.
>=20
> I have coded both Ruby and Python and for me some aspects of Ruby code
> is bass-ackwards and less than intuitive even though Ruby is supposed
> to be an easy to understand langauge that is intuitive, blah-blah-
> blah. For me Python seems far more intuitive than Ruby could ever
> be.
>=20
> Maybe I am just not cool enough to think Ruby is god's gift to
> programming but when I write code I want nothing less than performance
> relative to the amount of time I wish to spend writing code.
>=20
> If I wanted blazing performance I would write Aseembly Language (been
> there and done that back in the day when 8 bits was all there was) but
> one must commit a lot of time to hand-crafting Assembly code and most
> of the time I just don't have that much time to spend writing code.
>=20
> Once upon a time I used to code everything in C but that was well
> before C++ existed as anything other than an idea. And 'yes' I was
> coding some rather intense C code, back in the day, using C++ like
> constructs.
>=20
> Python uses some of the same syntax as Ruby. Python is dynamic like
> Ruby. Python source can be compiled into byte-codes and rolled-up
> into an .EXE for easy delivery to end users. I can code Windows
> services usign Python if I wanted to but I cannot code Windows
> services using Ruby whether I wanted to or not.
>=20
> When I write code that I wish to distribute I would rather not have to
> ship source code. I realize shipping source code is supposed to be
> "cool" but I think it is more "cool" to withhold my source code so my
> customers might be more tempted to buy my object-code than hack-up my
> source code. Also I think it is more "cool" to cause those who wish
> to reverse engineer my code have to spend more time doing so rather
> than simply reading the source code such as is the case with Ruby.
>=20
> Also if someone could code a JIT for Ruby like Psyco then someone
> would have done this already and we wouldn't have to talk about ruby2c
> or ruby2ext.
>=20
> The bottom line is this: "Ruby coders will always have to spend more
> time making their Ruby code run faster than Python coders will have to
> spend making their Python code run faster and Python code will always
> run faster than Ruby code even after YARV has been released to save
> the day until such time as someone codes a JIT for Ruby." <-- Wake me
> up when this statement proves to be false because I will be happy when
> the day comes when Ruby code runs faster than Python code and I don't
> have to ship Ruby source code.

Yup, a troll.

Someone go get the troll-repellent charms.
=2D-=20
Konrad Meyer <> http://konrad.sobertillnoon.com/

--nextPart2117252.tOTnPM945E
Content-Type: application/pgp-signature; name=signature.asc
Content-Description: This is a digitally signed message part.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (GNU/Linux)

iD8DBQBG9zjyCHB0oCiR2cwRAnGOAJ4pF467vbmRbahxAGpCK8 qupAmXJgCffyCu
YEhR3mcj6leoLFes6dqINBU=
=deee
-----END PGP SIGNATURE-----

--nextPart2117252.tOTnPM945E--

On Sep 22, 3:16 am, William James <w_a_x_...@yahoo.com> wrote:
> On Sep 20, 5:02 pm, Ruby Maniac <raych...@hotmail.com> wrote:
>
>
>
>
>
> > I am new to Ruby and I could use some expert advice as to how I can
> > make this code run faster.
>
> > def scramble(fname)
> > f = File.new(fname, "rb")
> > _fname = fname + ".scrambled"
> > begin
> > File.exist?(_fname) if File.delete(_fname)
> > rescue
> > end
> > ff = File.new(_fname, "wb+")
> > for l in f
> > l.each_byte{|c| ff.write((c | 0x80).chr) }
> > end
> > f.close()
> > ff.close()
> > end
>
> If you actually want speed, use a compiled language.
> This FreeBasic code is about 160 times as fast and
> demonstrates that using Python for any reason
> whatsoever is unjustified.
>
> Use Ruby when ease of programming is paramount;
> use something like FreeBasic or LuaJIT when speed
> is paramount. Never submit to Python perversion.
>
> sub scramble( filename as string )
> dim outname as string
> dim as integer in_handle, out_handle, i, size
> dim bytes(1 to 4096) as byte
>
> outname = filename & ".scrambled"
> in_handle = freefile
> open filename for binary access read as in_handle
> out_handle = freefile
> open outname for binary access write as out_handle
> while not eof( in_handle )
> size = seek( in_handle )
> get #in_handle, , bytes()
> size = seek( in_handle ) - size
> for i = 1 to size
> bytes(i) = bytes(i) or &h80
> next
> put #out_handle, , bytes(1), size
> wend
> close
> end sub- Hide quoted text -
>
> - Show quoted text -

Okay I downloaded Free BASIC and installed it but your code does not
work !

On the one hand, FreeBASIC is certainly worth looking into but on the
other hand - there ain't no debugger than I know of unless you know of
a debugger for FreeBASIC.

From: "Ruby Maniac" <>
>
> I have coded both Ruby and Python and for me some aspects of Ruby code
> is bass-ackwards and less than intuitive even though Ruby is supposed
> to be an easy to understand langauge that is intuitive, blah-blah-
> blah. For me Python seems far more intuitive than Ruby could ever
> be.

I must be missing the point of this discussion then?

If you like Python, find a job programming Python.

Problem solved?

>>> exit
'Use Ctrl-D (i.e. EOF) to exit.'
>>> quit
'Use Ctrl-D (i.e. EOF) to exit.'
>>>


Regards,

Bill

Ruby Maniac wrote:
> On Sep 22, 3:16 am, William James <w_a_x_...@yahoo.com> wrote:
> > On Sep 20, 5:02 pm, Ruby Maniac <raych...@hotmail.com> wrote:
> >
> >
> >
> >
> >
> > > I am new to Ruby and I could use some expert advice as to how I can
> > > make this code run faster.
> >
> > > def scramble(fname)
> > > f = File.new(fname, "rb")
> > > _fname = fname + ".scrambled"
> > > begin
> > > File.exist?(_fname) if File.delete(_fname)
> > > rescue
> > > end
> > > ff = File.new(_fname, "wb+")
> > > for l in f
> > > l.each_byte{|c| ff.write((c | 0x80).chr) }
> > > end
> > > f.close()
> > > ff.close()
> > > end
> >
> > If you actually want speed, use a compiled language.
> > This FreeBasic code is about 160 times as fast and
> > demonstrates that using Python for any reason
> > whatsoever is unjustified.
> >
> > Use Ruby when ease of programming is paramount;
> > use something like FreeBasic or LuaJIT when speed
> > is paramount. Never submit to Python perversion.
> >
> > sub scramble( filename as string )
> > dim outname as string
> > dim as integer in_handle, out_handle, i, size
> > dim bytes(1 to 4096) as byte
> >
> > outname = filename & ".scrambled"
> > in_handle = freefile
> > open filename for binary access read as in_handle
> > out_handle = freefile
> > open outname for binary access write as out_handle
> > while not eof( in_handle )
> > size = seek( in_handle )
> > get #in_handle, , bytes()
> > size = seek( in_handle ) - size
> > for i = 1 to size
> > bytes(i) = bytes(i) or &h80
> > next
> > put #out_handle, , bytes(1), size
> > wend
> > close
> > end sub- Hide quoted text -
> >
> > - Show quoted text -
>
> Okay I downloaded Free BASIC and installed it but your code does not
> work !
>
> On the one hand, FreeBASIC is certainly worth looking into but on the
> other hand - there ain't no debugger than I know of unless you know of
> a debugger for FreeBASIC.

Did it compile without errors? Did you make sure that no lines of the
source were broken by your newsgroup program? (If using google,
click on "More options" and "Show original".) I didn't include code
to invoke the subroutine; I left that up to you.

Debuggers are crutches; I never use them.

Here's a complete program. Change the filename that is read by
the program.


function scramble( filename as string ) as double
dim as integer in_handle, out_handle, i, size
dim bytes(1 to 4096) as byte

in_handle = freefile
open filename for binary access read as in_handle
out_handle = freefile
open filename & ".scrambled" for binary access write as out_handle
while not eof( in_handle )
size = seek( in_handle )
get #in_handle, , bytes()
size = seek( in_handle ) - size
for i = 1 to size
bytes(i) = bytes(i) or &h80
next
put #out_handle, , bytes(1), size
wend
close '-- Caution. Closes all open files.
return 0.0
end function

print "! ";
print -timer + scramble("testjunk.csv") + timer

On 9/22/07, Nobuyoshi Nakada <> wrote:
> Hi,
>
> At Fri, 21 Sep 2007 07:05:10 +0900,
> Ruby Maniac wrote in [ruby-talk:270110]:
> > def scramble(fname)
> > _fname = fname + ".scrambled"
> File.new(fname, "rb") do |f|
> File.new(_fname, "wb+") do |ff|
> ff.write(f.read.tr("\0-\177", "\200-\377"))
> end
> end
> > end
>
> --
> Nobu Nakada
>

And the benchmark results for the above code on a 6.7M file:

$ time ruby -rscramble -e 'scramble_original("trace")'

real 0m11.626s
user 0m9.781s
sys 0m1.844s

$ time ruby -rscramble -e 'scramble_nobu("trace")'

real 0m0.099s
user 0m0.044s
sys 0m0.032s

116x speedup and 7 less loc

$ cat scramble.rb
def scramble_original(fname)
f = File.new(fname, "rb")
_fname = fname + ".scrambled"
begin
File.exist?(_fname) if File.delete(_fname)
rescue
end
ff = File.new(_fname, "wb+")
for l in f
l.each_byte{|c| ff.write((c | 0x80).chr) }
end
f.close()
ff.close()
end

def scramble_nobu(fname)
File.open(fname, 'rb'){|in_f|
File.open(fname + '.scrambled', 'wb'){|out_f|
out_f.write(in_f.read.tr("\0-\177", "\200-\377"))
}
}
end

On Sep 22, 3:16 am, William James <w_a_x_...@yahoo.com> wrote:
> On Sep 20, 5:02 pm, Ruby Maniac <raych...@hotmail.com> wrote:
>
>
>
>
>
> > I am new to Ruby and I could use some expert advice as to how I can
> > make this code run faster.
>
> > def scramble(fname)
> > f = File.new(fname, "rb")
> > _fname = fname + ".scrambled"
> > begin
> > File.exist?(_fname) if File.delete(_fname)
> > rescue
> > end
> > ff = File.new(_fname, "wb+")
> > for l in f
> > l.each_byte{|c| ff.write((c | 0x80).chr) }
> > end
> > f.close()
> > ff.close()
> > end
>
> If you actually want speed, use a compiled language.
> This FreeBasic code is about 160 times as fast and
> demonstrates that using Python for any reason
> whatsoever is unjustified.
>
> Use Ruby when ease of programming is paramount;
> use something like FreeBasic or LuaJIT when speed
> is paramount. Never submit to Python perversion.
>
> sub scramble( filename as string )
> dim outname as string
> dim as integer in_handle, out_handle, i, size
> dim bytes(1 to 4096) as byte
>
> outname = filename & ".scrambled"
> in_handle = freefile
> open filename for binary access read as in_handle
> out_handle = freefile
> open outname for binary access write as out_handle
> while not eof( in_handle )
> size = seek( in_handle )
> get #in_handle, , bytes()
> size = seek( in_handle ) - size
> for i = 1 to size
> bytes(i) = bytes(i) or &h80
> next
> put #out_handle, , bytes(1), size
> wend
> close
> end sub- Hide quoted text -
>
> - Show quoted text -

Finally got this FreeBASIC code to work but it reports a runtime of 3
seconds on a machine that reports 0.811 seconds for Python using Psyco
so this means FreeBASIC is not anywhere near as fast as Python
considering the Python code reads the whole file and then performs a
single String Translate and then write the whole file in 3 operations.