«

Hi,

Once in a while the question pops up if it is possible to compile Ruby
code to native machine code. The answer has always been no. But I keep
wondering how hard it would really be to make this possible.

Ruby is written in C. And when Ruby parses a Ruby script it converts
each statement to a C call. Probably the same calls you can use on your
own in a Ruby C extension. So why wouldn't it be possible to parse a
Ruby script and convert all statements to Ruby C code and put it in a
*.c file (instead of calling the Ruby C statements directly). This *.c
file can then be compiled into machine code with a C compiler like gcc.
If each *.rb file is converted to a C file it could be compiled to a
dynamically loadable library which could then be used on require
statements (just like regular Ruby C extensions).

What I mean is, this...

class Example
def example
puts "Hello World!"
end
end

.... can also be written in C using the Ruby C API, am I right? So why
wouldn't it be possible to convert all Ruby code to C code using the
Ruby C API?

This would probably result in some performance gain (no need to parse
the code anymore at run-time), but for some people more important, you
can distribute your Ruby applications closed-source. In the future the
performance gain maybe could be increased by performing special
optimizations during the conversion process.

Am I right on this, or do I forget something important which makes the
above quite hard to do?

With kind regards,

Peter

"Nospam" <news.home.nl-> schrieb im Newsbeitrag
news:cdgcqd$67n$...
> Hi,
>
> Once in a while the question pops up if it is possible to compile Ruby
> code to native machine code. The answer has always been no. But I keep
> wondering how hard it would really be to make this possible.
>
> Ruby is written in C. And when Ruby parses a Ruby script it converts
> each statement to a C call. Probably the same calls you can use on your
> own in a Ruby C extension. So why wouldn't it be possible to parse a
> Ruby script and convert all statements to Ruby C code and put it in a
> *.c file (instead of calling the Ruby C statements directly). This *.c
> file can then be compiled into machine code with a C compiler like gcc.
> If each *.rb file is converted to a C file it could be compiled to a
> dynamically loadable library which could then be used on require
> statements (just like regular Ruby C extensions).
>
> What I mean is, this...
>
> class Example
> def example
> puts "Hello World!"
> end
> end
>
> ... can also be written in C using the Ruby C API, am I right? So why
> wouldn't it be possible to convert all Ruby code to C code using the
> Ruby C API?

I guess it *is* possible, but then you would have to bundle gcc with the
Ruby interpreter, because you must deal with dynamic redefinition of
methods and eval.

> This would probably result in some performance gain (no need to parse
> the code anymore at run-time),

AFAIK at runtime there is a parse phase and an execute phase. Each line
of code is only parsed once and then converted into some kind of byte code
(not the Java flavour), so you probably don't gain much time.

> but for some people more important, you
> can distribute your Ruby applications closed-source.

That would impose certain restrictions, especially you must forbid this:

# just a stupid example that loads different files
# depending on some runtime state
case some_var
when "foo"
require "x-as-foo-impl.rb"
when "bar"
require "x-as-bar-impl.rb"
when /^(\w+)-\w+$/
require "x-as-#{$1}-impl.rb"
else
require "x-default-impl.rb"
end

> In the future the
> performance gain maybe could be increased by performing special
> optimizations during the conversion process.
>
> Am I right on this, or do I forget something important which makes the
> above quite hard to do?

I think so, as noted above.

Regards

robert

Nospam wrote:
> Hi,
>
> Once in a while the question pops up if it is possible to compile Ruby
> code to native machine code. The answer has always been no. But I keep
> wondering how hard it would really be to make this possible.
>
> Ruby is written in C. And when Ruby parses a Ruby script it converts
> each statement to a C call. Probably the same calls you can use on your
> own in a Ruby C extension. So why wouldn't it be possible to parse a
> Ruby script and convert all statements to Ruby C code and put it in a
> *.c file (instead of calling the Ruby C statements directly). This *.c
> file can then be compiled into machine code with a C compiler like gcc.
> If each *.rb file is converted to a C file it could be compiled to a
> dynamically loadable library which could then be used on require
> statements (just like regular Ruby C extensions).
>
> What I mean is, this...
>
> class Example
> def example
> puts "Hello World!"
> end
> end
>
> .... can also be written in C using the Ruby C API, am I right? So why
> wouldn't it be possible to convert all Ruby code to C code using the
> Ruby C API?
>
> This would probably result in some performance gain (no need to parse
> the code anymore at run-time), but for some people more important, you
> can distribute your Ruby applications closed-source. In the future the
> performance gain maybe could be increased by performing special
> optimizations during the conversion process.

I remember that a long time ago, there was a ruby-to-c compiler (was it
called r2c?). But IIRC, there was only little performance gain. Remember
that you still need a Ruby parser, due to "eval". It would be nice, but
I'd even more like to see a bytecode compiler (written in pure Ruby
running on top of the bytecode interpreter).

Regards,

Michael

Michael Neumann wrote:
> Nospam wrote:
>
>> Hi,
>>
>> Once in a while the question pops up if it is possible to compile Ruby
>> code to native machine code. The answer has always been no. But I keep
>> wondering how hard it would really be to make this possible.
>>
>> Ruby is written in C. And when Ruby parses a Ruby script it converts
>> each statement to a C call. Probably the same calls you can use on
>> your own in a Ruby C extension. So why wouldn't it be possible to
>> parse a Ruby script and convert all statements to Ruby C code and put
>> it in a *.c file (instead of calling the Ruby C statements directly).
>> This *.c file can then be compiled into machine code with a C compiler
>> like gcc. If each *.rb file is converted to a C file it could be
>> compiled to a dynamically loadable library which could then be used on
>> require statements (just like regular Ruby C extensions).
>>
>> What I mean is, this...
>>
>> class Example
>> def example
>> puts "Hello World!"
>> end
>> end
>>
>> .... can also be written in C using the Ruby C API, am I right? So why
>> wouldn't it be possible to convert all Ruby code to C code using the
>> Ruby C API?
>>
>> This would probably result in some performance gain (no need to parse
>> the code anymore at run-time), but for some people more important, you
>> can distribute your Ruby applications closed-source. In the future the
>> performance gain maybe could be increased by performing special
>> optimizations during the conversion process.
>
>
> I remember that a long time ago, there was a ruby-to-c compiler (was it
> called r2c?). But IIRC, there was only little performance gain. Remember
> that you still need a Ruby parser, due to "eval". It would be nice, but
> I'd even more like to see a bytecode compiler (written in pure Ruby
> running on top of the bytecode interpreter).
>
> Regards,
>
> Michael
>
>

Yes, I have to vote for a bytecode compiler. Right now I'm using ruby
to develop some software on an embedded arm-linux device. I've found
that for this application, which is not very demanding of the system,
that ruby can perform pretty comparably to an equivalent C program. And
the development time it saves to write in a higher level language is
worth its weight in gold. But every time I run a ruby program it takes
it a significant and noticable amount of time to start up. This is
obviously due to the ruby parser compiling the text into bytecode every
time. If it was possible to precompile this bytecode and put that on the
target machine it would have significant advantages. Also, it would make
it possible to distribute the application without distributing the
source code to it. That isn't so important for us, but it may be for
others.

2 cents. Scott

il Mon, 19 Jul 2004 13:54:53 +0200, Nospam
<news.home.nl-> ha scritto::

I agree that this could be done, linking the runtime for dynamic
behaviour.
but the performance hit would still be in the method lookup phase, as
projects like python-to-c converter pointed out.
You need to do some clever type inference and runtime optimization,
and that won't be so easy.
Look out for starkiller (python compiler+type inference) for a
discussion on this.

On Mon, 19 Jul 2004 13:54:53 +0200, Nospam wrote:

> Hi,

Hi

> <snip>
>
> What I mean is, this...
>
> class Example
> def example
> puts "Hello World!"
> end
> end
>
> ... can also be written in C using the Ruby C API, am I right? So why
> wouldn't it be possible to convert all Ruby code to C code using the
> Ruby C API?

Yes, that's possible, but depending on the C code you write, there
isn't much performance gain. You can write any ruby code in C,
but if you just convert it litteraly to C, there isn't much performance
gain. For example when you write
arr[1] = 2 #arr is an Array
you could use rb_ary_store(arr, 1, INT2FIX(2)), but that wouldn't be
equivalent to the ruby code. When the Array#[] method would change, the
ruby code would see the change, but the C code wouldn't. The solution
would be to use something like rb_funcall(arr, rb_intern("[]"), 0), but
then there would be no performance benefit, because ruby still has to look
up the method. The point is that it is possible to write more efficient
ruby code in C, but it isn't really ruby code anymore.

>
> This would probably result in some performance gain (no need to parse
> the code anymore at run-time), but for some people more important, you
> can distribute your Ruby applications closed-source. In the future the
> performance gain maybe could be increased by performing special
> optimizations during the conversion process.
>

The parsing fase doesn't take so much performance, since it is only done
at loading time.

> Am I right on this, or do I forget something important which makes the
> above quite hard to do?
>
> With kind regards,
>
> Peter

Well, I do think it is possible to compile Ruby, but it would be to hard.
Firstly eval and module_eval should be thrown away, because they need to
be able to parse code at runtime.
Continuations make compiling very messy, since the stack needs to be
copied. This can also can cause troubles when interfacing with native
c-calls. (I wonder how the current Ruby-interpreter manages this...)

The best way is IMO to have a different language that resembles Ruby
as much as possible, but allows it to be compiled easily. (I am sure
Matz wouldn't allow Ruby to be crippled to allow compilation).
It would have limitations that Ruby doesn't have (no eval or continuations),
but it would be a nice alternative for coding in C. In fact code blocks
and closures aren't that difficult to compile, and I think any decent
compilable language should have them.

As for Ruby, the best performance gain would be by using a Just-In-Time
compiler. This way, the compiler can make assumptions about the code, and
recompile when this assumptions are wrong. Take for example the following
code:
5.times do { |i| puts i }
Here the compiler could inline the code for times and produce very
efficient code. The same goes for other standard library functions like
Array#[], etc. However it is still possible to redefine Numeric#times
(though I don't see a good reason to!). If the method would be redefined,
then the just-in-time compiler could recompile it. The same goes for
method arguments. When an argument to a method is always from the same
class, the compiler can call the methods on that object directly (without
lookup), or even inline them. However when different kinds of objects get
passed to the method, then the JIT-compiler could recompile the method, so
that it works with all objects.

In this way the JIT can create well performing code, and at the same time
keep the dynamic nature of Ruby. When methods are redefined, or added,
the jit-compiler makes the changes.
It would be nice to have such an interpreter, but it would be a
considerable amount of work to implement it, as it would need a
complete rewrite of the Ruby-interpreter.

Regards,
Kristof

Kristof Bastiaensen <> writes:

> Well, I do think it is possible to compile Ruby, but it would be to hard.
> Firstly eval and module_eval should be thrown away, because they need to
> be able to parse code at runtime.
> Continuations make compiling very messy, since the stack needs to be
> copied.

That's not strictly true. If you convert the code to
continuation-passing style, reifying continuations doesn't require
copying the stack. CHICKEN and many other Scheme compilers choose this
approach.

mikael

On Tue, 20 Jul 2004 00:00:22 +0900, Mikael Brockman wrote:

Hi,

> Kristof Bastiaensen <> writes:
>
>> Well, I do think it is possible to compile Ruby, but it would be to hard.
>> Firstly eval and module_eval should be thrown away, because they need to
>> be able to parse code at runtime.
>> Continuations make compiling very messy, since the stack needs to be
>> copied.
>
> That's not strictly true. If you convert the code to
> continuation-passing style, reifying continuations doesn't require
> copying the stack. CHICKEN and many other Scheme compilers choose this
> approach.
>
> mikael

That's interesting. Could you explain how that works?
My idea was that when a continuation is saved, it needs
to keep the information, where it is going to (return
adresses), and all local bindings. I would think the best
way is to save the stack, at least for compiled (machine)
code.

Kristof

> Once in a while the question pops up if it is
> possible to compile Ruby
> code to native machine code. The answer has always
> been no. But I keep
> wondering how hard it would really be to make this
> possible.

The answer is really yes. Why? People say no because
of the implementation on how it is supposed to work.
It is very possible to, but it would take time.


>
> Ruby is written in C. And when Ruby parses a Ruby
> script it converts
> each statement to a C call. Probably the same calls
> you can use on your
> own in a Ruby C extension. So why wouldn't it be
> possible to parse a
> Ruby script and convert all statements to Ruby C
> code and put it in a
> *.c file (instead of calling the Ruby C statements
> directly). This *.c
> file can then be compiled into machine code with a C
> compiler like gcc.

Evaluated code at runtime needs to be thought of.
There needs to be a small runtime running on top of
the compiled program.


> Am I right on this, or do I forget something
> important which makes the
> above quite hard to do?

Just very hard to do and no one wants to bother with
creating it. Which is why I emailed "Ruby
specification" on the mailing list about creating
rubycc. --David Ross



__________________________________
Do you Yahoo!?
Vote for the stars of Yahoo!'s next ad campaign!
http://advision.webevents.yahoo.com/...otelifeengine/

On Tuesday, July 20, 2004, 1:48:17 AM, David wrote:

>> Am I right on this, or do I forget something
>> important which makes the
>> above quite hard to do?

> Just very hard to do and no one wants to bother with
> creating it. Which is why I emailed "Ruby
> specification" on the mailing list about creating
> rubycc.

My fearless prediction: no-one will write a Ruby specification if you
don't. (Unless someone else is working hard on a compiler that we
don't know, or that I've forgotten.)

What is your rationale (in detail) for not looking at the Ruby source?

Lots of people would like to see a Ruby compiler, so any attempt by
yourself to create an English specification would almost certainly be
supported by several people testing it thoroughly for correctness.
Why not start now?

Cheers,
Gavin