Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > C++ > Re: benchmarks? java vs .net (sin and cos)

Reply
Thread Tools

Re: benchmarks? java vs .net (sin and cos)

 
 
Jon Skeet [C# MVP]
Guest
Posts: n/a
 
      06-05-2008
Razii <(E-Mail Removed)> wrote:
> On Wed, 4 Jun 2008 23:25:27 +0100, Jon Skeet [C# MVP]
> <(E-Mail Removed)> wrote:
>
> >That's changing the Java code. Now what about the C# code - and is that
> >with the big file, or the original benchmark way of creating thousands
> >of processes?

>
> The original benchmark uses this file
>
> http://shootout.alioth.debian.org/do...mcol-input.txt
>
> made 21,000 times bigger. Basically, concatenate the above file 21,000
> times, and you have the same file that I and shootout site do.
>
> As for C# using file, post the changes.


I did, early yesterday evening. I posted both a Java *and* a C# program
which allowed you to either pipe input to the program *or* provide a
filename.

I also posted the results of running said code - C# was slower than
Java when reading from stdin (at the time using Console.In - using
OpenStandardInput narrows the gap) and about the same as Java when
reading directly from a file.

--
Jon Skeet - <(E-Mail Removed)>
Web site: http://www.pobox.com/~skeet
Blog: http://www.msmvps.com/jon.skeet
C# in Depth: http://csharpindepth.com
 
Reply With Quote
 
 
 
 
Jon Skeet [C# MVP]
Guest
Posts: n/a
 
      06-05-2008
Jon Harrop <(E-Mail Removed)> wrote:
> Jon Skeet [C# MVP] wrote:
> > Mark Thornton <(E-Mail Removed)> wrote:
> >> > (Another nail in the coffin of the claim that ".NET is twice as slow as
> >> > Java" though,
> >> I don't see much in these benchmarks of the sort of code where the more
> >> mature HotSpot JIT could be expected to outshine .NET.

> >
> > Indeed. As I said before, it's pretty easy to construct code where Java
> > *does* outperform C#:
> > ...
> > As it is, Java outperforms .NET...

>
> That is Java outperforming C#, not .NET.


No, it's Java outperforming .NET calling a virtual method. It has
nothing to do with the code that the IL is generated in: if you have a
virtual method which isn't overridden, the .NET CLR will not inline
calls to it. The Hotspot VM will inline it until it first sees the
method being overridden. The difference here is in the jitting model,
not in the language.

Try it in F#, ensuring that the method is virtual and actually called
each time (consider that the method might be one which has side-
effects). I can't see how it would possibly have a different result.

--
Jon Skeet - <(E-Mail Removed)>
Web site: http://www.pobox.com/~skeet
Blog: http://www.msmvps.com/jon.skeet
C# in Depth: http://csharpindepth.com
 
Reply With Quote
 
 
 
 
Barry Kelly
Guest
Posts: n/a
 
      06-05-2008
Jon Harrop wrote:

> Jon Skeet [C# MVP] wrote:
> > Jon Harrop <(E-Mail Removed)> wrote:
> >> That is Java outperforming C#, not .NET.

> >
> > No, it's Java outperforming .NET calling a virtual method. It has
> > nothing to do with the code that the IL is generated in: if you have a
> > virtual method which isn't overridden, the .NET CLR will not inline
> > calls to it. The Hotspot VM will inline it until it first sees the
> > method being overridden. The difference here is in the jitting model,
> > not in the language.
> >
> > Try it in F#, ensuring that the method is virtual and actually called
> > each time (consider that the method might be one which has side-
> > effects). I can't see how it would possibly have a different result.

>
> This optimization can obviously be done statically by the compiler.


No - this optimization cannot be done statically for runtimes that
support dynamic code loading, as the CLR and JVM do. If the compiler
tries to replace 'callvirt' with 'call', the assembly will no longer be
verifiable, and if the assembly is dynamically loaded into the context
of another application, the virtual call semantics will be violated.

For example:

---8<---
using System;

public class B
{
public virtual void M()
{
Console.WriteLine("Hello from B");
}
}

public class App
{
static void Main()
{
CallM(new B());
}

public static void CallM(B instance)
{
instance.M();
}
}
--->8---

Compile this App.cs to App.exe, then disassemble with:

ildasm App.exe /out:App.il

Then edit App.il so that CallM is written thusly, applying the
"optimization":

---8<---
.method public hidebysig static void CallM(class B 'instance') cil
managed
{
// Code size 9 (0x9)
.maxstack 8
IL_0000: nop
IL_0001: ldarg.0
IL_0002: call instance void B::M()
// (above was 'callvirt')
IL_0007: nop
IL_0008: ret
} // end of method App::CallM
--->8---

Assemble with:

ilasm App.il

Verify with:

peverify App.exe

Observe error:

---8<---
Microsoft (R) .NET Framework PE Verifier. Version 3.5.21022.8
Copyright (c) Microsoft Corporation. All rights reserved.

[IL]: Error: [c:\proj\cli-callvirt\App.exe : App::CallM][offset
0x00000002] The 'this' parameter to the call must be the calling
method's 'this' parameter.
1 Error Verifying App.exe
--->8---

For a concrete example of virtual method call semantic violation,
consider a second application, App2.cs:

---8<---
using System;

public class D : B
{
public override void M()
{
Console.WriteLine("Hello from D!");
}
}

class App2
{
static void Main()
{
App.CallM(new D());
}
}
--->8---

Compile this with:

csc App2.cs /r:App.exe

Run App2.exe when App.exe is the version assembled from App.il modified
to use call instead of callvirt, and observe:

---8<---
Hello from B
--->8---

Recompile the original App.cs from source, then run App2, and observe:

---8<---
Hello from D!
--->8---

- the "obvious optimization" has broken virtual method call semantics.

Implementing the "optimization" for strictly private classes is not
useful either, as reflection with appropriate permissions, along with
runtime code generation, can see through such visibility constraints.

-- Barry

--
http://barrkel.blogspot.com/
 
Reply With Quote
 
Mark Thornton
Guest
Posts: n/a
 
      06-05-2008
Jon Skeet [C# MVP] wrote:
> Mark Thornton <(E-Mail Removed)> wrote:
>>> (Another nail in the coffin of the claim that ".NET is twice as slow as
>>> Java" though,

>
>> I don't see much in these benchmarks of the sort of code where the more
>> mature HotSpot JIT could be expected to outshine .NET.

>
> As it is, Java outperforms .NET (on my box) by about 3:1.
> Making GetNumber non-virtual, and .NET outperforms Java (on my box)
> about about 2:1.
>
> Pointless benchmark, of course, other than to show the inlining
> capabilities of Hotspot in the face of a virtual method which hasn't
> been overridden.
>


It does have real relevance. I have classes with multiple
implementations, but quite frequently a given process will only use one
of the possible implementations.

Mark Thornton
 
Reply With Quote
 
Jon Skeet [C# MVP]
Guest
Posts: n/a
 
      06-05-2008
Jon Harrop <(E-Mail Removed)> wrote:
> >> This optimization can obviously be done statically by the compiler.

> >
> > No - this optimization cannot be done statically for runtimes that
> > support dynamic code loading, as the CLR and JVM do...

>
> I just posted a working example where the optimization is done, so it is
> clearly not impossible. The only question is (subjectively) was my code
> equivalent.


Is your method virtual, and called virtually? If not, then it's not
equivalent. If it is, then the JIT can't inline it.

--
Jon Skeet - <(E-Mail Removed)>
Web site: http://www.pobox.com/~skeet
Blog: http://www.msmvps.com/jon.skeet
C# in Depth: http://csharpindepth.com
 
Reply With Quote
 
Jon Skeet [C# MVP]
Guest
Posts: n/a
 
      06-05-2008
Mark Thornton <(E-Mail Removed)> wrote:
> > Pointless benchmark, of course, other than to show the inlining
> > capabilities of Hotspot in the face of a virtual method which hasn't
> > been overridden.

>
> It does have real relevance. I have classes with multiple
> implementations, but quite frequently a given process will only use one
> of the possible implementations.


Fair enough - but it's only significant if:
a) the implementation is small enough to inline
b) the method call would be a significant hit, i.e. it's a bottleneck
in the app

Obviously I can't speak for your experience, but mine is certainly that
the two don't come together very often. More likely in the scientific
world than the business world, I suspect.

--
Jon Skeet - <(E-Mail Removed)>
Web site: http://www.pobox.com/~skeet
Blog: http://www.msmvps.com/jon.skeet
C# in Depth: http://csharpindepth.com
 
Reply With Quote
 
Mark Thornton
Guest
Posts: n/a
 
      06-05-2008
Jon Skeet [C# MVP] wrote:
> Mark Thornton <(E-Mail Removed)> wrote:
>>> Pointless benchmark, of course, other than to show the inlining
>>> capabilities of Hotspot in the face of a virtual method which hasn't
>>> been overridden.

>> It does have real relevance. I have classes with multiple
>> implementations, but quite frequently a given process will only use one
>> of the possible implementations.

>
> Fair enough - but it's only significant if:
> a) the implementation is small enough to inline
> b) the method call would be a significant hit, i.e. it's a bottleneck
> in the app
>
> Obviously I can't speak for your experience, but mine is certainly that
> the two don't come together very often. More likely in the scientific
> world than the business world, I suspect.
>


I do vehicle routing algorithms, so although the application is sold to
businesses, its core is more scientific/mathematical in nature. Using
100% CPU is routine. I'm working on achieving that for multi core
machines as well.

Mark Thornton
 
Reply With Quote
 
Barry Kelly
Guest
Posts: n/a
 
      06-05-2008
Jon Harrop wrote:

> Jon Skeet [C# MVP] wrote:
> > Jon Harrop <(E-Mail Removed)> wrote:
> >> That is Java outperforming C#, not .NET.

> >
> > No, it's Java outperforming .NET calling a virtual method. It has
> > nothing to do with the code that the IL is generated in: if you have a
> > virtual method which isn't overridden, the .NET CLR will not inline
> > calls to it. The Hotspot VM will inline it until it first sees the
> > method being overridden. The difference here is in the jitting model,
> > not in the language.
> >
> > Try it in F#, ensuring that the method is virtual and actually called
> > each time (consider that the method might be one which has side-
> > effects). I can't see how it would possibly have a different result.

>
> This optimization can obviously be done statically by the compiler. There is
> no assurance that F# will also fail to optimize this, particularly if you
> write a functional equivalent:


We're not talking about "functional equivalents". We're talking about
virtual methods that are callable by third-party code written in any CLR
language, i.e. methods for which a MethodInfo instance can be obtained
from reflection, and for which MethodInfo.IsVirtual returns true and
MethodInfo.IsFinal returns false.

Any code in F# or any other language which doesn't compile to something
that can be loaded in MethodInfo form via System.Reflection doesn't
apply, because it simply isn't a virtual method.

-- Barry

--
http://barrkel.blogspot.com/
 
Reply With Quote
 
Barry Kelly
Guest
Posts: n/a
 
      06-05-2008
Jon Skeet wrote:

> Mark Thornton <(E-Mail Removed)> wrote:
> > > Pointless benchmark, of course, other than to show the inlining
> > > capabilities of Hotspot in the face of a virtual method which hasn't
> > > been overridden.

> >
> > It does have real relevance. I have classes with multiple
> > implementations, but quite frequently a given process will only use one
> > of the possible implementations.

>
> Fair enough - but it's only significant if:
> a) the implementation is small enough to inline
> b) the method call would be a significant hit, i.e. it's a bottleneck
> in the app
>
> Obviously I can't speak for your experience, but mine is certainly that
> the two don't come together very often. More likely in the scientific
> world than the business world, I suspect.


FWIW, when one adopts the test-first style of development, the ability
to create mocks (whether by inheritance or the proxy capabilities of the
CLR/etc.) and use dependency injection frameworks (inversion of control
pattern) to abstract away your dependencies becomes very important.

It's then that the ability of the runtime to optimize virtual method
calls to statically dispatched calls becomes a lot more important,
because:

1) Usually, the only point of a method being virtual is so that it can
easily be mocked

2) Usually, when running "for real" and not for test, only a single
implementation of that method will be loaded into the runtime.

I wouldn't discard the utility of the virtual -> static call
optimization so trivially.

Of course, some kind of more direct language or runtime support for
mocking and / or inversion of control might be a better solution than
willy-nilly making every method virtual, as that has other, more severe
repercussions for designability and testability - classes with virtual
methods being a lot harder to test due to the extra code paths.

-- Barry

--
http://barrkel.blogspot.com/
 
Reply With Quote
 
Jon Skeet [C# MVP]
Guest
Posts: n/a
 
      06-05-2008
Barry Kelly <(E-Mail Removed)> wrote:
> > Obviously I can't speak for your experience, but mine is certainly that
> > the two don't come together very often. More likely in the scientific
> > world than the business world, I suspect.

>
> FWIW, when one adopts the test-first style of development, the ability
> to create mocks (whether by inheritance or the proxy capabilities of the
> CLR/etc.) and use dependency injection frameworks (inversion of control
> pattern) to abstract away your dependencies becomes very important.


True.

> It's then that the ability of the runtime to optimize virtual method
> calls to statically dispatched calls becomes a lot more important,
> because:
>
> 1) Usually, the only point of a method being virtual is so that it can
> easily be mocked
>
> 2) Usually, when running "for real" and not for test, only a single
> implementation of that method will be loaded into the runtime.


Absolutely. I don't dispute that at all - I just wonder how often those
virtual methods are both small enough to be inlined *and* critical for
performance.

I'm sure there are some cases where it makes a really huge difference -
but I suspect it makes a *slight* difference to a far greater
proportion of apps.

> I wouldn't discard the utility of the virtual -> static call
> optimization so trivially.


One reason it's so important in Java of course is that methods are
virtual by default. C# (and VB.NET, I believe) methods are non-virtual
by default. The defaults wouldn't make a difference if every developer
diligently thought about whether or not to make a method virtual every
time they wrote it - but of course, few people (myself *not* included)
are that diligent.

> Of course, some kind of more direct language or runtime support for
> mocking and / or inversion of control might be a better solution than
> willy-nilly making every method virtual, as that has other, more severe
> repercussions for designability and testability - classes with virtual
> methods being a lot harder to test due to the extra code paths.


Well, just because an interface has a virtual method doesn't mean it
has to be virtual in the implementing class. But yes, inheritence of
implementation certainly has a design cost associated with it.

--
Jon Skeet - <(E-Mail Removed)>
Web site: http://www.pobox.com/~skeet
Blog: http://www.msmvps.com/jon.skeet
C# in Depth: http://csharpindepth.com
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Hot Requirements: 1.Sr Java Developer,2.Java Developer (Java with EJB) Isaac Java 0 01-20-2011 08:41 PM
hey i am just started java,, can anyone tell me the use ,application, why java , importance of java.. manish sahu Java 3 02-14-2008 12:00 AM
Another Question: Java and other application, Java and hardware Roberto Faenza Java 4 02-25-2007 06:18 PM
[JAVA] [EVALUATION] - The Java Failure (Sorry: The Java(tm) Failure) Ilias Lazaridis Java 0 02-01-2005 10:32 AM
Job to convert Java App 1.3.1 to Java Newest of Java Michael Kintner Java 0 11-30-2003 04:42 AM



Advertisments