Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > C++ > Re: benchmarks? java vs .net

Reply
Thread Tools

Re: benchmarks? java vs .net

 
 
Jon Skeet [C# MVP]
Guest
Posts: n/a
 
      06-03-2008
Razii <> wrote:
> $ time partialsums 2500000 (.NET)
> 3.000000000 (2/3)^k
> 3160.817621887 k^-0.5
> 0.999999600 1/k(k+1)
> 30.314541510 Flint Hills
> 42.995233998 Cookson Hills
> 15.309017155 Harmonic
> 1.644933667 Riemann Zeta
> 0.693146981 Alternating Harmonic
> 0.785398063 Gregory
>
> real 0m1.530s
> user 0m0.000s
> sys 0m0.031s


That looks very much like Unix output rather than Windows. Are you
running under cygwin or something similar?

It would really help if you gave a full explanation of how *you* (as
opposed to the shoot-out) site are running your tests.

(I'd also argue that benchmarks taking only a second and a half to
complete aren't likely to have a good startup vs running program
balance.)

--
Jon Skeet - <>
Web site: http://www.pobox.com/~skeet
Blog: http://www.msmvps.com/jon.skeet
C# in Depth: http://csharpindepth.com
 
Reply With Quote
 
 
 
 
Jon Skeet [C# MVP]
Guest
Posts: n/a
 
      06-03-2008
Razii <> wrote:
> >That looks very much like Unix output rather than Windows. Are you
> >running under cygwin or something similar?

>
> Yes, I am running cygwin.


Has it occurred to you that that may have an effect on the results?

The benchmarks would be *much* better in my view if each benchmark
started up a single process and ran with it for a significant amount of
time. After all, that's far more realistic in terms of how almost all
software is actually run in the real world.

--
Jon Skeet - <>
Web site: http://www.pobox.com/~skeet
Blog: http://www.msmvps.com/jon.skeet
C# in Depth: http://csharpindepth.com
 
Reply With Quote
 
 
 
 
Jon Skeet [C# MVP]
Guest
Posts: n/a
 
      06-04-2008
On Jun 3, 11:15 pm, Razii <klgf...@mail.com> wrote:
> <sk...@pobox.com> wrote:
> >Has it occurred to you that that may have an effect on the results?

>
> No, it won't have a major effect.


And you know this because...?

Just claiming that it won't have an effect isn't exactly compelling
evidence.

Jon
 
Reply With Quote
 
Jon Skeet [C# MVP]
Guest
Posts: n/a
 
      06-04-2008
On Jun 4, 5:43 pm, Razii <nikjhlgf...@mail.com> wrote:
> <sk...@pobox.com> wrote:
> >And you know this because...?

>
> The one who asserts must prove.


I absolutely agree. Here's what I wrote:

<quote>Has it occurred to you that that may have an effect on the
results?</quote>

I don't see an assertion there - merely speculation that it *may* have
an effect.

Now here's what you wrote:

<quote>No, it won't have a major effect.</quote>

*That's* an assertion. Therefore by your own instructions, it's up to
you to prove your assertion.

If you're running a program in a non-standard way (and running .NET
under Cygwin certainly isn't the usual environment) then it's up to
you to investigate whether or not that has any effect.

Jon
 
Reply With Quote
 
Jon Skeet [C# MVP]
Guest
Posts: n/a
 
      06-04-2008
Razii <> wrote:
> ><quote>No, it won't have a major effect.</quote>
> >
> >*That's* an assertion. Therefore by your own instructions, it's up to
> >you to prove your assertion.

>
> There is no reason to believe that it "may have" any effect.


There absolutely is. What will be feeding the programs their standard
input? Cygwin. What will be launching the processes? Cygwin.

Will it definitely have a difference? Who knows. I'm not claiming it
definitely will - just that until you've proved that it doesn't, your
tests are invalid when considering how Java and .NET are *normally* run
on Windows.

> Cygwin offers easy way to time with time command. If you can time all
> the becnhmarks as easily on Windows and get different results, let us
> know.


What I *will* do is amend the sumcol test to be sensible in the first
place - to run *once* with a suitably large input. That way you can
easily measure the time taken within the app, discounting the
(massively repeated) startup costs.

--
Jon Skeet - <>
Web site: http://www.pobox.com/~skeet
Blog: http://www.msmvps.com/jon.skeet
C# in Depth: http://csharpindepth.com
 
Reply With Quote
 
Jon Skeet [C# MVP]
Guest
Posts: n/a
 
      06-04-2008
Jon Skeet [C# MVP] <> wrote:
> > Cygwin offers easy way to time with time command. If you can time all
> > the becnhmarks as easily on Windows and get different results, let us
> > know.

>
> What I *will* do is amend the sumcol test to be sensible in the first
> place - to run *once* with a suitably large input. That way you can
> easily measure the time taken within the app, discounting the
> (massively repeated) startup costs.


Okay, done - and it's interesting.

First here's a program to duplicate the original input file:
using System.IO;

class DuplicateSumCol
{
static void Main(string[] args)
{
string text = File.ReadAllText("sumcol-input.txt");
int iterations = int.Parse(args[0]);
File.Delete("sumcol-large.txt");
for (int i=0; i < iterations; i++)
{
File.AppendAllText("sumcol-large.txt", text);
}
}
}

I ran that with an input parameter of 10000 to get a 43MB file, which
appears to be large enough to give interesting results.

Then I refactored both the Java and C# versions to allow input either
from a file or from standard input. The bulk of the time is spent in a
single method, so it shouldn't make much difference - and if *either*
language is significantly hampered by this sort of refactoring, then
frankly it deserves all it gets. Here's the C# version:

using System;
using System.Diagnostics;
using System.IO;

class SumCol
{
static void Main(string[] args)
{
int sum;
Stopwatch sw = Stopwatch.StartNew();
if (args.Length == 0)
{
sum = Parse(Console.In);
}
else
{
using (TextReader reader = File.OpenText(args[0]))
{
sum = Parse(reader);
}
}
long elapsed = sw.ElapsedMilliseconds;
Console.WriteLine("Result: {0}", sum);
Console.WriteLine("Elapsed millis: {0}", elapsed);
}

static int Parse (TextReader reader)
{
int sum = 0;
string line;
while ( (line=reader.ReadLine()) != null)
{
sum += int.Parse(line);
}
return sum;
}
}

And here's the Java version:

import java.io.*;

public class SumCol
{
public static void main(String[] args) throws Exception
{
int sum;
long start = System.currentTimeMillis();

if (args.length==0)
{
sum = parse(System.in);
}
else
{
// For the sake of brevity, don't bother closing...
sum = parse(new BufferedInputStream
(new FileInputStream(args[0])));

}
long end = System.currentTimeMillis();

System.out.println("Result: " + sum);
System.out.println("Elapsed millis: " + (end-start));
}

static int parse(InputStream stream) throws IOException
{
int sum = 0;
StreamTokenizer lineTokenizer = new StreamTokenizer(stream);
while (lineTokenizer.nextToken() != StreamTokenizer.TT_EOF)
{
sum += (int) lineTokenizer.nval;
}
return sum;
}
}

I'm running .NET 3.5, and here's all the Java information:
java version "1.6.0_05"
Java(TM) SE Runtime Environment (build 1.6.0_05-b13)
Java HotSpot(TM) Client VM (build 10.0-b19, mixed mode, sharing)

The C# version was compiled with:
csc -o+ -debug- SumCol.cs

The Java version was compiled with:
javac SumCol.java

I'm running on an Intel Core 2 Duo with 32-bit Vista.

Results (in ms):

SumCol.exe < sumcol-large.txt: 4833
SumCol.exe sumcol-large.txt: 3327
java SumCol < sumcol-large.txt: 3321
java SumCol sumcol-large.txt: 3315


Feel free to suggest changes to how the Java version is run, of course
- and please test on your own machine.

My conclusion: Java and .NET are effectively neck and neck here in
terms of actual *processing*, but .NET's console handling slows it down
significantly. No great loss IMO - I very rarely pipe large input from
the console.

--
Jon Skeet - <>
Web site: http://www.pobox.com/~skeet
Blog: http://www.msmvps.com/jon.skeet
C# in Depth: http://csharpindepth.com
 
Reply With Quote
 
Jon Skeet [C# MVP]
Guest
Posts: n/a
 
      06-04-2008
Razii <> wrote:
> On Wed, 4 Jun 2008 19:05:19 +0100, Jon Skeet [C# MVP]
> <> wrote:
>
> >Will it definitely have a difference? Who knows.

>
> Dowsnload Cygwin
>
> http://www.cygwin.com/
>
> Time it and let us know if there is a difference.


I'd rather time a more *useful* benchmark (one that doesn't involve
starting thousands of processes) - so I've done exactly that.

My previous experiences with Cygwin have been far from great - I'd
rather not install it at all. As I say, if you want to justify running
..NET in a somewhat alien environment (you must admit that it's not
exactly the norm), you should check that it doesn't affect results.

--
Jon Skeet - <>
Web site: http://www.pobox.com/~skeet
Blog: http://www.msmvps.com/jon.skeet
C# in Depth: http://csharpindepth.com
 
Reply With Quote
 
Barry Kelly
Guest
Posts: n/a
 
      06-04-2008
Jon Skeet wrote:

> My previous experiences with Cygwin have been far from great - I'd
> rather not install it at all.


I'm not addressing Razzi, as I don't see his posts anymore, but I do
want to pipe up about Cygwin.

Cygwin works very well as long as you treat it almost like a terminal to
another operating system. Using the command-line utilities in a console
window running cmd.exe is usually a waste of time. Running bash in rxvt,
with an appropriately-configured ~/.bash{rc,_profile} etc. and
~/.Xdefaults (for rxvt settings), and it's a pleasure - so long as you
enjoy the Unix experience and want to replicate it on Windows.

With an appropriately constructed user/profile directory, you can get
almost identical shell experiences across most OSes.

-- Barry

--
http://barrkel.blogspot.com/
 
Reply With Quote
 
Jon Skeet [C# MVP]
Guest
Posts: n/a
 
      06-04-2008
Razii <> wrote:
> On Wed, 04 Jun 2008 22:13:49 +0100, Barry Kelly
> <> wrote:
>
> >I'm not addressing Razzi, as I don't see his posts anymore, but I do
> >want to pipe up about Cygwin.

>
> You definitely see my posts because my email changes with every post.


And it's completely inconceivable to you that a newsreader could filter
just on name, and not on email address?

Forte Agent, the newsreader Barry uses (by his headers, at least) has
precisely this capability. I've just tried it, and verified that it
filters out all of your posts.

Now, of course you *could* change your name as well as your email
address for every post - but there's really no legitimate reason for
doing that, and no point in irritating people just for the sake of it.
Heck, why do you even change email address each time?

--
Jon Skeet - <>
Web site: http://www.pobox.com/~skeet
Blog: http://www.msmvps.com/jon.skeet
C# in Depth: http://csharpindepth.com
 
Reply With Quote
 
kwikius
Guest
Posts: n/a
 
      06-04-2008

"Razii" <> wrote in message
news:...
> On Wed, 04 Jun 2008 22:13:49 +0100, Barry Kelly
> <> wrote:
>
>>I'm not addressing Razzi, as I don't see his posts anymore, but I do
>>want to pipe up about Cygwin.

>
> You definitely see my posts because my email changes with every post.
> Stop lying.


Voila , Ratboy , The definitive troll...

.... hmm... apologies.. I interrupted the battle of the trolls.. Ok...trolls
.... please feel free to continue...

regards
Andy Little


 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Hot Requirements: 1.Sr Java Developer,2.Java Developer (Java with EJB) Isaac Java 0 01-20-2011 08:41 PM
hey i am just started java,, can anyone tell me the use ,application, why java , importance of java.. manish sahu Java 3 02-14-2008 12:00 AM
[JAVA] [EVALUATION] - The Java Failure (Sorry: The Java(tm) Failure) Ilias Lazaridis Java 0 02-01-2005 10:32 AM
JAVA VIRTUAL MUCHINE OR SUN JAVA Fernando Kohan Firefox 1 11-14-2004 02:04 AM
Job to convert Java App 1.3.1 to Java Newest of Java Michael Kintner Java 0 11-30-2003 04:42 AM



Advertisments
 



1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57