«

On Sun, 26 Jun 2011 23:33:44 -0700, markspace <-@.> wrote:

>On 6/26/2011 8:39 PM, Gene Wirchenko wrote:
>
>> You are misunderstanding. The test code that I am referring to
>> is proof-of-concept code to test ideas, *not* my test cases.
>
>I don't see what you are doing then. Is the code you showed us not your
>"test/proof of concept?"

It is.

>First, a proof of concept is a little silly for a parser. Of course
>it's feasible. And then you are running timing test on it. Shouldn't
>you be doing time tests on something you intend to throw away? Profile
>the real code!

I wanted to know which way to jump on character identification. I
figured that squential would be bad though it is not as bad as I
thought. I did not know which of a binary search on a String or a
Treeset search would be faster. That was why I wrote the test.

>You seem to be making a lot of extra work for yourself, or at least
>confusing the heck out of us.

I am an experienced programmer, but I am not so experienced with
Java. I am trying to remedy the latter.

On the confusion, I see that a number of people have
micro-optimised for my code. I am looking at a bigger picture.

>> But I do not want to do that. I am writing a preprocessor to
>> process files like:
>
>You should be using input like what you showed to test then. The test
>code you showed us won't parse this correctly.

Of course not. The test code is just for dealing with
identifiers, and it is a simplified version to boot.

Sincerely,

Gene Wirchenko

In article <>,
Gene Wirchenko <> wrote:
> On 23 Jun 2011 23:11:50 GMT, (Stefan Ram)
> wrote:
>
> >Gene Wirchenko <> writes:
> >>So how would you have written this benchmark?
> >
> >http://stackoverflow.com/questions/5...chmark-in-java
> >http://www.kdgregory.com/index.php?p...microBenchmark
> >http://www.ibm.com/developerworks/ja...225/index.html
> >...
>
> Good links. Thank you.
>
> But my question was really about method calling to
> SequentialSearch(), BinarySearch(), and TreesetSearch().
>

Interesting how the discussion has veered off onto other subjects --
but hey, this is Usenet!

But you did get at least two replies focusing on reducing the
amount of code duplication, one from markspace and one from me[*].
It would be nice to know whether you found them useful. ?
[*] Message-ID: <>

--
B. L. Massingill
ObDisclaimer: I don't speak for my employers; they return the favor.

On 6/27/2011 1:53 PM, Gene Wirchenko wrote:
>
> I am an experienced programmer, but I am not so experienced with
> Java. I am trying to remedy the latter.


Honestly, what you've done so far is pretty crazy. I figured you were a
2nd year student who got in over his head on a personal project, or a
homework problem. No sane programmer would try to search for characters
the way you are.

If you want to tell us what your actual experience is, it might help.
I'm guessing your experience isn't actually programming, maybe HTML and
Flash or something. But I don't want to get into an argument here so if
you don't want to tell us then it's ok to drop it.


> On the confusion, I see that a number of people have
> micro-optimised for my code. I am looking at a bigger picture.


It's hard to give you a big picture. Well, a couple have tried. JavaCC
or parboiled or similar compiler-compiler would help you the most. But
besides that you've given us only a very micro example. We can't do
anything else with the code other than micro-optimize.

And on my system using a StringBuilder instead of '+' yields a 200%
speed up. Times go from 22 seconds to around 6. I don't really call
that micro.


>
> Of course not. The test code is just for dealing with
> identifiers, and it is a simplified version to boot.


Part of the problem is that you aren't really testing those three search
routines. You're testing other things like string concatenation,
because those are dominating the running time of your tests.

I'd start over. Download an IDE like NetBeans. Start a new project
with a single class with just the search routines (shown below). Use
Tools -> Create Unit Tests. That will at least generate a saner
framework for you to put your testing. It does need testing, and you
can make a couple of tests do timing for you. I think it'll help you
think about the problem more clearly too, things were kinda messy in there.

class TimingTesting
{

static String cParseString =
"//identifier//IDENTIFIER//a_b_c "
+"abc1234b5%$__dbl;one;two;three;END";
static String IdentChars =
"0123456789" + "ABCDEFGHIJKLMNOPQRSTUVWXYZ" + "_"
+ "abcdefghijklmnopqrstuvwxyz"; // sorted order!
static SortedSet<Character> IdentCharsSet
= new TreeSet<Character>();
static int nRepetitions = 1000000;


// Just these three methods and no more!!!

static boolean SequentialSearch(
char CurrChar )
{
boolean fFound = false;
for( int i = 0; i < IdentChars.length() && !fFound; i++ ) {
fFound = IdentChars.charAt( i ) == CurrChar;
}
return fFound;
}

static boolean BinarySearch(
char CurrChar )
{
int xLow = 0;
int xHigh = IdentChars.length() - 1;
int xTry;
boolean fFound = false;
while( xLow <= xHigh ) {
xTry = ( xLow + xHigh ) / 2;
if( CurrChar == IdentChars.charAt( xTry ) ) {
return true;
}
if( CurrChar < IdentChars.charAt( xTry ) ) {
xHigh = xTry - 1;
} else {
xLow = xTry + 1;
}
}
return false;
}

static boolean TreesetSearch(
char CurrChar )
{
return IdentCharsSet.contains( CurrChar );
}

}

On Mon, 27 Jun 2011 18:03:42 -0700, markspace <-@.> wrote:

>On 6/27/2011 1:53 PM, Gene Wirchenko wrote:
>>
>> I am an experienced programmer, but I am not so experienced with
>> Java. I am trying to remedy the latter.

>Honestly, what you've done so far is pretty crazy. I figured you were a
>2nd year student who got in over his head on a personal project, or a
>homework problem. No sane programmer would try to search for characters
>the way you are.

You do not know my requirements, but you claim I am not sane?

<plonk>

[snip]

Sincerely,

Gene Wirchenko

In article <>,
Gene Wirchenko <> wrote:
> On Thu, 23 Jun 2011 17:24:43 -0700, markspace <-@.> wrote:
>
> >On 6/23/2011 4:03 PM, Gene Wirchenko wrote:
> >>
> >> So how would you have written this benchmark?
>
> >Um, realistically? Is this really what you want to do?
> >
> > static boolean TreesetSearch( char CurrChar ) {
> > return IdentCharsSet.contains( CurrChar );
> > }
>
> Yes. I wanted a simple method call in the parser so I could
> cut-and-paste. I did not know if I would need more than one call. I
> am going to go with a Treeset so I will not have a separate method in
> the implementation.

Another "no separate method" approach would be to use the String
class's indexOf method:

static boolean StringLibSearch
(
char CurrChar
)
{
return IdentChars.indexOf(CurrChar) >= 0;
}

I added this to your benchmark suite and found it to give performance
comparable to the TreeSet implementation (indeed, usually it was a
bit faster). The overhead of building the TreeSet probably doesn't
matter in the grand scheme of things, and probably it also doesn't
matter a lot that every call to the TreeSet's "contains" method
(AFAIK) has to convert a character primitive to a Character object,
but -- <shrug>.

But if you're going to use a Set, why a TreeSet? As best I can tell,
you don't use/need the sorted-ness it provides. Just out of curiosity,
I also added to your benchmark suite something that declares the set
as a Set and creates it as an instance of HashSet, and the resulting
code was noticeably faster than any of the other alternatives.


And finally, I wondered how all of these methods compared to
something using regular expressions (the java.util.regex classes), so
i tried that too, replacing your whole parse code with the following:

import java.util.regex.*;

// ....

static Pattern IdentRegexPattern=Pattern.compile("[" + IdentChars + "]+");

// ....

// code to be called repeatedly from timing loop
static void ParseRegex()
{
Matcher IdentMatcher = IdentRegexPattern.matcher(cParseString);
String sIdent;
while (IdentMatcher.find())
{
sIdent = IdentMatcher.group();
if (nRepetitions==1)
System.out.println(sIdent);
}
}

// ....

This was a clear winner (with regard to performance) on the system
where I measured performance, *unless* I ran the tests with the
"-server" flag, in which case it took second place, behind the
HashSet-based approach. As I understand things, though, the
"-server" flag results in the compiler doing more to try to optimize
the code, including being more aggressive about eliminating dead
code, so I'm not entirely confident about the results I'm getting
being meaningful.

(Probably your actual code needs to do something other than
finding and printing identifiers, so the above code would need
some adjustment. Still, if you like regular expressions, it's
another possibility, maybe .... )

[ snip ]

--
B. L. Massingill
ObDisclaimer: I don't speak for my employers; they return the favor.

On Jul 3, 8:26*pm, blm...@myrealbox.com <blmblm.myreal...@gmail.com>
wrote:
> In article <jcu707d1fb592i91m40fsjtfg0784kn...@4ax.com>,
> Gene Wirchenko *<ge...@ocis.net> wrote:
>
>
>
>
>
>
>
>
>
> > On Thu, 23 Jun 2011 17:24:43 -0700, markspace <-@.> wrote:
>
> > >On 6/23/2011 4:03 PM, Gene Wirchenko wrote:
>
> > >> * * * So how would you have written this benchmark?
>
> > >Um, realistically? *Is this really what you want to do?
>
> > > *static boolean TreesetSearch( char CurrChar ) *{
> > > * * * * return IdentCharsSet.contains( CurrChar );
> > > * * }
>
> > * * *Yes. *I wanted a simple method call in the parser so I could
> > cut-and-paste. *I did not know if I would need more than one call. *I
> > am going to go with a Treeset so I will not have a separate method in
> > the implementation.
>
> Another "no separate method" approach would be to use the String
> class's indexOf method:
>
> * * static boolean StringLibSearch
> * * * * (
> * * * * *char CurrChar
> * * * * )
> * * * * {
> * * * * * * return IdentChars.indexOf(CurrChar) >= 0;
> * * * * }
>
> I added this to your benchmark suite and found it to give performance
> comparable to the TreeSet implementation (indeed, usually it was a
> bit faster). *The overhead of building the TreeSet probably doesn't
> matter in the grand scheme of things, and probably it also doesn't
> matter a lot that every call to the TreeSet's "contains" method
> (AFAIK) has to convert a character primitive to a Character object,
> but -- <shrug>.
>
> But if you're going to use a Set, why a TreeSet? *As best I can tell,
> you don't use/need the sorted-ness it provides. *Just out of curiosity,
> I also added to your benchmark suite something that declares the set
> as a Set and creates it as an instance of HashSet, and the resulting
> code was noticeably faster than any of the other alternatives.
>
> And finally, I wondered how all of these methods compared to
> something using regular expressions (the java.util.regex classes), so
> i tried that too, replacing your whole parse code with the following:
>
> * * import java.util.regex.*;
>
> * * // ....
>
> * * static Pattern IdentRegexPattern=Pattern.compile("[" + IdentChars + "]+");
>
> * * // ....
>
> * * // code to be called repeatedly from timing loop
> * * static void ParseRegex()
> * * {
> * * * * Matcher IdentMatcher = IdentRegexPattern.matcher(cParseString);
> * * * * String sIdent;
> * * * * while (IdentMatcher.find())
> * * * * {
> * * * * * * sIdent = IdentMatcher.group();
> * * * * * * if (nRepetitions==1)
> * * * * * * * * System.out.println(sIdent);
> * * * * }
> * * }
>
> * * // ....
>
> This was a clear winner (with regard to performance) on the system
> where I measured performance, *unless* I ran the tests with the
> "-server" flag, in which case it took second place, behind the
> HashSet-based approach. *As I understand things, though, the
> "-server" flag results in the compiler doing more to try to optimize
> the code, including being more aggressive about eliminating dead
> code, so I'm not entirely confident about the results I'm getting
> being meaningful.
>
> (Probably your actual code needs to do something other than
> finding and printing identifiers, so the above code would need
> some adjustment. *Still, if you like regular expressions, it's
> another possibility, maybe .... )
>
> [ snip ]
>

Your points are excellent, but the ongoing violations of the naming
conventions is making my brain hurt. Can't we please revert to
conformant names in our replies at least?

--
Lew

In article <20c411d9-8955-4f4c-9a78->,
lewbloch <> wrote:
> On Jul 3, 8:26 pm, blm...@myrealbox.com <blmblm.myreal...@gmail.com>
> wrote:
> > In article <jcu707d1fb592i91m40fsjtfg0784kn...@4ax.com>,
> > Gene Wirchenko <ge...@ocis.net> wrote:

[ snip ]

> > Another "no separate method" approach would be to use the String
> > class's indexOf method:
> >
> > static boolean StringLibSearch
> > (
> > char CurrChar
> > )
> > {
> > return IdentChars.indexOf(CurrChar) >= 0;
> > }
> >
> > I added this to your benchmark suite and found it to give performance
> > comparable to the TreeSet implementation (indeed, usually it was a
> > bit faster). The overhead of building the TreeSet probably doesn't
> > matter in the grand scheme of things, and probably it also doesn't
> > matter a lot that every call to the TreeSet's "contains" method
> > (AFAIK) has to convert a character primitive to a Character object,
> > but -- <shrug>.
> >
> > But if you're going to use a Set, why a TreeSet? As best I can tell,
> > you don't use/need the sorted-ness it provides. Just out of curiosity,
> > I also added to your benchmark suite something that declares the set
> > as a Set and creates it as an instance of HashSet, and the resulting
> > code was noticeably faster than any of the other alternatives.
> >
> > And finally, I wondered how all of these methods compared to
> > something using regular expressions (the java.util.regex classes), so
> > i tried that too, replacing your whole parse code with the following:
> >
> > import java.util.regex.*;
> >
> > // ....
> >
> > static Pattern IdentRegexPattern=Pattern.compile("[" + IdentChars + "]+");
> >
> > // ....
> >
> > // code to be called repeatedly from timing loop
> > static void ParseRegex()
> > {
> > Matcher IdentMatcher = IdentRegexPattern.matcher(cParseString);
> > String sIdent;
> > while (IdentMatcher.find())
> > {
> > sIdent = IdentMatcher.group();
> > if (nRepetitions==1)
> > System.out.println(sIdent);
> > }
> > }
> >
> > // ....
> >
> > This was a clear winner (with regard to performance) on the system
> > where I measured performance, *unless* I ran the tests with the
> > "-server" flag, in which case it took second place, behind the
> > HashSet-based approach. As I understand things, though, the
> > "-server" flag results in the compiler doing more to try to optimize
> > the code, including being more aggressive about eliminating dead
> > code, so I'm not entirely confident about the results I'm getting
> > being meaningful.
> >
> > (Probably your actual code needs to do something other than
> > finding and printing identifiers, so the above code would need
> > some adjustment. Still, if you like regular expressions, it's
> > another possibility, maybe .... )
> >
> > [ snip ]
> >
>
> Your points are excellent, but the ongoing violations of the naming
> conventions is making my brain hurt. Can't we please revert to
> conformant names in our replies at least?


Well .... I guess I figure it's a choice between two things that
seem desirable -- (1) working in with the conventions of the code
I'm modifying and (2) applying the conventions used by most Java
programmers. I chose the former, though it rather makes my brain
hurt as well. ?


--
B. L. Massingill
ObDisclaimer: I don't speak for my employers; they return the favor.

On 6/27/2011 4:12 PM, Gene Wirchenko wrote:
> On Mon, 27 Jun 2011 03:04:11 -0400, Joshua Cranmer
> <> wrote:
>
>> On 06/26/2011 11:42 PM, Gene Wirchenko wrote:
>>> Think simple version of the C preprocessor.
>>
>> Then why not use the C preprocessor?
>
> I could not find one that would run standalone on my system.

Almost all C compiler has a way to do only preprocessing.

GCC, MS, DEC/CPQ/HP etc. has.

Arne

On Thu, 21 Jul 2011 20:43:10 -0400, Arne Vajhøj <>
wrote:

>On 6/27/2011 4:12 PM, Gene Wirchenko wrote:
>> On Mon, 27 Jun 2011 03:04:11 -0400, Joshua Cranmer
>> <> wrote:
>>
>>> On 06/26/2011 11:42 PM, Gene Wirchenko wrote:
>>>> Think simple version of the C preprocessor.
>>>
>>> Then why not use the C preprocessor?
>>
>> I could not find one that would run standalone on my system.
>
>Almost all C compiler has a way to do only preprocessing.
>
>GCC, MS, DEC/CPQ/HP etc. has.

I did not want a C compiler. I simply wanted a preprocessor.

Sincerely,

Gene Wirchenko

On Jul 22, 12:35*pm, Gene Wirchenko <ge...@ocis.net> wrote:
> On Thu, 21 Jul 2011 20:43:10 -0400, Arne Vajhøj <a...@vajhoej.dk>
> wrote:
>
> >On 6/27/2011 4:12 PM, Gene Wirchenko wrote:
> >> On Mon, 27 Jun 2011 03:04:11 -0400, Joshua Cranmer
> >> <Pidgeo...@verizon.invalid> *wrote:
>
> >>> On 06/26/2011 11:42 PM, Gene Wirchenko wrote:
> >>>> * * * *Think simple version of the C preprocessor.
>
> >>> Then why not use the C preprocessor?
>
> >> * * * I could not find one that would run standalone on my system.
>
> >Almost all C compiler has a way to do only preprocessing.
>
> >GCC, MS, DEC/CPQ/HP etc. has.
>
> * * *I did not want a C compiler. *I simply wanted a preprocessor..

Awwwww. You get a free C compiler that gives you the preprocessor you
want, and you're complaining that it gives you more than you want?
Wow. That's pretty petulant.

--
Lew