Perl vs Java for specific task
 

Hello,

I have a rather odd question. My company is an all java/oracle shop.
We do everything is Java... no matter what it is... parsing of text
files, messaging, gui you name it. My question is this... is Perl so
much better at parsing text files and outputing that we would see a
substantial speed increase? We process about 10 million records in
flat files a day for reformatting before putting them in a DB.

Also, when it comes to Unix threading... which one would be better off
Java or perl? Essentially, we would break the 10 million down into 10
files... each file is done in a seperate thread... The program also
has to keep a hashmap of keys to make sure we don;t include duplicate
records and it must connect to oracle every once in a while... is
switching to perl worth it considering the investment and know how we
have in java? This is the only portion of the code we would consider
switching to perl...

ideas?

Thanks,

Joie

Re: Perl vs Java for specific task
 

linux_email2000@yahoo.com (John Smith) wrote in message news:<248a20a2.0408301038.4de5ee36@posting.google. com>...

> I have a rather odd question. My company is an all java/oracle shop.
> We do everything is Java... no matter what it is... parsing of text
> files, messaging, gui you name it. My question is this... is Perl so
> much better at parsing text files and outputing that we would see a
> substantial speed increase? We process about 10 million records in
> flat files a day for reformatting before putting them in a DB.
>
> Also, when it comes to Unix threading... which one would be better off
> Java or perl?

AFIAK most Java implementations have a much better threading
implementation than is found in Perl5. This is becase Java was
designed to be threaded from day-0. In Perl threading is very much an
afterthought.

Up until Perl5.8 Perl's threading was essentially useless and even now
you are usually better using forks than threads on Perl.

> Essentially, we would break the 10 million down into 10
> files... each file is done in a seperate thread... The program also
> has to keep a hashmap of keys to make sure we don;t include duplicate
> records and it must connect to oracle every once in a while... is
> switching to perl worth it considering the investment and know how we
> have in java? This is the only portion of the code we would consider
> switching to perl...

A Perl programmer could possibly write a solution using 10 separate
processes that would outperform a Java solution. But no even as a fan
of Perl I wouldn't say this alone was sufficient justification to
learn Perl.

This newsgroup does not exist (see FAQ). Please do not start threads
here.

Re: Perl vs Java for specific task
 

John Smith wrote:

> Hello,
>
> I have a rather odd question. My company is an all java/oracle shop.
> We do everything is Java... no matter what it is... parsing of text
> files, messaging, gui you name it. My question is this... is Perl so
> much better at parsing text files and outputing that we would see a
> substantial speed increase? We process about 10 million records in
> flat files a day for reformatting before putting them in a DB.

Yes, I'd think perl is that much better, it's designed for parsing text
data. It also uses a runtime model that tends to use higher level
operations, spending less time interpreting the model and more doing
work. As opposed to JRE which uses a low level byte code. Not to
mention that someone who knew the language could generate the parsing
code in perl faster and easier than in java.
>
> Also, when it comes to Unix threading... which one would be better off
> Java or perl? Essentially, we would break the 10 million down into 10
> files... each file is done in a seperate thread... The program also
> has to keep a hashmap of keys to make sure we don;t include duplicate
> records and it must connect to oracle every once in a while... is
> switching to perl worth it considering the investment and know how we
> have in java? This is the only portion of the code we would consider
> switching to perl...
>
> ideas?
>
> Thanks,
>
> Joie

Is the multithreaded design inherant to the problem or part of a java
style solution, as in watching multiple input files? While perl is weak
at multithreading, it is very good at not needing to multithread. An
optimal solution in perl would probably be structured differently than
an optimal solution in java.

Re: Perl vs Java for specific task
 

nobull@mail.com wrote in message news:<4dafc536.0408310040.17b62686@posting.google. com>...
> linux_email2000@yahoo.com (John Smith) wrote in message news:<248a20a2.0408301038.4de5ee36@posting.google. com>...
>
> > I have a rather odd question. My company is an all java/oracle shop.
> > We do everything is Java... no matter what it is... parsing of text
> > files, messaging, gui you name it. My question is this... is Perl so
> > much better at parsing text files and outputing that we would see a
> > substantial speed increase? We process about 10 million records in
> > flat files a day for reformatting before putting them in a DB.
> >
> > Also, when it comes to Unix threading... which one would be better off
> > Java or perl?
>
> AFIAK most Java implementations have a much better threading
> implementation than is found in Perl5. This is becase Java was
> designed to be threaded from day-0. In Perl threading is very much an
> afterthought.
>

Okay, I have heard this before... but what I need to do is this. I
need to parse 4 or 5 delimited text files that have between one and
two million records in them each. Now. I was thinking it would be
best to use perl because then I don't have the overhead of the JVM and
from my past memory perl was ALWAYS the choice for text parsing.
However, as you suggested... perl was not written originally to be
multithreaded.... so will launching > 1 perl program off in seperate
threads be a problem or is it simply that a single multi-threaded perl
application has issues?



> Up until Perl5.8 Perl's threading was essentially useless and even now
> you are usually better using forks than threads on Perl.
>
> > Essentially, we would break the 10 million down into 10
> > files... each file is done in a seperate thread... The program also
> > has to keep a hashmap of keys to make sure we don;t include duplicate
> > records and it must connect to oracle every once in a while... is
> > switching to perl worth it considering the investment and know how we
> > have in java? This is the only portion of the code we would consider
> > switching to perl...
>
> A Perl programmer could possibly write a solution using 10 separate
> processes that would outperform a Java solution. But no even as a fan
> of Perl I wouldn't say this alone was sufficient justification to
> learn Perl.

Well, I will take your statement into account... it seems as though it
might be worth looking into doing it in java as well. Any more tips
or info?



>
> This newsgroup does not exist (see FAQ). Please do not start threads
> here.