«

I have a PERL program that reads from a large text file and does some
processing on every line it reads. If this same program is written in
C or any other compiled language, would I get significant performance
improvement?
I am primarily asking about whether writing a program in Perl can
cause significant performance impact.

Thanks
Yash

Yash wrote:
>
> I have a PERL program that reads from a large text file and does some
> processing on every line it reads. If this same program is written in
> C or any other compiled language, would I get significant performance
> improvement?
> I am primarily asking about whether writing a program in Perl can
> cause significant performance impact.

If it's processing files, the speed will probably depend mainly on the
speed with which you can stuff in the data.

Since perl programs are compiled into an intermediate language which is
the interpreted, the performance impact is small.

--
Josef Möllers (Pinguinpfleger bei FSC)
If failure had no penalty success would not be a prize
-- T. Pratchett

On Fri, 09 Jan 2004 14:45:03 +0100, Josef Möllers wrote:

> Yash wrote:
>>
>> I have a PERL program that reads from a large text file and does some
>> processing on every line it reads. If this same program is written in C
>> or any other compiled language, would I get significant performance
>> improvement?
>> I am primarily asking about whether writing a program in Perl can cause
>> significant performance impact.

Writing a bad program in perl can, but changing language woun't help.

> If it's processing files, the speed will probably depend mainly on the
> speed with which you can stuff in the data.
>
> Since perl programs are compiled into an intermediate language which is
> the interpreted, the performance impact is small.

The speed will depend either on (as you said) disk access (data speed), or
(if the processing is heavy) the efficiency of the algorithm used
(calculation speed).

OP: take a look at your algorithm instead of changing language, unless the
kind of processing you do is typically handled better by a specific
language (read: if there are pre-optimized libraries available to do the
processing you do by hand today).

If you have one part of you algorithm that actually can benefit from
beeing made in C you can make a module in C and call it from a perl
program, so you would only need to port the bits to be optimized. (Look
for "Extending Perl" in your copy of programming perl.)


Why don't you post some code so people could give you suggestions for
improvements?


--
NPV

"the large print giveth, and the small print taketh away"
Tom Waits - Step right up

(Yash) wrote:
> I have a PERL program that reads from a large text file and does some
> processing on every line it reads.

What kind of processing? I have to travel some distance. Will a plane,
car, bicycle, or walking be better?

> If this same program is written in
> C or any other compiled language, would I get significant performance
> improvement?

From my general experience with programs easy to write in Perl, the rule of
thumb is that if the program would have been easy to write in C in the
first place, it will be much faster if written in C. If it would have been
hard to write in C, it won't be that much faster, anyway.

(Once you do all the hard stuff like array boundary checking, add flags for
missing or undef data and add code to check that flag at every step, write
your own hash and parsing functionality, etc., you basically have a
hand-made, bug- ridden Perl knock-off, and it will be about the same speed
or slower than real Perl).

YMMV

> I am primarily asking about whether writing a program in Perl can
> cause significant performance impact.

Yes, it can.

Xho

--
-------------------- http://NewsReader.Com/ --------------------
Usenet Newsgroup Service New Rate! $9.95/Month 50GB

On Fri, 9 Jan 2004 wrote:

> (Yash) wrote:
>
> > I am primarily asking about whether writing a program in Perl can
> > cause significant performance impact.
>
> Yes, it can.

I'm sure your answer is precisely correct: "yes, it *can*".

You don't need this to be pointed out, but perhaps some readers would
be helped by it: I would want to ask more importantly, was it the
right question? That needs to be understood in context. There's
usually little to be gained from software that mostly gets the right
answer fast, if it also sometimes gets the wrong answer, or crashes
horribly. C is a fine weapon for those skilled in the arts of
weaponry, but it's also rather good at shooting the practitioner in
the foot.

Perl is IMHO an excellent language for developing a reliable and
resilient prototype. For many purposes, the end of the propotype
stage is the earliest stage at which considerations of code
performance are appropriate to be considered (rules 1 and 2, and
possibly 3, of optimization are "don't optimize yet").

Many of those Perl prototypes go on to be reliable and resilient
production systems coded in Perl ("don't optimize in the wrong
place"). If and when it becomes evident that the code is too slow (to
the extent that it's uneconomic to solve the problem by throwing some
more CPU at it), that's the time to consider re-implementing. The
work that was put into the prototype won't by any means have been
wasted! And at least you'll have some code to profile, so you can see
where most of the time is being spent (and I have to say that's rarely
obvious ab initio, it becomes clear only when the application has been
prototyped and is actually running).

I don't myself recall a situation where an application was ever turned
from being impractical into feasible merely by recoding in a different
language. I do, however, recall an occasion where a task that needed
3 days computation was re-examined from scratch, and a totally
different algorithm was used, bringing the time down to about 20
minutes. You're not likely to get that scale of savings by mere
choice of coding language: you've got to examine the problem at a
fundamental level.

That's my "take" on the topic, anyhow. We physicists are reputed to
code FORTRAN in any language, by the way ;-}

In article < c.uk>,
Alan J. Flavell <> wrote:
:I don't myself recall a situation where an application was ever turned
:from being impractical into feasible merely by recoding in a different
:language.

Programming for very limited computers. Some languages draw up
large libraries that are not very divisible. If you have very
limited resources, then programming in something like Forth can
end up with a practical program where the C library would not fit;
if you have a few more resources, then C might fit where C++ did not.

--
When your posts are all alone / and a user's on the phone/
there's one place to check -- / Upstream!
When you're in a hurry / and propagation is a worry/
there's a place you can post -- / Upstream!

Alan J. Flavell wrote:

> On Fri, 9 Jan 2004 wrote:
>
> I don't myself recall a situation where an application was ever turned
> from being impractical into feasible merely by recoding in a different
> language. I do, however, recall an occasion where a task that needed
> 3 days computation was re-examined from scratch, and a totally
> different algorithm was used, bringing the time down to about 20
> minutes. You're not likely to get that scale of savings by mere
> choice of coding language: you've got to examine the problem at a
> fundamental level.
>

I've had one: we were trying to write a gui front end to some perl
command line tools. We thought writing the gui in perl would make it
easier as we'd be throwing the same data structures around the whole
time. But we found that it was slow and very very memory hungry,
abandoned it and wrote in C++, using sockets to call (a modified version
of) the command line tools.

Perl is very good at text processing, which is (presumably) what the OP
wants, so it's unlikely that another language would get significantly
better. But where something has been bolted on to perl, it may not join
very well - this was the case in our situation.

(For reference, this was using wxPerl bindings to the wxWindows C++
toolkit, then using the wxWindows toolkit directly in C++. wxPerl was I
think relativley new and being improved all the time, so may be better
by now, but at the time wasn't good enough).

Stuart

--
Quiquid latine dictum sit, altum viditur

On Fri, 09 Jan 2004 05:37:14 -0800, Yash wrote:
> I have a PERL program that reads from a large text file and does some
> processing on every line it reads. If this same program is written in C
> or any other compiled language, would I get significant performance
> improvement?

Normally, no. It all depends on your solution; If the algorithm which
takes care of the processing of each line stinks in Perl, it probably will
in any other languages too.

Consider this simple code:

while ( <DATA> ) {
chomp;
print $_ if ( /something/ );
}

I doubt that you will be able to write a similar program with any other
language that is _much_ faster. The time you'll eventually save will get
lost in a harder implementation.

> I am primarily asking about whether writing a program in Perl can cause
> significant performance impact.

Every language _can_ cause significant performance impact. If you don't
know how to utilize a language best possible, you _can_ end up with a
solution with a solution below par.

If you have an existing script written in Perl, I think that most people
here would be glad to review it for you and factor out any performance
issues. If you _still_ think that your Perl script is too slow, then you
should consider other solution (not necessarily being better).

--
Tore Aursand <>
"The purpose of all war is ultimately peace." -- Saint Augustine

In article <>,
Tore Aursand <> wrote:

:Consider this simple code:

: while ( <DATA> ) {
: chomp;
: print $_ if ( /something/ );
: }

Useful code ifyou need allthe lines to runtogether when printed out.
--
millihamlet: the average coherency of prose created by a single monkey
typing randomly on a keyboard. Usenet postings may be rated in mHl.
-- Walter Roberson

On Sat, 10 Jan 2004 04:19:18 +0000, Walter Roberson wrote:
>> Consider this simple code:
>>
>> while ( <DATA> ) {
>> chomp;
>> print $_ if ( /something/ );
>> }

> Useful code ifyou need allthe lines to runtogether when printed out.

Except that it doesn't _necessarily_ print all the lines out together. If
I wanted a script to do that, I would have used something like this;

print (chomp && $_) while ( <DATA> );

Thus my first could easily have been shortened to this:

print (/something/ && $_) while ( <DATA> );


--
Tore Aursand <>
"I am become Death, shatterer of worlds." -- J. Robert Oppenheimer,
upon witnessing the explosion of the first atomic bomb.