Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Perl > Help for a new Perl User - Looking for suggestion

Reply
Thread Tools

Help for a new Perl User - Looking for suggestion

 
 
scadav
Guest
Posts: n/a
 
      07-02-2004
I am new to Perl and I am trying for figure out how to solve this
problem. If anyone can give me some suggestions, I would greatly
appreciate it.

I am trying to read a log file and generate some statistics from it.
For simplicity purposes (only) I have edited some of my logs and code.

Below is an example of a log file which has 3 columns (separated by
commas). The first column contains a time stamp, the second column
contains a numeric identifier and the last column contains a NYSE
Symbol:


10:00,123,KO
10:00,124,KO
10:00,123,KO
10:00,123,KO
10:00,125,T
10:00,125,T
10:20,123,KO
10:20,123,KO
10:20,126,YY
10:20,123,KO
10:20,129,PP
10:40,145,YY
10:40,147,MM
11:00,123,KO
11:00,124,KO
11:00,123,KO
11:00,123,KO
11:00,125,T
11:00,125,T
11:20,123,KO

I am trying to determine at each time interval, how many times the
numeric identifier appears. For example, I would like my output to
look something like this:

TIME NUMERIC IDENTIFIER OCCURENCES SYMBOL
10:00 123 3 KO
10:00 124 1 KO
10:00 125 2 TO
10:20 123 3 KO
10:20 126 1 YY
10:20 129 1 PP
10:40 145 1 YY
10:40 147 1 MM
11:00 123 3 KO
11:00 125 2 KO
11:00 124 1 KO
11:20 123 1 KO

Please keep in mind that my log file contains roughly 70,000 rows of
data


I have been working on this some time and I am ABLE to determine the
total number of messages in the time period (see below), but I am
UNABLE to further break it out by numeric identifier. Can anyone
recommend how I would do this? Below is how far I have gotten in the
code so far:



SAMPLE OF CODE:

$samplelog = "test2.log";
open (IN, "$samplelog");

while ($rcd = <IN>){
@fields = split(/,/,$rcd);
$time{$fields[0]}++;

}

foreach $key (sort keys(%time)) {
print "$key $time{$key} \n";
}



Thank you
 
Reply With Quote
 
 
 
 
Jim Gibson
Guest
Posts: n/a
 
      07-02-2004
In article <(E-Mail Removed)> , scadav
<(E-Mail Removed)> wrote:

> I am new to Perl and I am trying for figure out how to solve this
> problem. If anyone can give me some suggestions, I would greatly
> appreciate it.
>
> I am trying to read a log file and generate some statistics from it.
> For simplicity purposes (only) I have edited some of my logs and code.
>
> Below is an example of a log file which has 3 columns (separated by
> commas). The first column contains a time stamp, the second column
> contains a numeric identifier and the last column contains a NYSE
> Symbol:
>

[data snipped (see below program, below)]

> I am trying to determine at each time interval, how many times the
> numeric identifier appears. For example, I would like my output to
> look something like this:
>
> TIME NUMERIC IDENTIFIER OCCURENCES SYMBOL
> 10:00 123 3 KO
> 10:00 124 1 KO
> 10:00 125 2 TO
> 10:20 123 3 KO
> 10:20 126 1 YY
> 10:20 129 1 PP
> 10:40 145 1 YY
> 10:40 147 1 MM
> 11:00 123 3 KO
> 11:00 125 2 KO
> 11:00 124 1 KO
> 11:20 123 1 KO
>
> Please keep in mind that my log file contains roughly 70,000 rows of
> data
>
>
> I have been working on this some time and I am ABLE to determine the
> total number of messages in the time period (see below), but I am
> UNABLE to further break it out by numeric identifier. Can anyone
> recommend how I would do this? Below is how far I have gotten in the
> code so far:
>
>
>
> SAMPLE OF CODE:
>
> $samplelog = "test2.log";
> open (IN, "$samplelog");
>
> while ($rcd = <IN>){
> @fields = split(/,/,$rcd);
> $time{$fields[0]}++;
>
> }
>
> foreach $key (sort keys(%time)) {
> print "$key $time{$key} \n";
> }


Use the entire record as your key. Use chomp first to remove the
newline at the end. Then split apart the record, which is also the key
to the hash, to do the printing:

Jim 48% cat scadav.pl
#!/usr/local/bin/perl

use strict;
use warnings;

my @fields;
my %time;

while (my $rcd = <DATA>){
chomp($rcd);
$time{$rcd}++;
}

foreach my $entry (sort keys(%time)) {
my($tim,$num,$id) = split(/,/,$entry);
printf " %5s %3d %3d %s\n", $tim, $num, $time{$entry}, $id;
}
__END__
10:00,123,KO
10:00,124,KO
10:00,123,KO
10:00,123,KO
10:00,125,T
10:00,125,T
10:20,123,KO
10:20,123,KO
10:20,126,YY
10:20,123,KO
10:20,129,PP
10:40,145,YY
10:40,147,MM
11:00,123,KO
11:00,124,KO
11:00,123,KO
11:00,123,KO
11:00,125,T
11:00,125,T
11:20,123,KO

Jim 49% ./scadav.pl
10:00 123 3 KO
10:00 124 1 KO
10:00 125 2 T
10:20 123 3 KO
10:20 126 1 YY
10:20 129 1 PP
10:40 145 1 YY
10:40 147 1 MM
11:00 123 3 KO
11:00 124 1 KO
11:00 125 2 T
11:20 123 1 KO
Jim 50%

If you wish to break down your data in different ways or sort, then you
need to consider more complicated solutions such as defining a
hash-of-hashes or a hash-of-arrays to store your multi-level data.

Some more pointers for those new to Perl:

1. Always put 'use strict' and 'use warnings' at the beginning of your
program, then declare all of your variables with 'my' or 'our' or use a
package name for globals.

2. Always check the results of an open call (and all other system
calls, as well):

open (IN, $samplelog) or die("Can't open $samplelog: $!";

3. There is no need to put double-quotes around $samplelog in the above.

4. Post further miscellaneous Perl questions to comp.lang.perl.misc, as
this newsgroup is defunct, but be sure and check the guidelines for
that newsgroup before doing so:

http://mail.augustmail.com/~tadmc/cl...uidelines.html
 
Reply With Quote
 
 
 
 
scadav
Guest
Posts: n/a
 
      07-03-2004
Jim Gibson <(E-Mail Removed)> wrote in
news:020720041255540732%(E-Mail Removed):

> In article <(E-Mail Removed)> , scadav
> <(E-Mail Removed)> wrote:
>
>> I am new to Perl and I am trying for figure out how to solve this
>> problem. If anyone can give me some suggestions, I would greatly
>> appreciate it.
>>
>> I am trying to read a log file and generate some statistics from it.
>> For simplicity purposes (only) I have edited some of my logs and
>> code.
>>
>> Below is an example of a log file which has 3 columns (separated by
>> commas). The first column contains a time stamp, the second column
>> contains a numeric identifier and the last column contains a NYSE
>> Symbol:
>>

> [data snipped (see below program, below)]
>
>> I am trying to determine at each time interval, how many times the
>> numeric identifier appears. For example, I would like my output to
>> look something like this:
>>
>> TIME NUMERIC IDENTIFIER OCCURENCES SYMBOL
>> 10:00 123 3 KO
>> 10:00 124 1 KO
>> 10:00 125 2 TO
>> 10:20 123 3 KO
>> 10:20 126 1 YY
>> 10:20 129 1 PP
>> 10:40 145 1 YY
>> 10:40 147 1 MM
>> 11:00 123 3 KO
>> 11:00 125 2 KO
>> 11:00 124 1 KO
>> 11:20 123 1 KO
>>
>> Please keep in mind that my log file contains roughly 70,000 rows of
>> data
>>
>>
>> I have been working on this some time and I am ABLE to determine the
>> total number of messages in the time period (see below), but I am
>> UNABLE to further break it out by numeric identifier. Can anyone
>> recommend how I would do this? Below is how far I have gotten in the
>> code so far:
>>
>>
>>
>> SAMPLE OF CODE:
>>
>> $samplelog = "test2.log";
>> open (IN, "$samplelog");
>>
>> while ($rcd = <IN>){
>> @fields = split(/,/,$rcd);
>> $time{$fields[0]}++;
>>
>> }
>>
>> foreach $key (sort keys(%time)) {
>> print "$key $time{$key} \n";
>> }

>
> Use the entire record as your key. Use chomp first to remove the
> newline at the end. Then split apart the record, which is also the key
> to the hash, to do the printing:
>
> Jim 48% cat scadav.pl
> #!/usr/local/bin/perl
>
> use strict;
> use warnings;
>
> my @fields;
> my %time;
>
> while (my $rcd = <DATA>){
> chomp($rcd);
> $time{$rcd}++;
> }
>
> foreach my $entry (sort keys(%time)) {
> my($tim,$num,$id) = split(/,/,$entry);
> printf " %5s %3d %3d %s\n", $tim, $num, $time{$entry}, $id;
> }
> __END__
> 10:00,123,KO
> 10:00,124,KO
> 10:00,123,KO
> 10:00,123,KO
> 10:00,125,T
> 10:00,125,T
> 10:20,123,KO
> 10:20,123,KO
> 10:20,126,YY
> 10:20,123,KO
> 10:20,129,PP
> 10:40,145,YY
> 10:40,147,MM
> 11:00,123,KO
> 11:00,124,KO
> 11:00,123,KO
> 11:00,123,KO
> 11:00,125,T
> 11:00,125,T
> 11:20,123,KO
>
> Jim 49% ./scadav.pl
> 10:00 123 3 KO
> 10:00 124 1 KO
> 10:00 125 2 T
> 10:20 123 3 KO
> 10:20 126 1 YY
> 10:20 129 1 PP
> 10:40 145 1 YY
> 10:40 147 1 MM
> 11:00 123 3 KO
> 11:00 124 1 KO
> 11:00 125 2 T
> 11:20 123 1 KO
> Jim 50%
>
> If you wish to break down your data in different ways or sort, then
> you need to consider more complicated solutions such as defining a
> hash-of-hashes or a hash-of-arrays to store your multi-level data.
>
> Some more pointers for those new to Perl:
>
> 1. Always put 'use strict' and 'use warnings' at the beginning of your
> program, then declare all of your variables with 'my' or 'our' or use
> a package name for globals.
>
> 2. Always check the results of an open call (and all other system
> calls, as well):
>
> open (IN, $samplelog) or die("Can't open $samplelog: $!";
>
> 3. There is no need to put double-quotes around $samplelog in the
> above.
>
> 4. Post further miscellaneous Perl questions to comp.lang.perl.misc,
> as this newsgroup is defunct, but be sure and check the guidelines for
> that newsgroup before doing so:
>
> http://mail.augustmail.com/~tadmc/cl...uidelines.html
>


Thank you for your assistance and suggestions it is much appreciated.
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Looking for pointers/suggestion - how to make a webbrowser with these restriction? Anthony Kong Python 4 09-26-2012 02:04 PM
Looking for suggestion on Digital Camera to Capture ID photo harishashim@gmail.com Digital Photography 7 06-15-2006 12:29 AM
Looking for a megapixel suggestion for a 5' by 6' B&W picture hectorlas@yahoo.com Digital Photography 7 01-31-2006 12:00 AM
help: need suggestion on an IDE for learning perl... Perl Misc 3 10-14-2003 04:45 AM
Perl Help - Windows Perl script accessing a Unix perl Script dpackwood Perl 3 09-30-2003 02:56 AM



Advertisments