Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Perl > log parsing: gaps between transactions

Reply
Thread Tools

log parsing: gaps between transactions

 
 
Arthur Dent
Guest
Posts: n/a
 
      12-02-2003
I have a log file in this format:

YYYYMMDD HHMMSS DATADATADATADATADATA RESULTCODE

and I want to be able to parse the log and gleen information about gaps in
time between transactions. For example, it would be nice to know that 8
times in the past 24 hours we had periods over 5 minutes where no
transactions were successful. In order to do that, I need to compare each
line against the next based on result code and time elapsed. To make this
a bit more tricky, not every line in the log is in this format (approx 45%
are) but I only want to parse the lines that are. To add another problem,
I am an inexperienced perl programmer. If someone can point me in the
right direction it would be most appreciated. I have all of the O'Reilly
books and have referenced them but nothing seems to be close to what I
need. I also did an extensive search on deja before posting.

Thanks,

Arthur
 
Reply With Quote
 
 
 
 
Jim Gibson
Guest
Posts: n/a
 
      12-03-2003
In article <qn8ihvdw8iyp.s3q484ucmkd5$(E-Mail Removed)>, Arthur Dent
<(E-Mail Removed)> wrote:

> I have a log file in this format:
>
> YYYYMMDD HHMMSS DATADATADATADATADATA RESULTCODE
>
> and I want to be able to parse the log and gleen information about gaps in
> time between transactions. For example, it would be nice to know that 8
> times in the past 24 hours we had periods over 5 minutes where no
> transactions were successful. In order to do that, I need to compare each
> line against the next based on result code and time elapsed. To make this
> a bit more tricky, not every line in the log is in this format (approx 45%
> are) but I only want to parse the lines that are. To add another problem,
> I am an inexperienced perl programmer. If someone can point me in the
> right direction it would be most appreciated. I have all of the O'Reilly
> books and have referenced them but nothing seems to be close to what I
> need. I also did an extensive search on deja before posting.
>
> Thanks,
>
> Arthur


Have you tried any code? Do you have access to the perl online
documentation (try "perldoc perl" at a command line)? Check out
'perldoc perlquickre' for regular expressions and 'perldoc perlfaq5'
for file I/O.

Do you know how to read text files? You can identify the transaction
lines and extract the time information with a regular expression such
as

/(\d{8})\s+(\d\d)(\d\d)(\d\d)/

This looks for and saves an 8-digit date, followed by some white space,
followed by 3 sets of 2-digit values. If your times are in 24-hour
format, you can compute a time-since-midnight easily. Here is a sample
program:

#!/opt/perl/bin/perl

use strict;
use warnings;

my $prev = 86400;
while(<DATA>) {
if( /(\d{8})\s+(\d\d)(\d\d)(\d\d)/ ) {
my $date = $1;
my $time = $4 + 60*( $3 + 60*$2);
my $gap = $time - $prev;
if( $gap > 300 ) {
print "date = $date, time = $time, prev = $prev, gap = $gap
seconds\n";
}
$prev = $time;
}
}

__DATA__
line 1
line 2
20031127 123456 data1 code1
line 3
line 4
20031127 123457 data2 code2
line 5
line 6
20031127 123958 data3 code3
line 7
line 8
20031127 124000 data4 code4
line 9
line 10
line 11

__END__


Output:
date = 20031127, time = 45598, prev = 45297, gap = 301 seconds

You will have to modify this program to handle the case around
midnight, but that shouldn't be too hard. If you need more accurate,
absolute time, check out the Time:arseDate module from
http://www.cpan.org/

FYI: this newsgroup is defunct. Try comp.lang.perl.misc in the future,
paying attention to the guidelines for that group, available at
http://mail.augustmail.com/~tadmc/cl...uidelines.html
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Gaps between controls dimstthomas@yahoo.com HTML 1 11-07-2006 02:58 PM
Why gaps in my meny in IE? greg HTML 4 01-29-2006 02:19 AM
Eliminate gaps between table cells... cool2005 Javascript 5 05-20-2005 10:19 PM
XSLT White Space Gaps Under Images Mark247 XML 1 09-02-2004 11:47 PM
Writing audio cd's without gaps??? Rob Mitchell Computer Support 2 06-30-2003 11:39 PM



Advertisments