Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Perl > Perl Misc > Scanning Multiple Log files for patterns continously

Reply
Thread Tools

Scanning Multiple Log files for patterns continously

 
 
tambekp@gmail.com
Guest
Posts: n/a
 
      08-03-2006
Hello,

I want to scan multiple log files for multiple patterns continously. I
am limited to the default perl modules (cant install any extra
modules). Here is what I am attempting to do. Any guidance, pointers to
achieve this are highly appreciated.

- I want to make a hash of patterns and files they need to be searched
against and then every pattern has a set threshold. If the threshold
exceeds in any of the files the script should alert. I dont want to
pass patterns/files as arguments but want to put them in a config file
or hash

- I want to have the script run in an infinite loop so that it scans
the files from the position it left the last scan.

- I have tried doing a hash of patterns=>filenames but this fails (and
rightly so) if same pattern has to be searched in different files as
the key of the hash does not remain unique in that case.

Any suggestions on how to go about implementing this?

Thanks in advance for all the help
-k

 
Reply With Quote
 
 
 
 
tambekp@gmail.com
Guest
Posts: n/a
 
      08-03-2006
Thanks for the pointers Sinan.

The readings you suggested should be a good start for me.

-k
A. Sinan Unur wrote:
> http://www.velocityreviews.com/forums/(E-Mail Removed) wrote in news:1154643704.384937.101190
> @s13g2000cwa.googlegroups.com:
>
> > - I want to make a hash of patterns and files they need to be searched
> > against and then every pattern has a set threshold. If the threshold
> > exceeds in any of the files the script should alert. I dont want to
> > pass patterns/files as arguments but want to put them in a config file
> > or hash
> >
> > - I want to have the script run in an infinite loop so that it scans
> > the files from the position it left the last scan.

>
> The first task is to write this for scanning only one file at a time.
> First off, I do not think keeping the file open is a good idea: The file
> may actually be deleted and re-created while your script is running (I
> don't think this is possible on Windows, but AFAIK, it is allowed on
> various *nix flavored OS). In that case, you would be scanning the wrong
> file.
>
> Instead, check out the discussion in
>
> perldoc -f seek
>
> > - I have tried doing a hash of patterns=>filenames but this fails (and
> > rightly so) if same pattern has to be searched in different files as
> > the key of the hash does not remain unique in that case.

>
> I am not sure what you are talking about here. Hash keys are always
> strings, and cannot be patterns.
>
> The most obvious structure would be:
>
> my %patterns_by_file = (
> log1 => [ qr/^ERROR/, qr/^WARNING/ ],
> log2 => [ qr/^INFO/, qr/^ALERT/ ],
> # etc
> );
>
> > Any suggestions on how to go about implementing this?

>
> Get the one file version working first. You might want to use or refer to
> File::Tail:
>
> http://search.cpan.org/~mgrabnar/Fil...0.99.3/Tail.pm for help.
>
> Then, possibly use Parallel::ForkManager to run multiple scanners
> simultaneously:
>
> http://search.cpan.org/~dlux/Paralle...ForkManager.pm
>
> Now, it is your turn to attack the problem, fill in the blanks, and come
> up with some code.
>
> In the mean time, please read the posting guidelines (esp. this section on
> posting code).
>
> Sinan
>
> --
> A. Sinan Unur <(E-Mail Removed)>
> (remove .invalid and reverse each component for email address)
>
> comp.lang.perl.misc guidelines on the WWW:
> http://augustmail.com/~tadmc/clpmisc...uidelines.html


 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
would this thread loop continously Junkone Ruby 1 01-02-2008 12:56 AM
getting XMLHttpRequest to loop continously to check for changes in an xml file libsfan01 Javascript 4 12-18-2006 06:13 PM
Minimizing Moire patterns when scanning large format negatives??? Terry Smith Digital Photography 56 06-05-2006 07:58 PM
Canon IXUS record video continously simonharbour@gmail.com Digital Photography 12 06-23-2005 10:08 AM
where to find good patterns and sources of patterns (was Re: singletons) crichmon C++ 4 07-07-2004 10:02 PM



Advertisments