Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Perl > Perl Misc > $line = <FH> returns incomplete lines

Reply
Thread Tools

$line = <FH> returns incomplete lines

 
 
martyjc2000@yahoo.co.uk
Guest
Posts: n/a
 
      03-24-2005
Hi

I have a logscanner script (don't we all!).
However I have recently discovered that if the log that it is reading
has, say half a line written, then a pause and then the rest of the
line is written, $line = <FH> will read in the first half of the line
first and then read in the second half of the line separately. Here is
a simple e.g. script:



#!/usr/bin/perl
# logscanner.pl

$|=1;

open (MYLOG, "/tmp/mylog");

while ( 1 )
{
while ( ($line = <MYLOG> ) )
{
print ("GOT THIS: ",$line);
}
seek (MYLOG,0, 1);
sleep (1);
}



To recreate problem, touch /tmp/mylog then run the above script.

Then in a separate terminal run this script which will let you write to
the log:



#!/opt/DKBperl/bin/perl
# writer.pl

$|=1;

open (LOG, ">>/tmp/mylog");
select LOG;
$|=1;
$stuff="";
while ( $stuff ne "exit" )
{
$stuff = <STDIN>;
if ( $stuff !~ /n$/ )
{
chomp ($stuff);
}
print LOG $stuff;
}
close (LOG);



The writer.pl script will write whatever you type into the log, without
buffering.

To make a newline enter n at the end of your line.

here is an e.g. run:

../writer.pl
line1n
line2n
line3 beginning
line3 endn


The final /tmp/mylog looks like this:
line1n
line2n
line3 beginningline3 endn


output from logscanner.pl is:
GOT THIS: line1n
GOT THIS: line2n
GOT THIS: line3 beginningGOT THIS: line3 endn

e.g. even though line3beginning did not have a new line, $line =
<MYLOG> returned that partial line. I thought it would wait until a
whole line terminated by a new line character is present.

Any ideas how I can change the logscanner script to only return the
whole line? This has to work in real time (e.g. in a similar way to
tail -f).

big thanks

Martin

 
Reply With Quote
 
 
 
 
martyjc2000@yahoo.co.uk
Guest
Posts: n/a
 
      03-24-2005
Just to clarify, I want the output of logscanner.pl should look like
this:

GOT THIS: line1n
GOT THIS: line2n
GOT THIS: line3 beginningline3 endn

NOT this:

GOT THIS: line1n
GOT THIS: line2n
GOT THIS: line3 beginningGOT THIS: line3 endn


(E-Mail Removed) wrote:
> Hi
>
> I have a logscanner script (don't we all!).
> However I have recently discovered that if the log that it is reading
> has, say half a line written, then a pause and then the rest of the
> line is written, $line = <FH> will read in the first half of the line
> first and then read in the second half of the line separately. Here

is
> a simple e.g. script:
>
>
>
> #!/usr/bin/perl
> # logscanner.pl
>
> $|=1;
>
> open (MYLOG, "/tmp/mylog");
>
> while ( 1 )
> {
> while ( ($line = <MYLOG> ) )
> {
> print ("GOT THIS: ",$line);
> }
> seek (MYLOG,0, 1);
> sleep (1);
> }
>
>
>
> To recreate problem, touch /tmp/mylog then run the above script.
>
> Then in a separate terminal run this script which will let you write

to
> the log:
>
>
>
> #!/opt/DKBperl/bin/perl
> # writer.pl
>
> $|=1;
>
> open (LOG, ">>/tmp/mylog");
> select LOG;
> $|=1;
> $stuff="";
> while ( $stuff ne "exit" )
> {
> $stuff = <STDIN>;
> if ( $stuff !~ /n$/ )
> {
> chomp ($stuff);
> }
> print LOG $stuff;
> }
> close (LOG);
>
>
>
> The writer.pl script will write whatever you type into the log,

without
> buffering.
>
> To make a newline enter n at the end of your line.
>
> here is an e.g. run:
>
> ./writer.pl
> line1n
> line2n
> line3 beginning
> line3 endn
>
>
> The final /tmp/mylog looks like this:
> line1n
> line2n
> line3 beginningline3 endn
>
>
> output from logscanner.pl is:
> GOT THIS: line1n
> GOT THIS: line2n
> GOT THIS: line3 beginningGOT THIS: line3 endn
>
> e.g. even though line3beginning did not have a new line, $line =
> <MYLOG> returned that partial line. I thought it would wait until a
> whole line terminated by a new line character is present.
>
> Any ideas how I can change the logscanner script to only return the
> whole line? This has to work in real time (e.g. in a similar way to
> tail -f).
>
> big thanks
>
> Martin


 
Reply With Quote
 
 
 
 
martyjc2000@yahoo.co.uk
Guest
Posts: n/a
 
      03-24-2005
Just to clarify, I want the output of logscanner.pl should look like
this:

GOT THIS: line1n
GOT THIS: line2n
GOT THIS: line3 beginningline3 endn

NOT this:

GOT THIS: line1n
GOT THIS: line2n
GOT THIS: line3 beginningGOT THIS: line3 endn


(E-Mail Removed) wrote:
> Hi
>
> I have a logscanner script (don't we all!).
> However I have recently discovered that if the log that it is reading
> has, say half a line written, then a pause and then the rest of the
> line is written, $line = <FH> will read in the first half of the line
> first and then read in the second half of the line separately. Here

is
> a simple e.g. script:
>
>
>
> #!/usr/bin/perl
> # logscanner.pl
>
> $|=1;
>
> open (MYLOG, "/tmp/mylog");
>
> while ( 1 )
> {
> while ( ($line = <MYLOG> ) )
> {
> print ("GOT THIS: ",$line);
> }
> seek (MYLOG,0, 1);
> sleep (1);
> }
>
>
>
> To recreate problem, touch /tmp/mylog then run the above script.
>
> Then in a separate terminal run this script which will let you write

to
> the log:
>
>
>
> #!/opt/DKBperl/bin/perl
> # writer.pl
>
> $|=1;
>
> open (LOG, ">>/tmp/mylog");
> select LOG;
> $|=1;
> $stuff="";
> while ( $stuff ne "exit" )
> {
> $stuff = <STDIN>;
> if ( $stuff !~ /n$/ )
> {
> chomp ($stuff);
> }
> print LOG $stuff;
> }
> close (LOG);
>
>
>
> The writer.pl script will write whatever you type into the log,

without
> buffering.
>
> To make a newline enter n at the end of your line.
>
> here is an e.g. run:
>
> ./writer.pl
> line1n
> line2n
> line3 beginning
> line3 endn
>
>
> The final /tmp/mylog looks like this:
> line1n
> line2n
> line3 beginningline3 endn
>
>
> output from logscanner.pl is:
> GOT THIS: line1n
> GOT THIS: line2n
> GOT THIS: line3 beginningGOT THIS: line3 endn
>
> e.g. even though line3beginning did not have a new line, $line =
> <MYLOG> returned that partial line. I thought it would wait until a
> whole line terminated by a new line character is present.
>
> Any ideas how I can change the logscanner script to only return the
> whole line? This has to work in real time (e.g. in a similar way to
> tail -f).
>
> big thanks
>
> Martin


 
Reply With Quote
 
John Bokma
Guest
Posts: n/a
 
      03-24-2005
wrote:

> Hi
>
> I have a logscanner script (don't we all!).


Nop, I use tail -f access_log

sometimes like:

tail -f access_log | grep -i googlebot

--
John Small Perl scripts: http://johnbokma.com/perl/
Perl programmer available: http://castleamber.com/
Happy Customers: http://castleamber.com/testimonials.html

 
Reply With Quote
 
Brian McCauley
Guest
Posts: n/a
 
      03-24-2005
http://www.velocityreviews.com/forums/(E-Mail Removed) wrote:

> I have a logscanner script (don't we all!).
> However I have recently discovered that if the log that it is reading
> has, say half a line written, then a pause and then the rest of the
> line is written, $line = <FH> will read in the first half of the line
> first and then read in the second half of the line separately.


Yes. <> will return the final unterminated line of a file.

Try File::Tail.

 
Reply With Quote
 
martyjc2000@yahoo.co.uk
Guest
Posts: n/a
 
      03-24-2005
Thanks

but, I really wanted to use core perl, rather than other modules.
Reason being that I use this perl script (not this perl script, this is
just an example script!) on hundreds of servers.

Was therefore hoping to find a magic answer which didn't rely on other
modules....

One workaround is to check each line that is read for a newline
character, if it doesn't exist then keep the line and append on the
next line that is read.... but thats not very elegent.

any other ideas???

 
Reply With Quote
 
A. Sinan Unur
Guest
Posts: n/a
 
      03-24-2005
(E-Mail Removed) wrote in
news:(E-Mail Removed) oups.com:

> Thanks


Please quote some context when you are replying.

> but, I really wanted to use core perl, rather than other modules.
> Reason being that I use this perl script (not this perl script, this
> is just an example script!) on hundreds of servers.


It is then all the more important not to re-invent the wheel.

> Was therefore hoping to find a magic answer which didn't rely on other
> modules....


OK.

> One workaround is to check each line that is read for a newline
> character, if it doesn't exist then keep the line and append on the
> next line that is read.... but thats not very elegent.
>
> any other ideas???


use the File::Tail module.

You could also read the source code of the File::Tail module, but cut &
paste reuse of code is much worse than reuse through modules. The latter
allows you to benefit from everyone else's work in maintaining the
modules.

Sinan
 
Reply With Quote
 
Sherm Pendley
Guest
Posts: n/a
 
      03-24-2005
(E-Mail Removed) wrote:

> but, I really wanted to use core perl, rather than other modules.
> Reason being that I use this perl script (not this perl script, this is
> just an example script!) on hundreds of servers.


Instead of limiting yourself this way - and not using CPAN modules is a
*major* limitation - I think it would be more productive for you to
research ways that you could manage CPAN modules without having to
individually install them on all those hundreds of machines.

For instance, Perl has the notion of a "vendor library" directory that's
included in @INC, along with the more commonly used core and site dirs. You
could configure your Perl(s) to use a network-mounted volume for this. A
single install to that volume would then make a module available for all of
the machines that have been configured to use it.

sherm--

--
Cocoa programming in Perl: http://camelbones.sourceforge.net
Hire me! My resume: http://www.dot-app.org
 
Reply With Quote
 
Jeff
Guest
Posts: n/a
 
      03-24-2005
Sherm Pendley wrote:
> Instead of limiting yourself this way - and not using CPAN modules is a
> *major* limitation - I think it would be more productive for you to
> research ways that you could manage CPAN modules without having to
> individually install them on all those hundreds of machines.
>
> For instance, Perl has the notion of a "vendor library" directory that's
> included in @INC, along with the more commonly used core and site dirs. You
> could configure your Perl(s) to use a network-mounted volume for this. A
> single install to that volume would then make a module available for all of
> the machines that have been configured to use it.


And if you don't want to reconfigure/recompile perl on all the machines,
you can add a BEGIN block to your scripts that adds the network location
to @INC via 'use lib'.

Laziness will help you write better programs.

~Jeff
 
Reply With Quote
 
Shawn Corey
Guest
Posts: n/a
 
      03-24-2005
A. Sinan Unur wrote:
> use the File::Tail module.
>
> You could also read the source code of the File::Tail module, but cut &
> paste reuse of code is much worse than reuse through modules. The latter
> allows you to benefit from everyone else's work in maintaining the
> modules.
>
> Sinan


Don't forget that many of the CPAN modules are released under GPL (or
similar licenses). If you cut & paste from them, your code is a derived
work which means you *must* publish it under the terms of GPL.

Check the terms of any open source license before you cut & paste.

--- Shawn
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Sorted Returns List and Reversed Returns Iterator ++imanshu Python 7 08-23-2008 04:25 AM
Hash#select returns an array but Hash#reject returns a hash... Srijayanth Sridhar Ruby 19 07-02-2008 12:49 PM
createImage sometime returns null and sometime returns non-null. vizlab Java 3 10-17-2007 11:21 AM
HttpWebResponse.GetResponseStream returns incomplete stream ThePants ASP .Net 9 08-31-2006 07:32 PM
block returns and hash element returns Trans Ruby 2 11-06-2005 12:15 PM



Advertisments