tailing a log file using c ...
 

Hi all,

problem - need to read a log (text) file looking for certain entries,
this file will still be written to by another 3rd party process, and
hence constantly require monitoring (dare I say tailing ?)

Is it possible / practical / efficient to 'tail' a log file using c.


Questions are :

1. If I was reading file and I reached EOF, would I see new file
writes - or would I have to re-open file to see new additions to it ?

2. Would I use unix 'tail' command fed into a c program ?

3. Any advice appreciated ?

Cheers

Tony

Re: tailing a log file using c ...
 

"Tony Stock" <tstock@hotmail.com> schrieb im Newsbeitrag
news:7ab87f9.0311200706.dfa03@posting.google.com.. .
> 1. If I was reading file and I reached EOF, would I see new file
> writes - or would I have to re-open file to see new additions to it ?

Depends on the operating system. On many UNIXes, and on Windows, yes, you
will see the new writes.

You have to open the file in a shared mode using unbuffered I/O (open(),
read(), close()).

Copy the data from read() to an internal buffer and print it when it's ready
(complete line).

Keep the file open and attempt to read() beyond the EOF. If an error is
returned, no new data is available, otherwise read as much as you can
process.


> 2. Would I use unix 'tail' command fed into a c program ?

You could also do that, calling system() to issue a tail command and
retrieving its output. But the other solution is simpler. :-)

Re: tailing a log file using c ...
 

"Ekkehard Morgenstern" <ekkehard.morgenstern@onlinehome.de> wrote:
> "Tony Stock" <tstock@hotmail.com> schrieb:
> > 1. If I was reading file and I reached EOF, would I see new
> > file writes - or would I have to re-open file to see new
> > additions to it ?

It's not guaranteed in standard C. When you open the file it may be
completely locked down so no other program could append log entries
to it.

> Depends on the operating system. On many UNIXes, and on Windows,
> yes, you will see the new writes.
>
> You have to open the file in a shared mode using unbuffered I/O
> (open(), read(), close()).

Unbuffered I/O is not part of Standard C, and is off-topic on comp.lang.c.
The open, read, close functions can be discussed on comp.unix.programmer,
even if you are writing code for Windows, since they are POSIX functions.

--
Simon.

Re: tailing a log file using c ...
 

In article <3fbd5911$0$13681$afc38c87@news.optusnet.com.au> ,
Simon Biber <news@ralminNOSPAM.cc> wrote:
>
>Unbuffered I/O is not part of Standard C, and is off-topic on comp.lang.c.
>The open, read, close functions can be discussed on comp.unix.programmer,
>even if you are writing code for Windows, since they are POSIX functions.

I don't think that true. The standard defines setvbuf() to set/change
the buffering mode of a stream.

#include <stdio.h>
int setvbuf(FILE *stream, char *buf, int mode , size_t size);

--
Rouben Rostamian

Re: tailing a log file using c ...
 

tstock@hotmail.com (Tony Stock) wrote:
>Hi all,
>
>problem - need to read a log (text) file looking for certain entries,
>this file will still be written to by another 3rd party process, and
>hence constantly require monitoring (dare I say tailing ?)
>
>Is it possible / practical / efficient to 'tail' a log file using c.

Yes.

>Questions are :
>
>1. If I was reading file and I reached EOF, would I see new file
>writes - or would I have to re-open file to see new additions to it ?

If you clear the eof condition and re-read the file, you'll see
new data.

>2. Would I use unix 'tail' command fed into a c program ?

If you had asked this on comp.unix.programmer the answer would
be to read the man page for popen(), but...

>3. Any advice appreciated ?

#include <stdio.h>
#include <stdlib.h>
#include <ctype.h>

int main(int argc, char *argv[])
{
char buf[BUFSIZ];
fpos_t pos;
FILE *fp;

if (argc != 2) {
exit(EXIT_FAILURE);
}

if (NULL == (fp = fopen(argv[1], "r"))) {
fprintf(stderr,"Cannot find file: %s\n", argv[1]);
exit(EXIT_FAILURE);
}

if ( 0 != fseek(fp, 0, SEEK_END)) {
fprintf(stderr,"Cannot read file: %s\n", argv[1]);
exit(EXIT_FAILURE);
}

if (-1 == fgetpos(fp, &pos)) {
fprintf(stderr,"Cannot read file: %s\n", argv[1]);
exit(EXIT_FAILURE);
}

for (;;) {
fgets(buf, BUFSIZ, fp);
if (ferror(fp)) {
fprintf(stderr,"Error reading file: %s\n", argv[1]);
exit(EXIT_FAILURE);
}
if (feof(fp)) {
fsetpos(fp, &pos);
clearerr(fp);
#if 0
/* non-ansi code! */
sleep(1); /* give up the cpu to other processes */
#endif
continue;
}
fgetpos(fp, &pos);
printf("%s", buf);
fflush(stdout);
}

return EXIT_SUCCESS;
}


--
Floyd L. Davidson <http://web.newsguy.com/floyd_davidson>
Ukpeagvik (Barrow, Alaska) floyd@barrow.com

Re: tailing a log file using c ...
 

Tony Stock wrote:
> Hi all,
>
> problem - need to read a log (text) file looking for certain entries,
> this file will still be written to by another 3rd party process, and
> hence constantly require monitoring (dare I say tailing ?)
>
> Is it possible / practical / efficient to 'tail' a log file using c.
>
> Questions are :
>
> 1. If I was reading file and I reached EOF, would I see new file
> writes - or would I have to re-open file to see new additions to it ?
>
> 2. Would I use unix 'tail' command fed into a c program ?

Off-topic here, but 'tail -f logfile | your_program' would be the
simplest, and most flexible, solution. Or use the syslog facility.

Re: tailing a log file using c ...
 

Thanks for the responses - food for thought.

It will be on a unix platform (unknown at present though)

Just a little confused, one poster said that it needs to be UNBUFFERED
io (i.e. open / read / close), but the sample code posted uses a
normal fileopen (fopen) ????


Thanks

Tony

Re: tailing a log file using c ...
 

tstock@hotmail.com (Tony Stock) wrote:
>Thanks for the responses - food for thought.
>
>It will be on a unix platform (unknown at present though)
>
>Just a little confused, one poster said that it needs to be UNBUFFERED
>io (i.e. open / read / close), but the sample code posted uses a
>normal fileopen (fopen) ????

Data will be delivered in chunks, and the specifics depend on
whether you use setvbuf (or not) to change the buffering.

A file (as opposed to a terminal) accessed by fopen() will default
to block buffering; hence, you won't see any changes until the
total equals the block size (probably BUFSIZ, defined in stdio.h,
and likely to be 4k or 8k.).

If your logs are line oriented (almost certainly true) you'll
probably want to use setvbuf to change access to line buffered, in
which case each newline will cause data to be available. Otherwise
a single line log entry might not be available until other log
entries occur in sufficient numbers to fill the buffer when block
buffering is used.

The other alternative is to turn off buffering (or use non-portable
low level unbuffered i/o functions), in which case the size of the
chunk will be 1 character, and you're program can happily thrash
away at each and every byte...

--
Floyd L. Davidson <http://web.newsguy.com/floyd_davidson>
Ukpeagvik (Barrow, Alaska) floyd@barrow.com

Re: tailing a log file using c ...
 

In <bpjre4$vu0$1@pc18.math.umbc.edu> rouben@pc18.math.umbc.edu (Rouben Rostamian) writes:

>In article <3fbd5911$0$13681$afc38c87@news.optusnet.com.au> ,
>Simon Biber <news@ralminNOSPAM.cc> wrote:
>>
>>Unbuffered I/O is not part of Standard C, and is off-topic on comp.lang.c.
>>The open, read, close functions can be discussed on comp.unix.programmer,
>>even if you are writing code for Windows, since they are POSIX functions.
>
>I don't think that true. The standard defines setvbuf() to set/change
>the buffering mode of a stream.
>
> #include <stdio.h>
> int setvbuf(FILE *stream, char *buf, int mode , size_t size);

This only suppresses the buffering performed by the standard C library.
Sometimes, you may need to suppress the buffering performed by the OS,
too, and setvbuf is very unlikely to help (although, in theory, it could).

But when people talk about unbuffered I/O (or raw I/O or low level I/O)
they usually mean I/O performed using the OS primitives, rather than the
(relatively high level) <stdio.h> stuff.

Dan
--
Dan Pop
DESY Zeuthen, RZ group
Email: Dan.Pop@ifh.de

Re: tailing a log file using c ...
 

Floyd Davidson <floyd@barrow.com> wrote:
>tstock@hotmail.com (Tony Stock) wrote:
>>Thanks for the responses - food for thought.
>>
>>It will be on a unix platform (unknown at present though)
>>
>>Just a little confused, one poster said that it needs to be UNBUFFERED
>>io (i.e. open / read / close), but the sample code posted uses a
>>normal fileopen (fopen) ????
>
>Data will be delivered in chunks, and the specifics depend on
>whether you use setvbuf (or not) to change the buffering.
>
>A file (as opposed to a terminal) accessed by fopen() will default
>to block buffering; hence, you won't see any changes until the
>total equals the block size (probably BUFSIZ, defined in stdio.h,
>and likely to be 4k or 8k.).
>
>If your logs are line oriented (almost certainly true) you'll
>probably want to use setvbuf to change access to line buffered, in
>which case each newline will cause data to be available. Otherwise
>a single line log entry might not be available until other log
>entries occur in sufficient numbers to fill the buffer when block
>buffering is used.
>
>The other alternative is to turn off buffering (or use non-portable
>low level unbuffered i/o functions), in which case the size of the
>chunk will be 1 character, and you're program can happily thrash
>away at each and every byte...

Well, all of that sounds good... but it doesn't do that.

The code that I posted uses fgets() for input, and acts as described
for line buffering because of that. By changing the input to fread()
behavior as described for unbuffered input can be obtained.

Each read either reaches the end of a line or the end of the
buffer (which ever is important for the mode used) _or_ it hits
an end of file condition. Hence the new data is always available.

--
Floyd L. Davidson <http://web.newsguy.com/floyd_davidson>
Ukpeagvik (Barrow, Alaska) floyd@barrow.com