Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > C Programming > how to read a file newly generated in real time?

Reply
Thread Tools

how to read a file newly generated in real time?

 
 
webinfinite@gmail.com
Guest
Posts: n/a
 
      08-16-2007
I am starting a new thread for this topic since previous problem has
been solved.

I have code like this:

#include <stdio.h>


int main(){

FILE *fp;

while(1){
fp = fopen("foo.txt", "r"); ;
if (fp != NULL)
break;
else{
printf("Waiting for the file\n");
}
}


printf("I am out.\n");
fclose(fp);
return 0;
}

foo.txt is generated after this code starts to execute. My problem
here is that even after foo.txt has been generated, the code is still
in a tight loop printing:

Waiting for the file
Waiting for the file
Waiting for the file
Waiting for the file
Waiting for the file
Waiting for the file

Until long after the file is generated, the code comes out. Is there
any solution to force the code to find out if this file has been
generated?

Thank you.

 
Reply With Quote
 
 
 
 
Jens Thoms Toerring
Guest
Posts: n/a
 
      08-16-2007
http://www.velocityreviews.com/forums/(E-Mail Removed) <(E-Mail Removed)> wrote:
> I am starting a new thread for this topic since previous problem has
> been solved.


> I have code like this:


> #include <stdio.h>


> int main(){
> FILE *fp;
> while(1){
> fp = fopen("foo.txt", "r"); ;
> if (fp != NULL)
> break;
> else{
> printf("Waiting for the file\n");
> }
> }
> printf("I am out.\n");
> fclose(fp);
> return 0;
> }


> foo.txt is generated after this code starts to execute. My problem
> here is that even after foo.txt has been generated, the code is still
> in a tight loop printing:


> Waiting for the file
> Waiting for the file
> Waiting for the file
> Waiting for the file
> Waiting for the file
> Waiting for the file


> Until long after the file is generated, the code comes out. Is there
> any solution to force the code to find out if this file has been
> generated?


What's a "long time"? If you're in such a tight loop you can get
hundreds of such lines of output is half a second (and maybe your
terminal can't keep up and takes a long time to just output all
those lines).

But perhaps the other application that creates the file creates
it with write-only permissions and only changes that once it's
done with writing everything it wants to the file? Of course
this is pure guesswork since there is no information to go by
except what you write, but it might be useful to check (with
probably system-specific functions, e.g. if you're running
UNIX you could check what errno tells you about the reasons)
why fopen() fails for that long.

Regards, Jens
--
\ Jens Thoms Toerring ___ (E-Mail Removed)
\__________________________ http://toerring.de
 
Reply With Quote
 
 
 
 
webinfinite@gmail.com
Guest
Posts: n/a
 
      08-16-2007
On 8 16 , 3 23 , (E-Mail Removed) (Jens Thoms Toerring) wrote:
> (E-Mail Removed) <(E-Mail Removed)> wrote:
> > I am starting a new thread for this topic since previous problem has
> > been solved.
> > I have code like this:
> > #include <stdio.h>
> > int main(){
> > FILE *fp;
> > while(1){
> > fp = fopen("foo.txt", "r"); ;
> > if (fp != NULL)
> > break;
> > else{
> > printf("Waiting for the file\n");
> > }
> > }
> > printf("I am out.\n");
> > fclose(fp);
> > return 0;
> > }
> > foo.txt is generated after this code starts to execute. My problem
> > here is that even after foo.txt has been generated, the code is still
> > in a tight loop printing:
> > Waiting for the file
> > Waiting for the file
> > Waiting for the file
> > Waiting for the file
> > Waiting for the file
> > Waiting for the file
> > Until long after the file is generated, the code comes out. Is there
> > any solution to force the code to find out if this file has been
> > generated?

>
> What's a "long time"? If you're in such a tight loop you can get
> hundreds of such lines of output is half a second (and maybe your
> terminal can't keep up and takes a long time to just output all
> those lines).
>
> But perhaps the other application that creates the file creates
> it with write-only permissions and only changes that once it's
> done with writing everything it wants to the file? Of course
> this is pure guesswork since there is no information to go by
> except what you write, but it might be useful to check (with
> probably system-specific functions, e.g. if you're running
> UNIX you could check what errno tells you about the reasons)
> why fopen() fails for that long.
>
> Regards, Jens
> --
> \ Jens Thoms Toerring ___ (E-Mail Removed)
> \__________________________ http://toerring.de- -
>
> - -


It takes the code more than 10 seconds to get the file. I would like
to find a solution that when you generate the file, it will be
captured by the code immediately. Is there a way? Thanks.

 
Reply With Quote
 
webinfinite@gmail.com
Guest
Posts: n/a
 
      08-16-2007
On 8 16 , 3 25 , "(E-Mail Removed)" <(E-Mail Removed)>
wrote:
> On 8 16 , 3 23 , (E-Mail Removed) (Jens Thoms Toerring) wrote:
>
>
>
>
>
> > (E-Mail Removed) <(E-Mail Removed)> wrote:
> > > I am starting a new thread for this topic since previous problem has
> > > been solved.
> > > I have code like this:
> > > #include <stdio.h>
> > > int main(){
> > > FILE *fp;
> > > while(1){
> > > fp = fopen("foo.txt", "r"); ;
> > > if (fp != NULL)
> > > break;
> > > else{
> > > printf("Waiting for the file\n");
> > > }
> > > }
> > > printf("I am out.\n");
> > > fclose(fp);
> > > return 0;
> > > }
> > > foo.txt is generated after this code starts to execute. My problem
> > > here is that even after foo.txt has been generated, the code is still
> > > in a tight loop printing:
> > > Waiting for the file
> > > Waiting for the file
> > > Waiting for the file
> > > Waiting for the file
> > > Waiting for the file
> > > Waiting for the file
> > > Until long after the file is generated, the code comes out. Is there
> > > any solution to force the code to find out if this file has been
> > > generated?

>
> > What's a "long time"? If you're in such a tight loop you can get
> > hundreds of such lines of output is half a second (and maybe your
> > terminal can't keep up and takes a long time to just output all
> > those lines).

>
> > But perhaps the other application that creates the file creates
> > it with write-only permissions and only changes that once it's
> > done with writing everything it wants to the file? Of course
> > this is pure guesswork since there is no information to go by
> > except what you write, but it might be useful to check (with
> > probably system-specific functions, e.g. if you're running
> > UNIX you could check what errno tells you about the reasons)
> > why fopen() fails for that long.

>
> > Regards, Jens
> > --
> > \ Jens Thoms Toerring ___ (E-Mail Removed)
> > \__________________________ http://toerring.de- -

>
> > - -

>
> It takes the code more than 10 seconds to get the file. I would like
> to find a solution that when you generate the file, it will be
> captured by the code immediately. Is there a way? Thanks.- -
>
> - -


I timed the code, the code will take 43 second to pick up the new
generated file. Anything can make this process quick?

 
Reply With Quote
 
Jens Thoms Toerring
Guest
Posts: n/a
 
      08-16-2007
(E-Mail Removed) <(E-Mail Removed)> wrote:
> On 8 16 , 3 23 , (E-Mail Removed) (Jens Thoms Toerring) wrote:
> > (E-Mail Removed) <(E-Mail Removed)> wrote:
> > > #include <stdio.h>
> > > int main(){
> > > FILE *fp;
> > > while(1){
> > > fp = fopen("foo.txt", "r"); ;
> > > if (fp != NULL)
> > > break;
> > > else{
> > > printf("Waiting for the file\n");
> > > }
> > > }
> > > printf("I am out.\n");
> > > fclose(fp);
> > > return 0;
> > > }
> > > foo.txt is generated after this code starts to execute. My problem
> > > here is that even after foo.txt has been generated, the code is still
> > > in a tight loop printing:

>
> > But perhaps the other application that creates the file creates
> > it with write-only permissions and only changes that once it's
> > done with writing everything it wants to the file? Of course
> > this is pure guesswork since there is no information to go by
> > except what you write, but it might be useful to check (with
> > probably system-specific functions, e.g. if you're running
> > UNIX you could check what errno tells you about the reasons)
> > why fopen() fails for that long.

>
> It takes the code more than 10 seconds to get the file. I would like
> to find a solution that when you generate the file, it will be
> captured by the code immediately. Is there a way? Thanks.


There's nothing in your program that would keep it form doing so
unless there are additional circumstances that would make fopen()
fail. One of those could be that the other program opens the file
with permissions that don't allow your program to open it for
reading - but you're the only one who can check for that. To
repeat: in your program fopen() should return successfully once
the file exists and it's permitted to open the file for reading.
One of these two conditions must not be satisfied for the 10
seconds you're writing about and only you have the possibility
to find out what it is - therefore I did recommend to resort to
system-specific methods (the C standard only requires that
fopen() returns NULL on failure but does not specify any methods
to obtain further informations) to determine why fopen() did fail,
those methods may tell you that the file either doesn't exist
when you assume it does or that you for some time don't have the
permissions to open it for reading.

Regards, Jens
--
\ Jens Thoms Toerring ___ (E-Mail Removed)
\__________________________ http://toerring.de
 
Reply With Quote
 
webinfinite@gmail.com
Guest
Posts: n/a
 
      08-16-2007
On 8 16 , 3 50 , (E-Mail Removed) (Jens Thoms Toerring) wrote:
> (E-Mail Removed) <(E-Mail Removed)> wrote:
> > On 8 16 , 3 23 , (E-Mail Removed) (Jens Thoms Toerring) wrote:
> > > (E-Mail Removed) <(E-Mail Removed)> wrote:
> > > > #include <stdio.h>
> > > > int main(){
> > > > FILE *fp;
> > > > while(1){
> > > > fp = fopen("foo.txt", "r"); ;
> > > > if (fp != NULL)
> > > > break;
> > > > else{
> > > > printf("Waiting for the file\n");
> > > > }
> > > > }
> > > > printf("I am out.\n");
> > > > fclose(fp);
> > > > return 0;
> > > > }
> > > > foo.txt is generated after this code starts to execute. My problem
> > > > here is that even after foo.txt has been generated, the code is still
> > > > in a tight loop printing:

>
> > > But perhaps the other application that creates the file creates
> > > it with write-only permissions and only changes that once it's
> > > done with writing everything it wants to the file? Of course
> > > this is pure guesswork since there is no information to go by
> > > except what you write, but it might be useful to check (with
> > > probably system-specific functions, e.g. if you're running
> > > UNIX you could check what errno tells you about the reasons)
> > > why fopen() fails for that long.

>
> > It takes the code more than 10 seconds to get the file. I would like
> > to find a solution that when you generate the file, it will be
> > captured by the code immediately. Is there a way? Thanks.

>
> There's nothing in your program that would keep it form doing so
> unless there are additional circumstances that would make fopen()
> fail. One of those could be that the other program opens the file
> with permissions that don't allow your program to open it for
> reading - but you're the only one who can check for that. To
> repeat: in your program fopen() should return successfully once
> the file exists and it's permitted to open the file for reading.
> One of these two conditions must not be satisfied for the 10
> seconds you're writing about and only you have the possibility
> to find out what it is - therefore I did recommend to resort to
> system-specific methods (the C standard only requires that
> fopen() returns NULL on failure but does not specify any methods
> to obtain further informations) to determine why fopen() did fail,
> those methods may tell you that the file either doesn't exist
> when you assume it does or that you for some time don't have the
> permissions to open it for reading.
>
> Regards, Jens
> --
> \ Jens Thoms Toerring ___ (E-Mail Removed)
> \__________________________ http://toerring.de- -
>
> - -


The way I am doing this is very simiple. Under the same directory of
the executable,there is a file named "boo.txt". I start the code in
one terminal. Then in the other terminal, I just type: "mv boo.txt
foo.txt".

I was hoping the code capture this immediately, but it didn't. It
takes more than 40 seconds to take this change. It takes too long. I
was wondering if there is any efficient way to reduce this 40 seconds
to say maybe 5 seconds?

 
Reply With Quote
 
Francine.Neary@googlemail.com
Guest
Posts: n/a
 
      08-16-2007
On Aug 16, 11:59 pm, "(E-Mail Removed)" <(E-Mail Removed)>
wrote:
> On 8 16 , 3 50 , (E-Mail Removed) (Jens Thoms Toerring) wrote:
>
>
>
> > (E-Mail Removed) <(E-Mail Removed)> wrote:
> > > On 8 16 , 3 23 , (E-Mail Removed) (Jens Thoms Toerring) wrote:
> > > > (E-Mail Removed) <(E-Mail Removed)> wrote:
> > > > > #include <stdio.h>
> > > > > int main(){
> > > > > FILE *fp;
> > > > > while(1){
> > > > > fp = fopen("foo.txt", "r"); ;
> > > > > if (fp != NULL)
> > > > > break;
> > > > > else{
> > > > > printf("Waiting for the file\n");
> > > > > }
> > > > > }
> > > > > printf("I am out.\n");
> > > > > fclose(fp);
> > > > > return 0;
> > > > > }
> > > > > foo.txt is generated after this code starts to execute. My problem
> > > > > here is that even after foo.txt has been generated, the code is still
> > > > > in a tight loop printing:

>
> > > > But perhaps the other application that creates the file creates
> > > > it with write-only permissions and only changes that once it's
> > > > done with writing everything it wants to the file? Of course
> > > > this is pure guesswork since there is no information to go by
> > > > except what you write, but it might be useful to check (with
> > > > probably system-specific functions, e.g. if you're running
> > > > UNIX you could check what errno tells you about the reasons)
> > > > why fopen() fails for that long.

>
> > > It takes the code more than 10 seconds to get the file. I would like
> > > to find a solution that when you generate the file, it will be
> > > captured by the code immediately. Is there a way? Thanks.

>
> > There's nothing in your program that would keep it form doing so
> > unless there are additional circumstances that would make fopen()
> > fail. One of those could be that the other program opens the file
> > with permissions that don't allow your program to open it for
> > reading - but you're the only one who can check for that. To
> > repeat: in your program fopen() should return successfully once
> > the file exists and it's permitted to open the file for reading.
> > One of these two conditions must not be satisfied for the 10
> > seconds you're writing about and only you have the possibility
> > to find out what it is - therefore I did recommend to resort to
> > system-specific methods (the C standard only requires that
> > fopen() returns NULL on failure but does not specify any methods
> > to obtain further informations) to determine why fopen() did fail,
> > those methods may tell you that the file either doesn't exist
> > when you assume it does or that you for some time don't have the
> > permissions to open it for reading.

>
> > Regards, Jens
> > --
> > \ Jens Thoms Toerring ___ (E-Mail Removed)
> > \__________________________ http://toerring.de- -

>
> > - -

>
> The way I am doing this is very simiple. Under the same directory of
> the executable,there is a file named "boo.txt". I start the code in
> one terminal. Then in the other terminal, I just type: "mv boo.txt
> foo.txt".
>
> I was hoping the code capture this immediately, but it didn't. It
> takes more than 40 seconds to take this change. It takes too long. I
> was wondering if there is any efficient way to reduce this 40 seconds
> to say maybe 5 seconds?


<OT>
As you're using *nix, you could try looking at FAM (http://oss.sgi.com/
projects/fam/)
</OT>

 
Reply With Quote
 
Jens Thoms Toerring
Guest
Posts: n/a
 
      08-16-2007
(E-Mail Removed) <(E-Mail Removed)> wrote:
> The way I am doing this is very simiple. Under the same directory of
> the executable,there is a file named "boo.txt". I start the code in
> one terminal. Then in the other terminal, I just type: "mv boo.txt
> foo.txt".


> I was hoping the code capture this immediately, but it didn't. It
> takes more than 40 seconds to take this change. It takes too long. I
> was wondering if there is any efficient way to reduce this 40 seconds
> to say maybe 5 seconds?


This can only be a system-specific problem and, as far as I can
see, it's definitely not related to the C language. I don't know
what your system is so I can't even tell you where to ask (the
commands you mention look a bit like UNIX, but for example under
Linux I couldn't reproduce anything similar to what you write,
once I rename the file to foo.txt your program stops immediately).
So all I can recommend is to take this to a group that deals with
the details of the system you are using since that problem is un-
likely to be related in any way to C. If it's a UNIX-like system
you could go to comp.unix.programmer but please write explicitely
what kind of UNIX you're running, this can make a lot of a diffe-
rence.
Regards, Jens
--
\ Jens Thoms Toerring ___ (E-Mail Removed)
\__________________________ http://toerring.de
 
Reply With Quote
 
Al Balmer
Guest
Posts: n/a
 
      08-16-2007
On Thu, 16 Aug 2007 22:09:12 -0000, "(E-Mail Removed)"
<(E-Mail Removed)> wrote:

(This is not really on topic here)

>Until long after the file is generated, the code comes out.


Of course it does. The tight loop has filled all the buffers the
system can provide, the buffer in the printer, etc, and it will keep
printing until they empty. In fact, if there's no more buffer space,
it will probably suspend your program and resume it to print the
remaining hundreds or thousands of lines.

> Is there
>any solution to force the code to find out if this file has been
>generated?


Of course, but the best solution is probably dependent on your system.
Ask in an appropriate newsgroup.

In the meantime, you can do some things. Put a delay in the loop so
that you print an entire line before checking the file again. If you
can't wait that long, check the file more often but don't print as
often. You can do that by either checking elapsed time and doing the
print only when (say) two seconds have elapsed, or by printing only
every Nth time around the loop, N determined by experiment.

You can combine these things, too. Maybe you can put a one second
delay in the loop, and print the message every 10 iterations, for
example.

--
Al Balmer
Sun City, AZ
 
Reply With Quote
 
Keith Thompson
Guest
Posts: n/a
 
      08-17-2007
"(E-Mail Removed)" <(E-Mail Removed)> writes:
> I am starting a new thread for this topic since previous problem has
> been solved.
>
> I have code like this:
>
> #include <stdio.h>
>
>
> int main(){
>
> FILE *fp;
>
> while(1){
> fp = fopen("foo.txt", "r"); ;
> if (fp != NULL)
> break;
> else{
> printf("Waiting for the file\n");
> }
> }
>
>
> printf("I am out.\n");
> fclose(fp);
> return 0;
> }
>
> foo.txt is generated after this code starts to execute. My problem
> here is that even after foo.txt has been generated, the code is still
> in a tight loop printing:
>
> Waiting for the file
> Waiting for the file
> Waiting for the file
> Waiting for the file
> Waiting for the file
> Waiting for the file
>
> Until long after the file is generated, the code comes out. Is there
> any solution to force the code to find out if this file has been
> generated?


Like others here, I suspect that printing "Waiting for the file" in a
tight loop is the real problem.

Try this version of your program. If the problem goes away, then it
was probably caused by output buffering. If it doesn't, then it's
probably something system-specific; try comp.unix.programmer (and be
prepared to offer more details about your system).

#include <stdio.h>
int main(void)
{
unsigned long count = 0;
FILE *fp;
while (1) {
fp = fopen("foo.txt", "r");
if (fp != NULL) {
break;
}
else {
count ++;
}
}
printf("Done, count = %ld\n", count);
fclose(fp);
return 0;
}

--
Keith Thompson (The_Other_Keith) (E-Mail Removed) <http://www.ghoti.net/~kst>
San Diego Supercomputer Center <*> <http://users.sdsc.edu/~kst>
"We must do something. This is something. Therefore, we must do this."
-- Antony Jay and Jonathan Lynn, "Yes Minister"
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Re: How include a large array? Edward A. Falk C Programming 1 04-04-2013 08:07 PM
File.read(fname) vs. File.read(fname,File.size(fname)) Alex Dowad Ruby 4 05-01-2010 08:20 AM
file.read() doesn't read the whole file Sreejith K Python 24 03-24-2009 12:20 PM
How to test a text file is newly created? www Java 3 09-28-2007 12:15 PM
How to test a text file is newly created? www Java 4 09-28-2007 03:47 AM



Advertisments