Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > C++ > Help with "read" issue please

Reply
Thread Tools

Help with "read" issue please

 
 
ZafT
Guest
Posts: n/a
 
      07-23-2004
Thanks in advance for any tips that might get me going in the right
direction.

I am working on a simple exercise for school that is supposed to use read to
read a file (about 10 MB). I am supposed to change the buffer size and see
how this affects the read time. In other words, the buffer is supposed to
limit how much of the file gets read per call, and cause some change in
speed. I am supposed to do the same with fread as well, but I'm just asking
for help on read for now.

Everything works like a charm if I have the buffer set to the same size as
the file, but if I make the buffer size smaller than the file, I get a core
dump - segmentation fault.

Here's all the code. Do I need to put the data somewhere or clear the
buffer after a pass or something? I'm guessing I'm stepping out of my
buffer when it gets full.

#include <fstream>
#include <time.h>
#include <iostream>
using namespace std;


void main(){

cout << "start \n" ;

char buff[100];
int length;
time_t seconds;

ifstream inputFile ("TEST", ios::binary);

inputFile.seekg (0, ios::end);
length = inputFile.tellg();
inputFile.seekg (0, ios::beg);

seconds = time (NULL);

cout << seconds << " Seconds at start \n" << length << " bytes\n" ;

// If buffer is same size as the file, I have no problem, but if I
drop the buff size - CRASH!
while(inputFile.read(buff,10000000)){
}

seconds = time(NULL);

cout << seconds << " Seconds at finish \n" ;

}


Any help is appreciated!


Shane



 
Reply With Quote
 
 
 
 
Phlip
Guest
Posts: n/a
 
      07-23-2004
ZafT wrote:

> char buff[100];


> while(inputFile.read(buff,10000000)){


The only thing read() knows about your buf is where it begins. read() relies
on you to tell it where the buf array ends. That's why lying in the second
argument is not a good idea. Pass the actual length of the buf array.

You are learning that the C languages are very mechanical, and will compile
any well-formed source code you give it, regardless of whether it makes
sense.

--
Phlip
http://industrialxp.org/community/bi...UserInterfaces


 
Reply With Quote
 
 
 
 
ZafT
Guest
Posts: n/a
 
      07-23-2004

"Phlip" <(E-Mail Removed)> wrote in message
news:XH0Mc.2707$(E-Mail Removed) m...
> ZafT wrote:
>
> > char buff[100];

>
> > while(inputFile.read(buff,10000000)){

>
> The only thing read() knows about your buf is where it begins. read()

relies
> on you to tell it where the buf array ends. That's why lying in the second
> argument is not a good idea. Pass the actual length of the buf array.
>> --

> Phlip
> http://industrialxp.org/community/bi...UserInterfaces
>
>


Hmmm...

I agree that normally things would work better if I passed a buffer the size
of the file, but I have been asked to do the opposite for the sake of
reading the file in chunks to test performance. Is there no way to do this?

Thanks for letting me know I'm not crazy at least The help is very
appreciated.

Shane



 
Reply With Quote
 
ZafT
Guest
Posts: n/a
 
      07-23-2004

"ZafT" <(E-Mail Removed)> wrote in message
news:(E-Mail Removed)...
>
> "Phlip" <(E-Mail Removed)> wrote in message
> news:XH0Mc.2707$(E-Mail Removed) m...
> > ZafT wrote:
> >
> > > char buff[100];

> >
> > > while(inputFile.read(buff,10000000)){

> >
> > The only thing read() knows about your buf is where it begins. read()

> relies
> > on you to tell it where the buf array ends. That's why lying in the

second
> > argument is not a good idea. Pass the actual length of the buf array.
> >> --

> > Phlip
> >

http://industrialxp.org/community/bi...UserInterfaces
> >
> >

>
> Hmmm...
>
> I agree that normally things would work better if I passed a buffer the

size
> of the file, but I have been asked to do the opposite for the sake of
> reading the file in chunks to test performance. Is there no way to do

this?
>
> Thanks for letting me know I'm not crazy at least The help is very
> appreciated.
>
> Shane
>
>
>


Here's the description of my exercise -

Your program will read the input file several times: for each of the two
cases above, you will use a read size of 1, 2, 5, 10, 20, 50, 100, 200, 500,
1000, 2000, 5000, 10000, 20000, and 50000. In other words, your program will
first open the input file, read its entire contents 1-byte at a time using
the read system call, then close the file. Your program will then open the
file again and read though the file, reading 2-bytes at a time, and so on,
up to 50,000 bytes at a time.

I'm not sure how to accomplish this without changing the buffer.






 
Reply With Quote
 
Phlip
Guest
Posts: n/a
 
      07-23-2004
ZafT wrote:

> > > char buff[100];

> >
> > > while(inputFile.read(buff,10000000)){

> >
> > The only thing read() knows about your buf is where it begins. read()

> relies
> > on you to tell it where the buf array ends. That's why lying in the

second
> > argument is not a good idea. Pass the actual length of the buf array.


> Hmmm...
>
> I agree that normally things would work better if I passed a buffer the

size
> of the file, but I have been asked to do the opposite for the sake of
> reading the file in chunks to test performance. Is there no way to do

this?

You didn't read my reply, or the help files for read, or something.

read() doesn't give a rat's ass how big your buffer is. It will read the
amount specified in its second argument. If that number is too big. read()
will write characters outside your buf array. Your buf is 100 and your read
says 10000000. You are writing all over your stack.

Look these topics up in your tutorial before proceding.

--
Phlip
http://industrialxp.org/community/bi...UserInterfaces


 
Reply With Quote
 
John Harrison
Guest
Posts: n/a
 
      07-23-2004
On Thu, 22 Jul 2004 22:08:00 -0700, ZafT <(E-Mail Removed)> wrote:

>
> "ZafT" <(E-Mail Removed)> wrote in message
> news:(E-Mail Removed)...
>>
>> "Phlip" <(E-Mail Removed)> wrote in message
>> news:XH0Mc.2707$(E-Mail Removed) m...
>> > ZafT wrote:
>> >
>> > > char buff[100];
>> >
>> > > while(inputFile.read(buff,10000000)){
>> >
>> > The only thing read() knows about your buf is where it begins. read()

>> relies
>> > on you to tell it where the buf array ends. That's why lying in the

> second
>> > argument is not a good idea. Pass the actual length of the buf array.
>> >> --
>> > Phlip
>> >

> http://industrialxp.org/community/bi...UserInterfaces
>> >
>> >

>>
>> Hmmm...
>>
>> I agree that normally things would work better if I passed a buffer the

> size
>> of the file, but I have been asked to do the opposite for the sake of
>> reading the file in chunks to test performance. Is there no way to do

> this?
>>
>> Thanks for letting me know I'm not crazy at least The help is very
>> appreciated.
>>
>> Shane
>>
>>
>>

>
> Here's the description of my exercise -
>
> Your program will read the input file several times: for each of the two
> cases above, you will use a read size of 1, 2, 5, 10, 20, 50, 100, 200,
> 500,
> 1000, 2000, 5000, 10000, 20000, and 50000. In other words, your program
> will
> first open the input file, read its entire contents 1-byte at a time
> using
> the read system call, then close the file. Your program will then open
> the
> file again and read though the file, reading 2-bytes at a time, and so
> on,
> up to 50,000 bytes at a time.
>
> I'm not sure how to accomplish this without changing the buffer.
>


You obviously have some sort of blind spot here, here's two cases

// read the entire file 100 bytes at a time
char buf[100];
while (file.read(buf, 100))
{
}

// read the entire file 50000 bytes at a time
char buf[50000];
while (file.read(buf, 50000))
{
}

That's all there is to it.

john
 
Reply With Quote
 
ZafT
Guest
Posts: n/a
 
      07-23-2004

> read() doesn't give a rat's ass how big your buffer is. It will read the
> amount specified in its second argument. If that number is too big. read()
> will write characters outside your buf array. Your buf is 100 and your

read
> says 10000000. You are writing all over your stack.
>
> Look these topics up in your tutorial before proceding.
>
> --
> Phlip
> http://industrialxp.org/community/bi...UserInterfaces
>
>


Thanks for the help. I guessed that I was going a bit out of bounds, but
since the teacher referred to the buffer as the point of manipulation in
class, I thought read() would go until the buffer was full, then make
another pass at it or something. At least now I know I'm wrong.
Unfortionately, I'm not sure how to proceed now.

Can I read 100 byte chunks, ie..

char buff[100];

while(inputFile.read(buff,100)){
// fill buffer
// empty buffer
// go back and read the next 100 bytes
}

??

btw, I did not intentionally ignore your advice - I just misinterpreted it.

Shane





 
Reply With Quote
 
ZafT
Guest
Posts: n/a
 
      07-23-2004

> You obviously have some sort of blind spot here, here's two cases
>
> // read the entire file 100 bytes at a time
> char buf[100];
> while (file.read(buf, 100))
> {
> }
>
> // read the entire file 50000 bytes at a time
> char buf[50000];
> while (file.read(buf, 50000))
> {
> }
>
> That's all there is to it.
>
> john


It's sad that I actually tried that, but both read and fread read in the
file so fast that I assumed that I was doing something wrong. I thought
that I was reading only the first ___ bytes and ending without continuing on
through the rest of the file. At least I can call the program done now and
report my results!

Thanks for all of the help!


 
Reply With Quote
 
John Harrison
Guest
Posts: n/a
 
      07-23-2004
On Thu, 22 Jul 2004 22:41:34 -0700, ZafT <(E-Mail Removed)> wrote:

>
>> You obviously have some sort of blind spot here, here's two cases
>>
>> // read the entire file 100 bytes at a time
>> char buf[100];
>> while (file.read(buf, 100))
>> {
>> }
>>
>> // read the entire file 50000 bytes at a time
>> char buf[50000];
>> while (file.read(buf, 50000))
>> {
>> }
>>
>> That's all there is to it.
>>
>> john

>
> It's sad that I actually tried that, but both read and fread read in the
> file so fast that I assumed that I was doing something wrong. I thought
> that I was reading only the first ___ bytes and ending without
> continuing on
> through the rest of the file. At least I can call the program done now
> and
> report my results!
>
> Thanks for all of the help!
>


There are several things you should be aware of in interpreting your
results. First there are several other buffers between the buffer in your
code and the file. ifstream will maintain its own buffer, and it very
likely that the operating system will have its own buffers as well, and
the hardware could also have its own buffers.

So don't assume that

char buf[100];
while (file.read(buf, 100))
{
}

results in bytes being phyically read 100 at a time from the file.

Another thing to be aware of it that on some operating systems you could
get different results on different occasions. For instance it could be
that if you run your program twice it will run faster the second time.
This is because the operating system is 'remembering' that you read the
file before and so storing part or all of the file in memory assuming that
you will want to read it again. Another case of buffering.

john
 
Reply With Quote
 
ZafT
Guest
Posts: n/a
 
      07-23-2004
..............
>
> So don't assume that
>
> char buf[100];
> while (file.read(buf, 100))
> {
> }
>
> results in bytes being phyically read 100 at a time from the file.
>
> Another thing to be aware of it that on some operating systems you could
> get different results on different occasions. For instance it could be
> that if you run your program twice it will run faster the second time.
> This is because the operating system is 'remembering' that you read the
> file before and so storing part or all of the file in memory assuming that
> you will want to read it again. Another case of buffering.
>
> john


It's like you were reading my mind. I was just wondering why it ran so much
faster the second time. heh. Well it seems as though this experiment for
comparing read and fread is not going to give any kind of raw result.
That's okay though. I've learned a few things tonight. I appreciate your
time very much. This group has been good to me as long as I try everything
myself first and post enough code so people don't have to read my mind.




 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Please please please help this guy with his open source java app casioculture@gmail.com Java 4 05-05-2005 08:24 AM
Console profile for Windows app in VC++ - PLEASE PLEASE PLEASE HELP! MuZZy C++ 7 01-07-2005 08:40 PM
Computer problems please please please help Nick Computer Support 0 06-04-2004 08:49 PM
HELP! HELP! PLEASE, PLEASE, PLEASE tpg comcntr Computer Support 11 02-15-2004 06:22 PM
please help... ...me learn C++ please please please :) KK C++ 2 10-14-2003 02:08 PM



Advertisments