Velocity Reviews

Velocity Reviews (http://www.velocityreviews.com/forums/index.php)
-   C++ (http://www.velocityreviews.com/forums/f39-c.html)
-   -   Connecting File Streams (http://www.velocityreviews.com/forums/t459878-connecting-file-streams.html)

woessner@gmail.com 01-17-2007 06:35 PM

Connecting File Streams
 
This is not a really well-formed question so please bear with me.

I have a very large binary file (tens of GB). I would like to write a
quick program to take a contiguous piece of the file and save it to
another file. Conceptually, this is very simple:

Open ifstream
Open ofstream
Seek ifstream to beginning of piece
Read ifstream in to buffer
Write buffer to ofstream

I'm not very familiar with streams, but I think even I could handle
that. My concern is the with the buffer. I'm going to want to copy
far more data than will fit in memory. That means I will have to use a
loop to read and write the data in chunks. OK, that's still not a
problem. But it does raise the question of how big to make the buffer.
My gut instinct tells me to make the buffer as big as possible, but
I'm not really sure.

I'm hoping there's a way to avoid this problem, altogether. Is it
possible to "connect" an ifstream and an ofstream? Conceptually, this
is what I'm after:

Open ifstream
Open ofstream
Connect ifstream to ofstream
Seek ifstream to beginning of piece
Read ifstream (automatically writes to ofstream)

My hope is that the stream classes will allocate whatever buffers they
need and that those buffers will be appropriately sized.

I've looked through the Josuttis book a little bit. I read the section
on connecting streams, but that seems to be for synchronization. Does
anybody know if this is possible (and a good idea)? If not, I'm open
to any other suggestions.

Thanks in advance,
Bill


Yahooooooooo 01-18-2007 07:17 AM

Re: Connecting File Streams
 
streambuf may help to connect these streams

std::streambuf
std::cout.rdbuf



woessner@gmail.com wrote:
> This is not a really well-formed question so please bear with me.
>
> I have a very large binary file (tens of GB). I would like to write a
> quick program to take a contiguous piece of the file and save it to
> another file. Conceptually, this is very simple:
>
> Open ifstream
> Open ofstream
> Seek ifstream to beginning of piece
> Read ifstream in to buffer
> Write buffer to ofstream
>
> I'm not very familiar with streams, but I think even I could handle
> that. My concern is the with the buffer. I'm going to want to copy
> far more data than will fit in memory. That means I will have to use a
> loop to read and write the data in chunks. OK, that's still not a
> problem. But it does raise the question of how big to make the buffer.
> My gut instinct tells me to make the buffer as big as possible, but
> I'm not really sure.
>
> I'm hoping there's a way to avoid this problem, altogether. Is it
> possible to "connect" an ifstream and an ofstream? Conceptually, this
> is what I'm after:
>
> Open ifstream
> Open ofstream
> Connect ifstream to ofstream
> Seek ifstream to beginning of piece
> Read ifstream (automatically writes to ofstream)
>
> My hope is that the stream classes will allocate whatever buffers they
> need and that those buffers will be appropriately sized.
>
> I've looked through the Josuttis book a little bit. I read the section
> on connecting streams, but that seems to be for synchronization. Does
> anybody know if this is possible (and a good idea)? If not, I'm open
> to any other suggestions.
>
> Thanks in advance,
> Bill




All times are GMT. The time now is 04:04 AM.

Powered by vBulletin®. Copyright ©2000 - 2014, vBulletin Solutions, Inc.
SEO by vBSEO ©2010, Crawlability, Inc.