On 2007-09-15 22:25, Flash Gordon <> wrote:
> Peter J. Holzer wrote, On 15/09/07 22:05:
>> On 2007-09-15 18:12, Flash Gordon <> wrote:
>>> Charlie Gordon wrote, On 15/09/07 13:32:
>>>> "Wade Ward" <> a écrit dans le message de news:
>>>> ...
>>
>> [creating a file of ca. 2.5 GB ]
>>
>>>>> The benchmark to beat is eight minutes. Sleep on it.
>> [...]
>>>> given todays average harware performance, 1 minute seems a good goal for
>>>> this benchmark.
>>>> using fwrite with a decent buffer size should do it.
>>> Pick the right system and the right method and I would expect closer to
>>> 1s. Actually, I would expect a lot *less* than 1s. E.g.
>>>
>>> markg@brenda:~$ rm /tmp/big
>>> markg@brenda:~$ time ./a.out
>>>
>>> real 0m0.002s
>>> user 0m0.000s
>>> sys 0m0.000s
>>> markg@brenda:~$ ls -l /tmp/big
>>> -rw-r--r-- 1 markg markg 2282899 2007-09-15 19:02 /tmp/big
>>> markg@brenda:~$
>>>
>>> For the earlier mentioned size of 2563695577 my method did not work,
>>
>> That's a bit of a cop-out. If your file is 1123 times smaller one would
>> expect your program to be 1123 times faster.> In fact, the difference
>> can be expected to be even larger since 2.2 MB fit comfortably into the
>> buffer cache of just about any contamporary PC, but 2.5 GB do not - so
>> you are hitting only main memory while Richard's program is hitting the
>> (much slower) disk.
>
> The trick I used meant it did not write much at all to the disk. If I
> bothered to sort things out so that it produced a 1TB file it would
> still take the same time.
I guessed that. But since you didn't disclose your "trick" even though
it probably has already been mentioned in this thread, your quoted
timing doesn't provide much evidence.
>>> and for various reasons (including portability) my method might not be
>>> suitable to the OP. I did not use standard C to do this so the code is
>>> not topical here.
>>
>> One method which uses only standard C and which is much faster on some
>> systems has already been mentioned: fseeking to the desired position and
>> writing one byte. However, fseeking beyond LONG_MAX may not be
>> supported.
>
> The method I was using allowed specifying a 64 bit offset, but at the
> size specified it failed so it is possible that there is another limit
> being imposed on the file size.
Accessing files larger than 2GB is a bit tricky on most 32 bit systems.
Typically you have to use a special function (open64 or fopen64) to open
them or at least pass a special flag or compile the program in a special
way. For example, on Linux (or any other system using the glibc), the
following program which uses only standard C functions doesn't work if
compiled with "gcc foo.c -o foo", but does work if compiled with "gcc
-D_FILE_OFFSET_BITS=64 foo.c -o foo":
#include <errno.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
int main(void)
{
int errupt = 0;
unsigned long size = 2563695577;
long o1 = size/2;
long o2 = size - o1 - 1;
FILE *fp;
if ((fp = fopen("bigfile", "wb")) == NULL) {
fprintf(stderr, "%s: cannot open %s: %s\n",
__FILE__, "bigfile", strerror(errno));
exit(1);
}
if (fseek(fp, o1, SEEK_SET) != 0) {
fprintf(stderr, "%s: cannot fseek to position %ld, SEEK_SET: %s\n",
__FILE__, o1, strerror(errno));
exit(1);
}
if (fseek(fp, o2, SEEK_CUR) != 0) {
fprintf(stderr, "%s: cannot fseek to position %ld, SEEK_CUR: %s\n",
__FILE__, o2, strerror(errno));
exit(1);
}
if (putc('x', fp) == EOF) {
fprintf(stderr, "%s: cannot write 1 char: %s\n",
__FILE__, strerror(errno));
exit(1);
}
if (fclose(fp) == EOF) {
fprintf(stderr, "%s: cannot close file: %s\n",
__FILE__, strerror(errno));
exit(1);
}
return 0;
}
Alternatively I could have used fopen64 instead of fopen. Or ditching
stdio altogether and using POSIX I/O functions ater opening the file
with open64 or open(... O_LARGEFILE ...).
hp
--
_ | Peter J. Holzer | I know I'd be respectful of a pirate
|_|_) | Sysadmin WSR | with an emu on his shoulder.
| | |
|
__/ |
http://www.hjp.at/ | -- Sam in "Freefall"
Similar Threads
That makes me wonder how fast I could make it if I were
