Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > C Programming > file of exact size

Reply
Thread Tools

file of exact size

 
 
Wade Ward
Guest
Posts: n/a
 
      09-17-2007


"Erik Trulsson" <(E-Mail Removed)> wrote in message
news:(E-Mail Removed)...
> Wade Ward <(E-Mail Removed)> wrote:
>>
>>
>>
>> "Erik Trulsson" <(E-Mail Removed)> wrote in message
>> news:(E-Mail Removed)...
>>
>>> If I had wanted to restrict myself to C99 you could use a single count
>>> variable of
>>> type 'long long' and just a single loop instead of two, but I did it
>>> this
>>> way
>>> to be portable to implementations not supporting 'long long' as well.

>> Doing it the way you did, you made the program readable.

>
> If I had used 'long long' and a single loop, then the program would have
> been
> even more readable - but less portable.

bla bla bla ginger

>>> It might also be worth noting that writing a single character at a time
>>> is
>>> not the most
>>> efficient way of doing it - but improving that is left as an exercise
>>> for
>>> the reader.

It's important that this character be both in source and execution.

>>> Modifying the program so that it can be used to create a file with an
>>> arbitrary size
>>> is also left as an exercise.
>>>
>>> Oh, and the fclose() calls in the program are not actually necessary -
>>> see
>>> if you can
>>> figure out why.

>> Because windows doesn't care.

>
> No, nothing to do with Windows or any implementation-specific behaviour.
> The answer is simply that when a program exits normally (by returning from
> main() or
> by calling exit() ) then any open files will be flushed and closed
> automatically.
> (If a program exits abnormally - e.g. by calling abort(), then the C
> standard does
> not guarantee that files will be close correctly. There is also no
> guarantee that
> malloc()ed memory will be released upon exit of a program - although it is
> on

I have a different guarantee that said memory is freed. windows, dude.
>> Tip your waitresses. Today is our ½ St. Pats day, and I'm not sauced.
>> If
>> anyone wants to help me with tensor multiplication on a computer Walter,
>> please comment in c.l.f.

If it's holiday for michael, it's a holiday for those who know the
archangel.

>
> --
> <Your kiss is on my list, weather chick I cant resist going crazy inform
> Bernie the lobo howls.>
> Erik Trulsson
> http://www.velocityreviews.com/forums/(E-Mail Removed)

--
Wade Ward
(E-Mail Removed)
"Ich denk an Dich
und laß 'nen fliegen."
--Nena, announcing that she just farted


 
Reply With Quote
 
 
 
 
Wade Ward
Guest
Posts: n/a
 
      09-17-2007
C_Dreamer. killfile 1.

--
Wade Ward
(E-Mail Removed)
"Ich denk an Dich
und laß 'nen fliegen."
--Nena, announcing that she just farted
"Keith Thompson" <(E-Mail Removed)> wrote in message
news:(E-Mail Removed)...
> "Wade Ward" <(E-Mail Removed)> writes:
>> "Keith Thompson" <(E-Mail Removed)> wrote in message
>> news:(E-Mail Removed)...
>>> "Wade Ward" <(E-Mail Removed)> writes:
>>>> "Keith Thompson" <(E-Mail Removed)> wrote in message
>>>> news:(E-Mail Removed)...
>>>>> Why do you want to do this? Creating a file of an exact specified
>>>>> size without saying anything about its contents seems like a very odd
>>>>> requirement.
>>>> What do you mean? When you're erasing information you get it right to
>>>> the
>>>> byte.
>>> [...]
>>>
>>> How does creating a file erase information?

>> A lot of it has to do with rules of evidence. I think I can type for
>> five
>> paragraphs, so I'm going for it.
>>
>> Kenny is playing. I like having peoplearound who are by orders of
>> magnitude
>> drunker than I. Kenny actually should be the poster child for liver
>> health.
>> It is when people imbibe under dessert(sic) conditions where the danger
>> enters. The answer to 99% of your questions is "water."

> [snip more of the same]
>
> Creating a file does not erase information, unless it does so
> incidentally by clobbering some existing file. If you wanted to ask
> about how to erase information, you should have done so rather than
> wasting our time with irrelevancies.
>
> You recently claimed that I had called you a troll, when in fact I'm
> reasonably sure that I had never done so. Ok, fine. You're a troll.
> Don't expect any help from me (or, quite likely, from anyone else
> here) until and unless you stop trolling. I suspect that this may
> require you to post only while sober, but I don't care.
>
> Bye.
>
> --
> Keith Thompson (The_Other_Keith) (E-Mail Removed)
> <http://www.ghoti.net/~kst>
> San Diego Supercomputer Center <*>
> <http://users.sdsc.edu/~kst>
> "We must do something. This is something. Therefore, we must do this."
> -- Antony Jay and Jonathan Lynn, "Yes Minister"



 
Reply With Quote
 
 
 
 
Charlie Gordon
Guest
Posts: n/a
 
      09-17-2007
"Tor Rustad" <(E-Mail Removed)> a écrit dans le message de
news: (E-Mail Removed)...
> Charlie Gordon wrote:
>> "Richard Heathfield" <(E-Mail Removed)> a écrit dans le message de
>> news: (E-Mail Removed)...
>>> Peter J. Holzer said:
>>>
>>>> On 2007-09-15 12:32, Charlie Gordon <(E-Mail Removed)> wrote:
>>>>> "Wade Ward" <(E-Mail Removed)> a ?rit dans le message de news:
>>>>> (E-Mail Removed)...
>>> <snip>
>>>>>> The benchmark to beat is eight minutes. Sleep on it.
>>>>>>
>>>> [...]
>>>>> given todays average harware performance, 1 minute seems a good goal
>>>>> for this benchmark.
>>>>> using fwrite with a decent buffer size should do it.
>>>> % ./heathfield > foo
>>>> please wait
>>>> ./heathfield > foo 45.09s user 6.26s system 79% cpu 1:04.24 total
>>>>
>>>> Looks like Richard's program reaches the goal even without fwrite.
>>> Gosh! That makes me wonder how fast I could make it if I were
>>> actually trying to make it fast.
>>>
>>> But of course it doesn't quite make me wonder *enough*...

>>
>> Don't get carried away, if you are writing actual bytes to an actual hard
>> drive without tricks such as compression, delayed commit... The speed
>> limit is the drive throughput which averages 40-50 MB per second these
>> days.

>
> Writing ~2 Gb to a file, is not exactly a fast method:
>
> # cat mr_big.c
> #include <unistd.h>
>
> int main(void)
> {
> int rc;
>
> rc = truncate("/big", 2282899UL);
>
> if (rc)
> perror("/big");
>
> return rc;
> }
>
> # touch /big
> # time ./a.out
>
> real 0m0.002s
> user 0m0.000s
> sys 0m0.004s
>
>
> so ISO C, is clearly not the best tool around for this job.


This solution for POSIX systems has been mentioned already.
Your program does not make a 2+GB file, but a small 2.2 megabyte one.
Changing the number to the requested size will not work on most 32 bit
systems.
Most unix users cannot create files in the root directory either, and don't
program as root.

--
Chqrlie.


 
Reply With Quote
 
Army1987
Guest
Posts: n/a
 
      09-17-2007
On Sat, 15 Sep 2007 10:48:14 +0000, Erik Trulsson wrote:

> Wade Ward <(E-Mail Removed)> wrote:
>> How do I use the C Programming Language to create a files that is 2563695577
>> bytes. The one line I think to know in this program is:
>> long long m = 2563695577;

[...]
> int res;
> long count1=2147483647; /* The largest value a 'long' is guaranteed to hold */
> long count2= 416211930; /* 416211930 == 2563695577 - 2147483647 */

You could use unsigned long...

--
Army1987 (Replace "NOSPAM" with "email")
If you're sending e-mail from a Windows machine, turn off Microsoft's
stupid “Smart Quotes” feature. This is so you'll avoid sprinkling garbage
characters through your mail. -- Eric S. Raymond and Rick Moen

 
Reply With Quote
 
Erik Trulsson
Guest
Posts: n/a
 
      09-17-2007
Army1987 <(E-Mail Removed)> wrote:
> On Sat, 15 Sep 2007 10:48:14 +0000, Erik Trulsson wrote:
>
>> Wade Ward <(E-Mail Removed)> wrote:
>>> How do I use the C Programming Language to create a files that is 2563695577
>>> bytes. The one line I think to know in this program is:
>>> long long m = 2563695577;

> [...]
>> int res;
>> long count1=2147483647; /* The largest value a 'long' is guaranteed to hold */
>> long count2= 416211930; /* 416211930 == 2563695577 - 2147483647 */

> You could use unsigned long...


That would be too simple...


--
<Insert your favourite quote here.>
Erik Trulsson
(E-Mail Removed)
 
Reply With Quote
 
Tor Rustad
Guest
Posts: n/a
 
      09-17-2007
Charlie Gordon wrote:
> "Tor Rustad" <(E-Mail Removed)> a écrit dans le message de


[...]

>> # touch /big
>> # time ./a.out
>>
>> real 0m0.002s
>> user 0m0.000s
>> sys 0m0.004s
>>
>>
>> so ISO C, is clearly not the best tool around for this job.

>
> This solution for POSIX systems has been mentioned already.


So? I timed it.

> Your program does not make a 2+GB file, but a small 2.2 megabyte one.


Ooops, that number was a copy-paste from another post, I didn't pay
attention to it! However, creating a 2 Mb or a 2 Gb file with this
method, shouldn't make much of a time difference:

# time ./mr_big
File '/big' of 4Gb has been created

real 0m0.002s
user 0m0.000s
sys 0m0.000s
# ls -l /big
-rw-r--r-- 1 root root 4294967295 2007-09-17 20:04 /big

right?

> Changing the number to the requested size will not work on most 32 bit
> systems.


On a 32-bit system, I used

gcc -D_FILE_OFFSET_BITS=64 mr_big.c -o mr_big


of course, you need file system support for >2 Gb files too.


> Most unix users cannot create files in the root directory either, and don't
> program as root.


Most UNIX users... isn't a Norse God.

--
Tor <torust [at] online [dot] no>
 
Reply With Quote
 
user923005
Guest
Posts: n/a
 
      09-17-2007
On Sep 15, 12:32 am, "Wade Ward" <(E-Mail Removed)> wrote:
> How do I use the C Programming Language to create a files that is 2563695577
> bytes. The one line I think to know in this program is:
> long long m = 2563695577;
>
> EOF on my implementation is carriage return line feed (I think). I don't
> know if that is the 2563695578th byte or even the 2563695578th and the
> 2563695579th.
>
> The number in question is about two and a half billion. I can't remember
> what the minumum maximum is for this datatype, but I think I'm within an
> order of magnitude. Telling me to read the manual won't work, because I
> don't have a c compiler.


#include <stdio.h>
#include <stdlib.h>
/*
C:\tmp>factor 2563695577
first trying brute force division by small primes
PRIME FACTOR 1123
PRIME FACTOR 2282899
*/
static const unsigned char gunk[2282899]={0};
static unsigned char buf[4194304]={0};

int main(void)
{
size_t index;
FILE *f = fopen("gunk.dat", "wb");
if (f) {
if (setvbuf(f, buf, _IOFBF, sizeof(buf)) != 0)
printf("Incorrect type or size of buffer for gunk.dat
(default unchanged)\n");
else
printf("gunk.dat has a buffer of %u bytes\n", (unsigned)
sizeof buf);
for (index = 0; index < 1123; index++) {
size_t written = fwrite(gunk, 1, sizeof gunk, f);
if (written != sizeof gunk) {
perror("Failure creating file\n");
exit(EXIT_FAILURE);
}
}
} else {
perror("failed to open file gunk.dat\n");
exit(EXIT_FAILURE);
}
fclose(f);
return 0;
}

 
Reply With Quote
 
Army1987
Guest
Posts: n/a
 
      09-17-2007
On Sat, 15 Sep 2007 00:32:14 -0700, Wade Ward wrote:

> How do I use the C Programming Language to create a files that is 2563695577
> bytes.

army1987@army1987-laptop:~$ cat largefile.c
#include <stdio.h>
#include <string.h>
#include <errno.h>
#include <stdlib.h>
#define SIZE 2563695577UL
int main(int argc, char *argv[])
{
unsigned long remaining = SIZE;
unsigned long result = 0;
unsigned char buffer[BUFSIZ];
FILE *out;

if (argc >= 2) {
out = fopen(argv[1], "w");
if (out == NULL) {
fprintf(stderr, "Cannot open '%s' for writing: %s\n",
argv[1], strerror(errno));
return EXIT_FAILURE;
}
} else
out = stdout;
memset(buffer, '_', BUFSIZ);
while (remaining > BUFSIZ) {
result += fwrite(buffer, 1, BUFSIZ, out);
remaining -= BUFSIZ;
}
result += fwrite(buffer, 1, remaining, out);
result += (putc('\n', out) != EOF);
fprintf(stderr, "%lu bytes output.\n", result);
if (fclose(out) != 0) {
perror("Error while closing file");
result = 0;
}
return (result == SIZE + 1) ? EXIT_SUCCESS : EXIT_FAILURE;
}
army1987@army1987-laptop:~$ time ./largefile lf
File size limit exceeded (core dumped)

real 1m25.936s
user 0m0.484s
sys 0m4.968s
army1987@army1987-laptop:~$ time rm lf

real 0m0.747s
user 0m0.000s
sys 0m0.208s
army1987@army1987-laptop:~$ time ./largefile > lf
2563695578 bytes output.

real 1m31.676s
user 0m0.692s
sys 0m5.680s
army1987@army1987-laptop:~$ time rm lf

real 0m2.693s
user 0m0.000s
sys 0m0.256s
army1987@army1987-laptop:~$

Note how 1) by writing to stdout I didn't encounter the limit, and
2) the system time to remove the file is proportional to the size,
but the total time for the file larger than LONG_MAX is about
three times larger than to delete the "small" file, 3) writing to
stdout is 26.67 MB/s, whereas writing to a fopen()ed stream is
23.83 MB/s. (I don't feel like doing tens of tests to see whether
that's just a statistical fluctuation, and if I did the results
would be probably OT here.)

> EOF on my implementation is carriage return line feed (I think).

It isn't (unless by EOF you mean the last character of the file,
which is a very improper meaning).
--
Army1987 (Replace "NOSPAM" with "email")
If you're sending e-mail from a Windows machine, turn off Microsoft's
stupid “Smart Quotes” feature. This is so you'll avoid sprinkling garbage
characters through your mail. -- Eric S. Raymond and Rick Moen

 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Using exact-size structs to go thru raw byte buffers toe@lavabit.com C Programming 5 02-22-2008 11:04 PM
mega pixels, file size, image size, and print size - Adobe Evangelists Frank ess Digital Photography 0 11-14-2006 05:08 PM
New window - exact size... Domestos HTML 17 06-30-2005 04:39 PM
Specifying exact font point size bob HTML 4 06-18-2004 06:33 PM
OK you pedants - what is the *exact* size of a 35mm image? Alan F Cross Digital Photography 5 08-05-2003 10:47 PM



Advertisments