«

Morris Dovey wrote:

> Mark Bluemel wrote:
>
>> My point was that the OP had clearly been trying to produce a textual
>> representation.
>>
>> From context, I would not have expected the OP to understand why
>> Santosh's suggestion was not compatible with his/her original goal.
>
> I agree with you - that's why I wrote what you snipped:
>
>>> A special-purpose double -> text conversion routine can provide a
>>> fair amount of speed improvement.
>
> My thinking is that if speed is critical to LilacSkin, then the
> textual representation can be produced somewhat faster (perhaps
> almost twice as fast) by substituting a user-written function for
> the standard formatting routine.
>
> Once that's been done, fwrite() may indeed be an appropriate
> mechanism. I'm actually not sure whether Santosh mentally skipped
> a step or was advocating binary output.

I was responding to the following text by the OP:

<quote>
In several forums, I saw that fwrite function is better, but I don't
know how to use it.
Can you help me, please ?
</quote>

On 13 fév, 15:13, Morris Dovey <mrdo...@iedu.com> wrote:
> Mark Bluemel wrote:
> > How does dumping the data in binary form compare to writing a
> > tab-delimited textual representation?
>
> If the textual representation is being done by fprintf(), writing
> the unconverted data in binary form is _much_ faster - and, of
> course, may produce problems if there's a subsequent need to read
> the data on a different machine.
>
> A special-purpose double -> text conversion routine can provide a
> fair amount of speed improvement. A real-world example was
> developed in this CLC thread:
>
> http://groups.google.com/group/comp....d/thread/eb329...
>
> (mind the wrap)
>
> --
> Morris Dovey
> DeSoto Solar
> DeSoto, Iowa USAhttp://www.iedu.com/DeSoto

Your idea is to use your "convert" function and to fwrite the
converted buffer ?
If yes, can I cast an long long in double to directly use it ?

Thanks !

On 13 fév, 19:51, LilacSkin <lpaul...@iseb.fr> wrote:
> On 13 fév, 15:13, Morris Dovey <mrdo...@iedu.com> wrote:
>
>
>
> > Mark Bluemel wrote:
> > > How does dumping the data in binary form compare to writing a
> > > tab-delimited textual representation?
>
> > If the textual representation is being done by fprintf(), writing
> > the unconverted data in binary form is _much_ faster - and, of
> > course, may produce problems if there's a subsequent need to read
> > the data on a different machine.
>
> > A special-purpose double -> text conversion routine can provide a
> > fair amount of speed improvement. A real-world example was
> > developed in this CLC thread:
>
> >http://groups.google.com/group/comp....d/thread/eb329...
>
> > (mind the wrap)
>
> > --
> > Morris Dovey
> > DeSoto Solar
> > DeSoto, Iowa USAhttp://www.iedu.com/DeSoto
>
> Your idea is to use your "convert" function and to fwrite the
> converted buffer ?
> If yes, can I cast an long long in double to directly use it ?
>
> Thanks !

In fact the most important thing is to write faster.
I have 1,6 Mbyte/s of raw data to write during a couple of days.
Because of the amount of data, I can't reformat their after.
The data are stored in a char buffer and I need to put in a file as a
signed long long.
The size of the buzzer is 8 Mbyte, refreshed every 5 sec.

I think the best way is to format them and use a fwrite or _write
function .

LilacSkin wrote:
) In fact the most important thing is to write faster.
) I have 1,6 Mbyte/s of raw data to write during a couple of days.
) Because of the amount of data, I can't reformat their after.
) The data are stored in a char buffer and I need to put in a file as a
) signed long long.
) The size of the buzzer is 8 Mbyte, refreshed every 5 sec.

I assume 1,6 Mbyte/sec is the input speed.
Am I right that that's 200.000 numbers per second ?
I'm assuming one number takes about 16 bytes in the output file, so that's
actually 3,2 Mbyte/s to write to the file.

You can try having one thread formatting the block, and then have another
actually write the file. That way, you're not waiting for file I/O.

Here's some pseudocode for double-buffering, to give you an idea:
(Note that write() is not standard C.)

BUFSIZE = 4096;

buffer[BUFSIZE*2 + 32];
buf1 = buffer;
buf2 = buffer + BUFSIZE;
end = buffer + BUFSIZE*2;
flag = 0;

formatter_loop:
ptr = buf1;
while(1) {
while (ptr < buf2);
ptr += sprintf(ptr, "%lld\t", number);
while (flag)
/* wait */;
flag = buf1;
while (ptr < end)
ptr += sprintf(ptr, "%lld\t", number);
while (flag)
/* wait */;
flag = buf2;
memcpy(buf1, end, ptr - end);
ptr -= BUFSIZE*2;
}

writer_loop:
while(1) {
while (!flag)
/* wait */;
write(filedes, flag, BUFSIZE);
flag = 0;
}


SaSW, Willem
--
Disclaimer: I am in no way responsible for any of the statements
made in the above text. For all I know I might be
drugged or something..
No I'm not paranoid. You all think I'm paranoid, don't you !
#EOT

LilacSkin wrote:
>
> On 13 fév, 15:13, Morris Dovey <mrdo...@iedu.com> wrote:
> > Mark Bluemel wrote:
> > > How does dumping the data in binary form compare to writing a
> > > tab-delimited textual representation?
> >
> > If the textual representation is being done by fprintf(), writing
> > the unconverted data in binary form is _much_ faster - and, of
> > course, may produce problems if there's a subsequent need to read
> > the data on a different machine.
> >
> > A special-purpose double -> text conversion routine can provide a
> > fair amount of speed improvement. A real-world example was
> > developed in this CLC thread:
> >
> > http://groups.google.com/group/comp....d/thread/eb329...
> >
> > (mind the wrap)
> >
> > --
> > Morris Dovey
> > DeSoto Solar
> > DeSoto, Iowa USAhttp://www.iedu.com/DeSoto
>
> Your idea is to use your "convert" function and to fwrite the
> converted buffer ?
> If yes, can I cast an long long in double to directly use it ?

You probably won't want to use _that_ convert function because it
does more than you want (double -> string) when you're really
wanting (long long -> string).

Just throw away the part that deals with the fractional part of
the double - and if all of your values are positive, throw away
the sign logic, too. That won't leave very much - and it
shouldn't waste many cycles.

--
Morris Dovey
DeSoto Solar
DeSoto, Iowa USA
http://www.iedu.com/DeSoto

LilacSkin wrote:

> #include <stdio.h>
>
> void convert(long long v,char *s,int sz,int dp)
> { char *p = s + sz;
> long long x = 1LL;
> int sign = v < 0.0;
>
> if (sign) v = -v;
> while (dp--) x *= 10;
> x = x * v;
> *p-- = '\0';
> do
> { *p-- = '0' + (x % 10);
> } while ((p >= s) && (x /= 10));
> while (p >= s) *p-- = '0';
> if (sign) *s = '-';
> }

Pretty close. Since you're not working with a double, you can
dispense with the fractional part logic and shorten to something
like:

void convert(long long x,char *s,int sz)
{ char *p = s + sz;
int sign = x < 0;

if (sign) x = -x;
*p-- = '\0';
do
{ *p-- = '0' + (x % 10);
} while ((p >= s) && (x /= 10));
while (p >= s) *p-- = ' ';
if (sign) *s = '-';
}

--
Morris Dovey
DeSoto Solar
DeSoto, Iowa USA
http://www.iedu.com/DeSoto

#include <stdio.h>

void convert(long long v,char *s,int sz,int dp)
{ char *p = s + sz;
long long x = 1LL;
int sign = v < 0.0;

if (sign) v = -v;
while (dp--) x *= 10;
x = x * v;

*p-- = '\0';
do
{ *p-- = '0' + (x % 10);
} while ((p >= s) && (x /= 10));

while (p >= s) *p-- = '0';

if (sign) *s = '-';

}

int main(void)
{ long long test = -123456789012345;
char buffer[64];

convert(test,buffer,25,5);
printf("'%s'\n",buffer);
// then fwrite(buffer...)

return 0;

}

LilacSkin <> writes:

> #include <stdio.h>
>
> void convert(long long v,char *s,int sz,int dp)
> { char *p = s + sz;
> long long x = 1LL;
> int sign = v < 0.0;
>
> if (sign) v = -v;
> while (dp--) x *= 10;
> x = x * v;
>
> *p-- = '\0';
> do
> { *p-- = '0' + (x % 10);
> } while ((p >= s) && (x /= 10));
>
> while (p >= s) *p-- = '0';
>
> if (sign) *s = '-';
>
> }
>
> int main(void)
> { long long test = -123456789012345;
> char buffer[64];
>
> convert(test,buffer,25,5);
> printf("'%s'\n",buffer);
> // then fwrite(buffer...)
>
> return 0;
>
> }

You may want to re-consider. Numbers are usually converted to a
decimal string representation so the we (humans) can read them. If
your program is producing such a volume of data that it is hard to get
converted output fast enough, will the resulting data ever be read by
a person? I doubt it. You'd need a program to scan just one second's
worth.

It might pay to store the data as native binary numbers. That is
probably what was being suggested when someone said use "fwrite".

Of course, if the output *has* to be processed by an existing program
that needs decimal input, then the conversion has to happen some time,
but it could be done later, at leisure, so to speak. It might even be
possible to process into decimal only the portion you are interested in.
Maybe the binary data could be indexed for rapid access to specific
parts. Without knowing the ultimate fate of this data, it is hard to
be more specific, but I suggest you look beyond your current headache
of not being able to use fprintf because it seems too slow.

--
Ben.

LilacSkin wrote:

> I have 1,6 Mbyte/s of raw data to write during a couple of days.
> Because of the amount of data, I can't reformat their after.
> The data are stored in a char buffer and I need to put in a file as a
> signed long long.

So why were you even trying to use printf, which is for formatted
output?

If you have to write to the file as signed long long, then fwrite is
appropriate.

You really could help us to help you if you explained the requirement
better.

On 14 fév, 09:59, Mark Bluemel <mark_blue...@pobox.com> wrote:
> LilacSkin wrote:
> > I have 1,6 Mbyte/s of raw data to write during a couple of days.
> > Because of the amount of data, I can't reformat their after.
> > The data are stored in a char buffer and I need to put in a file as a
> > signed long long.
>
> So why were you even trying to use printf, which is for formatted
> output?
>
> If you have to write to the file as signed long long, then fwrite is
> appropriate.
>
> You really could help us to help you if you explained the requirement
> better.

I need to write the data in text file !
Morris, I try your code, the problem is, it puts a space character
before a positive number.