«

On Thu, 20 Oct 2011 16:32:06 -0400, Michael Angelo Ravera
<> wrote:

> The reason that you code fails is that you can't generally realloc what
> you haven't [m|c|0]alloc'd.

Wrong. From the C99 standard, ISO/IEC 9899:1999 (E)

7.20.3.4 The realloc function

Synopsis
#include <stdlib.h>
void *realloc(void *ptr, size_t size);
....
If ptr is a null pointer, the realloc function behaves like the
malloc function for the specified size.

--
Morris Keesan --

"Malcolm McLean" <> wrote in message
news:e3302961-696e-4b9f-bc77-...
> On Oct 20, 3:31 pm, "BartC" <b...@freeuk.com> wrote:
>>
>> Someone doing 'ReadLine' is probably interested in line-oriented files.
>>
>> A file containing with a single multi-GB line is likely not
>> line-oriented!
>> Or is not using the expected newline sequence.
>>
> The problem is that it is impossible to define a reasonable line
> length.

A good guide might be how long you are prepared to spend scrolling from one
end to another.

My rough calculations show that a line 30,000 characters long would take
about a minute to scroll from one end to the other (paging a full width,
twice a second). 30,000 characters seems perfectly reasonable (I use 2048
myself and rarely have problems).

(30,000 characters also requires a virtual screen 200 feet wide. A line
buffer 1 billion characters long requires a virtual screen some 1300 miles
across and would take 3 weeks to scroll. No-one can possibly pretend that
that is still one line of text!)

If a limit of a few thousand characters is not enough, then a different
approach is needed; something in between ReadLine and ReadFile; ReadBlock
perhaps, as the file cannot be considered line-oriented, not in the
conventional, human-readable sense.

--
Bartc

"BartC" <> writes:
> "Malcolm McLean" <> wrote in message
> news:e3302961-696e-4b9f-bc77-...
>> On Oct 20, 3:31 pm, "BartC" <b...@freeuk.com> wrote:
>>>
>>> Someone doing 'ReadLine' is probably interested in line-oriented files.
>>>
>>> A file containing with a single multi-GB line is likely not
>>> line-oriented!
>>> Or is not using the expected newline sequence.
>>>
>> The problem is that it is impossible to define a reasonable line
>> length.
>
> A good guide might be how long you are prepared to spend scrolling from one
> end to another.
[snip]
> If a limit of a few thousand characters is not enough, then a different
> approach is needed; something in between ReadLine and ReadFile; ReadBlock
> perhaps, as the file cannot be considered line-oriented, not in the
> conventional, human-readable sense.

Text isn't necessarily meant to be human-readable.

--
Keith Thompson (The_Other_Keith) kst- <http://www.ghoti.net/~kst>
"We must do something. This is something. Therefore, we must do this."
-- Antony Jay and Jonathan Lynn, "Yes Minister"

On 10/21/11 11:34 AM, Keith Thompson wrote:
> "BartC"<> writes:
>> "Malcolm McLean"<> wrote in message
>> news:e3302961-696e-4b9f-bc77-...
>>> On Oct 20, 3:31 pm, "BartC"<b...@freeuk.com> wrote:
>>>>
>>>> Someone doing 'ReadLine' is probably interested in line-oriented files.
>>>>
>>>> A file containing with a single multi-GB line is likely not
>>>> line-oriented!
>>>> Or is not using the expected newline sequence.
>>>>
>>> The problem is that it is impossible to define a reasonable line
>>> length.
>>
>> A good guide might be how long you are prepared to spend scrolling from one
>> end to another.
> [snip]
>> If a limit of a few thousand characters is not enough, then a different
>> approach is needed; something in between ReadLine and ReadFile; ReadBlock
>> perhaps, as the file cannot be considered line-oriented, not in the
>> conventional, human-readable sense.
>
> Text isn't necessarily meant to be human-readable.

Indeed. The longest single "line" I have in a file is 1.3GB (a JSON
representation of a rather large filesystem).

--
Ian Collins

On Thu, 2011-10-20, Ben Bacarisse wrote:
> Markus Wichmann <> writes:
....

> I am not sure how you decided that this suits the OP's use case. I
> think the original posted code was just a cut down example, so it's not
> obvious that silently splitting lines that are longer than 255
> characters suits the OP.
>
>> No dynamic allocation where it isn't necessary. It is not really
>> necessary in I/O, because there is never a buffer big enough for that,
>> so don't program as though there were!
>
> I think you'd be annoyed if, say, awk behaved like this. Given a big
> enough buffer, your shell might get away with it, but I expect it
> doesn't try.

I'm too lazy to look for the context, but I assume you are talking
about varying input line lengths here.

I think it's a bit more complicated than that. Assuming some
line-oriented input format:

- It's usually wrong to set a limit (say, 8192 bytes) and pretend
anything longer is two or more lines.

- It's usually wrong to accept *any* length by using malloc()/realloc(),
and then start swapping and crashing when someone feeds you a
10GB line.

I assume a decent awk sets some limit (far higher than any reasonable
input) and exits gracefully with an error message when it gets
anything larger.

- Many file/data formats put a limit to the line length anyway.
NNTP, SMTP. IIRC also the C language says a compiler is allowed
to bail out on lines longer than N characters.

/Jorgen

--
// Jorgen Grahn <grahn@ Oo o. . .
\X/ snipabacken.se> O o .

"Keith Thompson" <kst-> wrote in message
news:...
> "BartC" <> writes:

>> If a limit of a few thousand characters is not enough, then a different
>> approach is needed; something in between ReadLine and ReadFile; ReadBlock
>> perhaps, as the file cannot be considered line-oriented, not in the
>> conventional, human-readable sense.
>
> Text isn't necessarily meant to be human-readable.

In that case there should be no pretence that a ReadLine function actually
reads a 'line' at all, but any sequence of characters of any length.

That's why I suggested such a function should be called Readblock, with all
the problems that that would introduce (such as possibly bringing down the
system in the case of a rogue file with 'lines' too long to fit into either
physical or virtual memory).

Then 'ReadLine' can be used for the straightforward cases such as reading
configuration files or command lines. Lines which are too long are reported
as errors (which in all probability they are).

--
Bartc

Jorgen Grahn <grahn+> writes:
>- It's usually wrong to set a limit (say, 8192 bytes) and pretend
> anything longer is two or more lines.

Untested:

int readline
( FILE * const source,
int( * const target )( int ch, void * obj ),
void * const object )
{ int c; int result = 1; while( result == 1 )
{ c = fgetc( source ); result =
c == EOF ? 2 :
ferror( source )? 3 :
c == '\n' ? 0 :
target( c, object )? 4 : result; }
return result; }

On Oct 21, 11:30*am, "BartC" <b...@freeuk.com> wrote:
> "Keith Thompson" <ks...@mib.org> wrote in message
>
> > Text isn't necessarily meant to be human-readable.
>
> In that case there should be no pretence that a ReadLine function actually
> reads a 'line' at all, but any sequence of characters of any length.
>
There's a difference between human-inspectable and human-readable.

In the case of my program, no-one can really make sense of a matrix
with 1000 factors and 6000 cases. However you can certainly look at it
to see that the factors have a reasonable balance of set and missing,
the labels are right, there's the right number of cases (one per
gene), and so on.
--
Visit my website
http://www.malcolmmclean.site11.com/www

"BartC" <> writes:
> "Keith Thompson" <kst-> wrote in message
> news:...
>> "BartC" <> writes:
>
>>> If a limit of a few thousand characters is not enough, then a different
>>> approach is needed; something in between ReadLine and ReadFile; ReadBlock
>>> perhaps, as the file cannot be considered line-oriented, not in the
>>> conventional, human-readable sense.
>>
>> Text isn't necessarily meant to be human-readable.
>
> In that case there should be no pretence that a ReadLine function
> actually reads a 'line' at all, but any sequence of characters of any
> length.
>
> That's why I suggested such a function should be called Readblock,
> with all the problems that that would introduce (such as possibly
> bringing down the system in the case of a rogue file with 'lines' too
> long to fit into either physical or virtual memory).
>
> Then 'ReadLine' can be used for the straightforward cases such as
> reading configuration files or command lines. Lines which are too long
> are reported as errors (which in all probability they are).

A block of characters terminated by a newline character is what we call
a "line".

--
Keith Thompson (The_Other_Keith) kst- <http://www.ghoti.net/~kst>
"We must do something. This is something. Therefore, we must do this."
-- Antony Jay and Jonathan Lynn, "Yes Minister"

(Stefan Ram) writes:

> Jorgen Grahn <grahn+> writes:
>>- It's usually wrong to set a limit (say, 8192 bytes) and pretend
>> anything longer is two or more lines.
>
> Untested:
>
> int readline
> ( FILE * const source,
> int( * const target )( int ch, void * obj ),
> void * const object )
> { int c; int result = 1; while( result == 1 )
> { c = fgetc( source ); result =
> c == EOF ? 2 :
> ferror( source )? 3 :
> c == '\n' ? 0 :
> target( c, object )? 4 : result; }
> return result; }

Unread.

--
Ben.