Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > C Programming > Re: getting perl and C working together in a way that makes sense

Reply
Thread Tools

Re: getting perl and C working together in a way that makes sense

 
 
Malcolm McLean
Guest
Posts: n/a
 
      02-01-2013
On Friday, February 1, 2013 7:19:06 AM UTC, Cal Dershowitz wrote:
>
> I think the intent of this program speaks for itself, and it cuts across
> some of the greatest standing arguments in the C world: how do you
> declare and handle the input of a string whose size you don't know?
>

You're reading the whole file as a single string, so you need a slurp() function.

This is very easy to write if you know something about the platform you're
writing on, very hard to write portably, because files can be bigger than
the total memory, or even address space. Also because there's no standard
filesize() function, though there's almost always a platform-specfic one.

 
Reply With Quote
 
 
 
 
Keith Thompson
Guest
Posts: n/a
 
      02-01-2013
Cal Dershowitz <(E-Mail Removed)> writes:
> On 02/01/2013 01:50 AM, Malcolm McLean wrote:
>> On Friday, February 1, 2013 7:19:06 AM UTC, Cal Dershowitz wrote:
>>> I think the intent of this program speaks for itself, and it cuts across
>>> some of the greatest standing arguments in the C world: how do you
>>> declare and handle the input of a string whose size you don't know?
>>>

>> You're reading the whole file as a single string, so you need a
>> slurp() function.
>>
>> This is very easy to write if you know something about the platform you're
>> writing on, very hard to write portably, because files can be bigger than
>> the total memory, or even address space. Also because there's no standard
>> filesize() function, though there's almost always a platform-specfic one.

>
> right Malcolm:
>
> use File::Slurp;
>
> It solves the problem of filelength generally, as far as C is concerned.
>
> You have such an insight.


I'm sure that, by "a slurp() function", Malcolm meant a function
*written in C* that reads the contents of a file. He didn't mention
Perl's File::Slurp package.

The main problem is determining how big the buffer needs to be. Most
systems have a non-portable way to determine the size of a file; the
standard doesn't provide a portable way to do that, other than by
reading the entire file. And the file size can change between the time
you determine it and the time you read it. You can implement an
expanding buffer with realloc(), but that can use memory ineffiently.

--
Keith Thompson (The_Other_Keith) http://www.velocityreviews.com/forums/(E-Mail Removed) <http://www.ghoti.net/~kst>
Working, but not speaking, for JetHead Development, Inc.
"We must do something. This is something. Therefore, we must do this."
-- Antony Jay and Jonathan Lynn, "Yes Minister"
 
Reply With Quote
 
 
 
 
BartC
Guest
Posts: n/a
 
      02-01-2013
"Keith Thompson" <(E-Mail Removed)> wrote in message
news:(E-Mail Removed)...


> The main problem is determining how big the buffer needs to be. Most
> systems have a non-portable way to determine the size of a file; the
> standard doesn't provide a portable way to do that, other than by
> reading the entire file. And the file size can change between the time
> you determine it and the time you read it.


The file can change in size and content even while you're trying to read it
from start to finish.

But if you had to worry about all the possibilities, working with files
becomes near impossible.

Sometimes, you just have to assume that that 10-line configuration text file
you're trying to read into memory is going to be better behaved than a
multi-gigabyte continuously-updated database file on some remote network.

--
Bartc

 
Reply With Quote
 
Keith Thompson
Guest
Posts: n/a
 
      02-01-2013
"BartC" <(E-Mail Removed)> writes:
> "Keith Thompson" <(E-Mail Removed)> wrote in message
> news:(E-Mail Removed)...
>
>> The main problem is determining how big the buffer needs to be. Most
>> systems have a non-portable way to determine the size of a file; the
>> standard doesn't provide a portable way to do that, other than by
>> reading the entire file. And the file size can change between the time
>> you determine it and the time you read it.

>
> The file can change in size and content even while you're trying to read it
> from start to finish.
>
> But if you had to worry about all the possibilities, working with files
> becomes near impossible.
>
> Sometimes, you just have to assume that that 10-line configuration text file
> you're trying to read into memory is going to be better behaved than a
> multi-gigabyte continuously-updated database file on some remote network.


Unless that assumption permits an attacker to break your system's
security by violating it.

--
Keith Thompson (The_Other_Keith) (E-Mail Removed) <http://www.ghoti.net/~kst>
Working, but not speaking, for JetHead Development, Inc.
"We must do something. This is something. Therefore, we must do this."
-- Antony Jay and Jonathan Lynn, "Yes Minister"
 
Reply With Quote
 
Les Cargill
Guest
Posts: n/a
 
      02-02-2013
Keith Thompson wrote:
> Cal Dershowitz <(E-Mail Removed)> writes:
>> On 02/01/2013 01:50 AM, Malcolm McLean wrote:
>>> On Friday, February 1, 2013 7:19:06 AM UTC, Cal Dershowitz wrote:
>>>> I think the intent of this program speaks for itself, and it cuts across
>>>> some of the greatest standing arguments in the C world: how do you
>>>> declare and handle the input of a string whose size you don't know?
>>>>
>>> You're reading the whole file as a single string, so you need a
>>> slurp() function.
>>>
>>> This is very easy to write if you know something about the platform you're
>>> writing on, very hard to write portably, because files can be bigger than
>>> the total memory, or even address space. Also because there's no standard
>>> filesize() function, though there's almost always a platform-specfic one.

>>
>> right Malcolm:
>>
>> use File::Slurp;
>>
>> It solves the problem of filelength generally, as far as C is concerned.
>>
>> You have such an insight.

>
> I'm sure that, by "a slurp() function", Malcolm meant a function
> *written in C* that reads the contents of a file. He didn't mention
> Perl's File::Slurp package.
>
> The main problem is determining how big the buffer needs to be. Most
> systems have a non-portable way to determine the size of a file; the
> standard doesn't provide a portable way to do that, other than by
> reading the entire file.


The 'C' library provides two entry points, ftell() and fseek().
For any random file system, they may not actually work reasonably
( especially for that 9 track tape drive ) but they are quite standard
and if it's a spinning disk or a FLASH drive, they almost certainly will.

So there exists a relatively simple pattern for this. Will they
work on the filesystem before you? I recommend testing.


> And the file size can change between the time
> you determine it and the time you read it.


Oh well.

> You can implement an
> expanding buffer with realloc(), but that can use memory ineffiently.
>


--
Les Cargill
 
Reply With Quote
 
Bart van Ingen Schenau
Guest
Posts: n/a
 
      02-02-2013
On Fri, 01 Feb 2013 23:50:11 -0600, Les Cargill wrote:

> Keith Thompson wrote:
>> The main problem is determining how big the buffer needs to be. Most
>> systems have a non-portable way to determine the size of a file; the
>> standard doesn't provide a portable way to do that, other than by
>> reading the entire file.

>
> The 'C' library provides two entry points, ftell() and fseek(). For any
> random file system, they may not actually work reasonably ( especially
> for that 9 track tape drive ) but they are quite standard and if it's a
> spinning disk or a FLASH drive, they almost certainly will.


Except that, according to the C standard, the result from ftell() on a
text file does not have any meaning beyond "this is a position you can
seek back to". It does not have to be related to the file size in any
meaningful way.
And fseek(0, SEEK_END, fp) is not required to work on a binary file.
So, the combination of fseek and ftell guaranteed to work in determining
how large a file is.

>
> So there exists a relatively simple pattern for this. Will they work on
> the filesystem before you? I recommend testing.


That might tell you if it works on the platforms you tested. I does not
tell you anything about the platforms you did not test.

Bart v Ingen Schenau
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Re: getting perl and C working together in a way that makes sense Jens Schweikhardt C Programming 24 02-04-2013 08:12 PM
Re: getting perl and C working together in a way that makes sense Jorgen Grahn C Programming 0 02-02-2013 09:55 AM
Re: getting perl and C working together in a way that makes sense Mark Bluemel C Programming 2 02-01-2013 06:58 PM
Re: getting perl and C working together in a way that makes sense Keith Thompson C Programming 0 02-01-2013 04:14 PM
Re: getting perl and C working together in a way that makes sense Johann Klammer C Programming 0 02-01-2013 08:30 AM



Advertisments