Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > C Programming > size_t problems

Reply
Thread Tools

size_t problems

 
 
jacob navia
Guest
Posts: n/a
 
      08-29-2007
I am trying to compile as much code in 64 bit mode as
possible to test the 64 bit version of lcc-win.

The problem appears now that size_t is now 64 bits.

Fine. It has to be since there are objects that are more than 4GB
long.

The problem is, when you have in thousands of places

int s;

// ...
s = strlen(str) ;

Since strlen returns a size_t, we have a 64 bit result being
assigned to a 32 bit int.

This can be correct, and in 99.9999999999999999999999999%
of the cases the string will be smaller than 2GB...

Now the problem:

Since I warn each time a narrowing conversion is done (since
that could loose data) I end up with hundreds of warnings each time
a construct like int a = strlen(...) appears. This clutters
everything, and important warnings go lost.


I do not know how to get out of this problem. Maybe any of you has
a good idea? How do you solve this when porting to 64 bits?

jacob
 
Reply With Quote
 
 
 
 
Malcolm McLean
Guest
Posts: n/a
 
      08-29-2007

"jacob navia" <(E-Mail Removed)> wrote in message
news:46d5c46d$0$5108$(E-Mail Removed)...
>I am trying to compile as much code in 64 bit mode as
> possible to test the 64 bit version of lcc-win.
>
> The problem appears now that size_t is now 64 bits.
>
> Fine. It has to be since there are objects that are more than 4GB
> long.
>
> The problem is, when you have in thousands of places
>
> int s;
>
> // ...
> s = strlen(str) ;
>
> Since strlen returns a size_t, we have a 64 bit result being
> assigned to a 32 bit int.
>
> This can be correct, and in 99.9999999999999999999999999%
> of the cases the string will be smaller than 2GB...
>
> Now the problem:
>
> Since I warn each time a narrowing conversion is done (since
> that could loose data) I end up with hundreds of warnings each time
> a construct like int a = strlen(...) appears. This clutters
> everything, and important warnings go lost.
>
>
> I do not know how to get out of this problem. Maybe any of you has
> a good idea? How do you solve this when porting to 64 bits?
>

There's a very obvious answer to that one. As a compiler-writer, youa re in
a position to do it.

--
Free games and programming goodies.
http://www.personal.leeds.ac.uk/~bgy1mm

 
Reply With Quote
 
 
 
 
Kenneth Brody
Guest
Posts: n/a
 
      08-29-2007
jacob navia wrote:
[... "64-bit compiler" with 64-bit size_t ...]
> The problem is, when you have in thousands of places
>
> int s;
>
> // ...
> s = strlen(str) ;
>
> Since strlen returns a size_t, we have a 64 bit result being
> assigned to a 32 bit int.
>
> This can be correct, and in 99.9999999999999999999999999%
> of the cases the string will be smaller than 2GB...
>
> Now the problem:
>
> Since I warn each time a narrowing conversion is done (since
> that could loose data) I end up with hundreds of warnings each time
> a construct like int a = strlen(...) appears. This clutters
> everything, and important warnings go lost.
>
> I do not know how to get out of this problem. Maybe any of you has
> a good idea? How do you solve this when porting to 64 bits?


Well, some people will probably claim that those hundreds of warnings
are a good thing, as strlen() returns size_t and not int. However,
if you are bombarded with hundreds of such warnings, many people will
simply start ignoring all of the warnings, and the "real" ones will
be lost in the noise.

Perhaps a flag that says "only display the first N instances of this
warning"?

Perhaps you could make int 64 bits as well?

--
+-------------------------+--------------------+-----------------------+
| Kenneth J. Brody | www.hvcomputer.com | #include |
| kenbrody/at\spamcop.net | www.fptech.com | <std_disclaimer.h> |
+-------------------------+--------------------+-----------------------+
Don't e-mail me at: <(E-Mail Removed)>

 
Reply With Quote
 
Ben Pfaff
Guest
Posts: n/a
 
      08-29-2007
jacob navia <(E-Mail Removed)> writes:

> I am trying to compile as much code in 64 bit mode as
> possible to test the 64 bit version of lcc-win.
>
> The problem appears now that size_t is now 64 bits.
>
> Fine. It has to be since there are objects that are more than 4GB
> long.
>
> The problem is, when you have in thousands of places
>
> int s;
>
> // ...
> s = strlen(str) ;
>
> Since strlen returns a size_t, we have a 64 bit result being
> assigned to a 32 bit int.


I'd suggest fixing the code that does this to use size_t instead
of int. size_t is correct. int is, at best, an approximation to
correct. We've just had a pretty long thread with Malcolm McLean
discussing this very topic; perhaps you should refer to that
thread, if you're not already aware of it.
--
char a[]="\n .CJacehknorstu";int putchar(int);int main(void){unsigned long b[]
={0x67dffdff,0x9aa9aa6a,0xa77ffda9,0x7da6aa6a,0xa6 7f6aaa,0xaa9aa9f6,0x11f6},*p
=b,i=24;for(;p+=!*p;*p/=4)switch(0[p]&3)case 0:{return 0;for(p--;i--;i--)case+
2:{i++;if(i)break;else default:continue;if(0)case 1utchar(a[i&15]);break;}}}
 
Reply With Quote
 
Keith Thompson
Guest
Posts: n/a
 
      08-29-2007
jacob navia <(E-Mail Removed)> writes:
> I am trying to compile as much code in 64 bit mode as
> possible to test the 64 bit version of lcc-win.
>
> The problem appears now that size_t is now 64 bits.
>
> Fine. It has to be since there are objects that are more than 4GB
> long.
>
> The problem is, when you have in thousands of places
>
> int s;
>
> // ...
> s = strlen(str) ;
>
> Since strlen returns a size_t, we have a 64 bit result being
> assigned to a 32 bit int.
>
> This can be correct, and in 99.9999999999999999999999999%
> of the cases the string will be smaller than 2GB...
>
> Now the problem:
>
> Since I warn each time a narrowing conversion is done (since
> that could loose data) I end up with hundreds of warnings each time
> a construct like int a = strlen(...) appears. This clutters
> everything, and important warnings go lost.
>
>
> I do not know how to get out of this problem. Maybe any of you has
> a good idea? How do you solve this when porting to 64 bits?


Why didn't you get the same warnings in 32-bit mode? If int and
size_t are both 32 bits, INT_MAX < SIZE_MAX, and there are values of
size_t that cannot be stored in an int. If the "narrowing conversion"
warning is based on the sizes of the type rather than the ranges, I'd
say you've just discovered a compiler bug.

If you're getting hundreds of warnings, it's because you have hundreds
of instances of potential loss of information.

Note that a conversion to a signed type of a value that doesn't fit in
that type yields an implementation-defined result (or, in C99, raises
an implementation-defined signal). In theory, the result could be
more than just a loss of information.

The problem is to distinguish cases where the conversion can't
actually overflow at execution times from the cases where it can.

Sufficiently clever dataflow analysis in the compiler might eliminate
some of the errors. If, given
int s = strlen(str);
the compiler knows enough about how the value of str that it can be
sure it's no longer than INT_MAX bytes, it can eliminate the warning.
But I don't know if it's practical, or even possible to eliminate
enough of the warnings this way. Doing this in most cases is hard;
doing it in all cases might be equivalent to solving the halting
problem. (That latter is only a guess.)

(Making int 64 bits won't solve the problem, since INT_MAX will still
be less than SIZE_MAX.)

You can filter the compiler's output to eliminate warnings about
narrowing implicit conversions (or, if available, use a compiler
option to turn off that particular warning), but that could miss cases
that could actually overflow.

In my opinion, the warnings are legitimate. The ideal solution is not
to suppress them, but to fix the code, assigning the result of
strlen() to a size_t rather than to an int. (Or I suppose you could
use a cast to shut up the compiler if you're *certain* the result can
never exceed INT_MAX, but that's not what I'd do.)

By compiling the code in 64-bit mode, you've discovered a number of
dormant bugs in the code.

--
Keith Thompson (The_Other_Keith) http://www.velocityreviews.com/forums/(E-Mail Removed) <http://www.ghoti.net/~kst>
San Diego Supercomputer Center <*> <http://users.sdsc.edu/~kst>
"We must do something. This is something. Therefore, we must do this."
-- Antony Jay and Jonathan Lynn, "Yes Minister"
 
Reply With Quote
 
Keith Thompson
Guest
Posts: n/a
 
      08-29-2007
"Malcolm McLean" <(E-Mail Removed)> writes:
> "jacob navia" <(E-Mail Removed)> wrote in message
> news:46d5c46d$0$5108$(E-Mail Removed)...

[...]
>>I am trying to compile as much code in 64 bit mode as
>> possible to test the 64 bit version of lcc-win.
>>
>> The problem appears now that size_t is now 64 bits.

[...]

>> int s;
>>
>> // ...
>> s = strlen(str) ;
>>
>> Since strlen returns a size_t, we have a 64 bit result being
>> assigned to a 32 bit int.
>>
>> This can be correct, and in 99.9999999999999999999999999%
>> of the cases the string will be smaller than 2GB...
>>
>> Now the problem:
>>
>> Since I warn each time a narrowing conversion is done (since
>> that could loose data) I end up with hundreds of warnings each time
>> a construct like int a = strlen(...) appears. This clutters
>> everything, and important warnings go lost.
>>
>>
>> I do not know how to get out of this problem. Maybe any of you has
>> a good idea? How do you solve this when porting to 64 bits?
>>

> There's a very obvious answer to that one. As a compiler-writer, youa
> re in a position to do it.


I presume the solution you're suggesting is to make int 64 bits. How
does this help? strlen() still returns size_t, and if int and size_t
are both 64 bits, there will still be size_t values that cannot be
stored in an int.

--
Keith Thompson (The_Other_Keith) (E-Mail Removed) <http://www.ghoti.net/~kst>
San Diego Supercomputer Center <*> <http://users.sdsc.edu/~kst>
"We must do something. This is something. Therefore, we must do this."
-- Antony Jay and Jonathan Lynn, "Yes Minister"
 
Reply With Quote
 
Malcolm McLean
Guest
Posts: n/a
 
      08-29-2007

"Ben Pfaff" <(E-Mail Removed)> wrote in message
news:(E-Mail Removed)...
> I'd suggest fixing the code that does this to use size_t instead
> of int. size_t is correct. int is, at best, an approximation to
> correct. We've just had a pretty long thread with Malcolm McLean
> discussing this very topic; perhaps you should refer to that
> thread, if you're not already aware of it.
>

Yup. As I said, if people would use size_t consistently for every single
calculation that ultimately ends up in an array index there wouldn't be such
a problem. The reality is that people won't, and lots of code doesn't.

--
Free games and programming goodies.
http://www.personal.leeds.ac.uk/~bgy1mm

 
Reply With Quote
 
Malcolm McLean
Guest
Posts: n/a
 
      08-29-2007

"Keith Thompson" <(E-Mail Removed)> wrote in message
news:(E-Mail Removed)...
> "Malcolm McLean" <(E-Mail Removed)> writes:
>> "jacob navia" <(E-Mail Removed)> wrote in message
>> news:46d5c46d$0$5108$(E-Mail Removed)...

> [...]
>>>I am trying to compile as much code in 64 bit mode as
>>> possible to test the 64 bit version of lcc-win.
>>>
>>> The problem appears now that size_t is now 64 bits.

> [...]
>
>>> int s;
>>>
>>> // ...
>>> s = strlen(str) ;
>>>
>>> Since strlen returns a size_t, we have a 64 bit result being
>>> assigned to a 32 bit int.
>>>
>>> This can be correct, and in 99.9999999999999999999999999%
>>> of the cases the string will be smaller than 2GB...
>>>
>>> Now the problem:
>>>
>>> Since I warn each time a narrowing conversion is done (since
>>> that could loose data) I end up with hundreds of warnings each time
>>> a construct like int a = strlen(...) appears. This clutters
>>> everything, and important warnings go lost.
>>>
>>>
>>> I do not know how to get out of this problem. Maybe any of you has
>>> a good idea? How do you solve this when porting to 64 bits?
>>>

>> There's a very obvious answer to that one. As a compiler-writer, youa
>> re in a position to do it.

>
> I presume the solution you're suggesting is to make int 64 bits. How
> does this help? strlen() still returns size_t, and if int and size_t
> are both 64 bits, there will still be size_t values that cannot be
> stored in an int.
>

Yes, but then you'd need an extremely long string to break the code, so the
warning can be suppressed with some confidence that it won't cause a
malfunction.

--
Free games and programming goodies.
http://www.personal.leeds.ac.uk/~bgy1mm




 
Reply With Quote
 
jacob navia
Guest
Posts: n/a
 
      08-29-2007
Keith Thompson wrote:
> Why didn't you get the same warnings in 32-bit mode? If int and
> size_t are both 32 bits, INT_MAX < SIZE_MAX, and there are values of
> size_t that cannot be stored in an int. If the "narrowing conversion"
> warning is based on the sizes of the type rather than the ranges, I'd
> say you've just discovered a compiler bug.
>


2GB strings are the most you can get under the windows schema in 32 bits.

> If you're getting hundreds of warnings, it's because you have hundreds
> of instances of potential loss of information.
>


Yes, "*POTENTIALLY*" I could be missing all those strings longer
than 4GB (!!!). But I do not care about those

> Note that a conversion to a signed type of a value that doesn't fit in
> that type yields an implementation-defined result (or, in C99, raises
> an implementation-defined signal). In theory, the result could be
> more than just a loss of information.
>


Only for strings >2GB Keith. Let's keep it realistic!

> The problem is to distinguish cases where the conversion can't
> actually overflow at execution times from the cases where it can.
>
> Sufficiently clever dataflow analysis in the compiler might eliminate
> some of the errors. If, given
> int s = strlen(str);
> the compiler knows enough about how the value of str that it can be
> sure it's no longer than INT_MAX bytes, it can eliminate the warning.
> But I don't know if it's practical, or even possible to eliminate
> enough of the warnings this way. Doing this in most cases is hard;
> doing it in all cases might be equivalent to solving the halting
> problem. (That latter is only a guess.)
>
> (Making int 64 bits won't solve the problem, since INT_MAX will still
> be less than SIZE_MAX.)
>
> You can filter the compiler's output to eliminate warnings about
> narrowing implicit conversions (or, if available, use a compiler
> option to turn off that particular warning), but that could miss cases
> that could actually overflow.
>
> In my opinion, the warnings are legitimate. The ideal solution is not
> to suppress them, but to fix the code, assigning the result of
> strlen() to a size_t rather than to an int. (Or I suppose you could
> use a cast to shut up the compiler if you're *certain* the result can
> never exceed INT_MAX, but that's not what I'd do.)
>
> By compiling the code in 64-bit mode, you've discovered a number of
> dormant bugs in the code.
>


There isn't any string longer than a few K in this program!
Of course is a potential bug, but it is practically impossible!
 
Reply With Quote
 
jacob navia
Guest
Posts: n/a
 
      08-29-2007
Ben Pfaff wrote:
> jacob navia <(E-Mail Removed)> writes:
>
>> I am trying to compile as much code in 64 bit mode as
>> possible to test the 64 bit version of lcc-win.
>>
>> The problem appears now that size_t is now 64 bits.
>>
>> Fine. It has to be since there are objects that are more than 4GB
>> long.
>>
>> The problem is, when you have in thousands of places
>>
>> int s;
>>
>> // ...
>> s = strlen(str) ;
>>
>> Since strlen returns a size_t, we have a 64 bit result being
>> assigned to a 32 bit int.

>
> I'd suggest fixing the code that does this to use size_t instead
> of int. size_t is correct. int is, at best, an approximation to
> correct. We've just had a pretty long thread with Malcolm McLean
> discussing this very topic; perhaps you should refer to that
> thread, if you're not already aware of it.


The problem is that if I change those ints into size_t's they
are unsigned, and they will produce problems when comparing them with
signed quantities, making MORE modifications necessary in a cascade
of modifications that would surely introduce bugs...

I have *already* introduced (int)strlen(...) in many places...
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
reinterpret_cast<std::size_t>(p) and reinterpret_cast<std::size_t&>() Alex Vinokur C++ 1 02-06-2011 07:48 AM
Casting from const pair<const unsigned char*, size_t>* to constpair<unsigned char*, size_t>* Alex Vinokur C++ 9 10-13-2008 05:05 PM
Re: for(size_t a=begin();a!=end();++a){} Chris \( Val \) C++ 2 07-14-2003 06:31 AM
Re: size_t ... standards Howard Hinnant C++ 5 06-30-2003 07:22 PM
Re: size_t ... standards Howard Hinnant C++ 0 06-29-2003 05:45 PM



Advertisments