Velocity Reviews

Velocity Reviews (http://www.velocityreviews.com/forums/index.php)
-   C Programming (http://www.velocityreviews.com/forums/f42-c-programming.html)
-   -   64 bit porting (http://www.velocityreviews.com/forums/t434470-64-bit-porting.html)

Mohanasundaram 07-09-2004 12:58 PM

64 bit porting
 
Hi All,

We are working on porting a product written in C and C++ from 32 bit
to 64 bit. We need to maintain both 32 bit and 64 bit versions in the
future. We took the 32 bit source code and copiled it using a 64 bit
compiler and fixed all the compilation warnings. Compilation went
through fine but the product breaks in lots of places. We understood
that porting a 32 bit code to 64 bit platform is not just a matter of
compilation. We have to handle the problems which will not be caught
by the compiler due to the change from ILP32 to LP64. So we are trying
to list out all the possible problems that might accour due to the
change from size of long and pointer from 4 bytes to 8 bytes with int
still being 4 bytes. We have listed out few possibilites of the places
where bugs might creep up. We want to validate the correctness of the
points and would like to add more in to this list. Please help us.

1. Change all the long to integers blindly. But not integers to long.
We think this might solve the following problems
(a) Code written using bitwise operators assuming that the size of
long is 4 will create problems
(b) Getting the offsets of the fields in structures by not using
OFFSET macro will create problems when the structe has longs
(c) Manipulating the long data bytewise by breaking them using
pointers like long i = 1; char a = ((char *)&i)[0];
etc.....

2. Check for all the library functions which returns long, like atol
and make sure no code is written assuming that the reaturn value
is of 4 bytes. Or consider changing atol to atoi or simlar functions.

3. If C style memory allocation is used insted of "new" then there are
possibility for bugs.
long *ptr = malloc(4*2);
in 32 bit compilation the above statement will allocate 8 bytes of
memory and ptr can be used as an array of two elements.
But in 64 bit compilation it will allocate 8 bytes and the number of
elements in the array is one. So if code is written
assuming that the number of elements is two then it will break. So
all "malloc"s "calloc"s and "realloc"s should be checked.

4. If the pointers are casted to integers anywhere it has to be
checked.
For example
int a = 10;
int *ptr = &a;
int b = reinterpret_cast<int>(a);
The above code will crete problems in 64 bit compilation since
pointer is 8 bytes and int is 4 bytes. So it has to be
changed to
long b = reinterpret_cast<long>(a);

5. Getting the offsets of the structure fields by assuming the size of
the fields and not using OFFSET macro. How does this stuff
work in case of unions or classes

6. size_t is a 32 bit quantity in 32 bit compilation wherein it grows
to 64 in 64 bit compilation.

7. The format specifiers should be checked for example
in printfs and scanfs
long i = <some expression>;
printf("%d",i);
will not be a big problem as far as the result is converned but it
will print wrong values when the value is i is very
big and exceeds the limit of integer.

Thanks a lot for your time.

Regards,
Mohan.

Dan Pop 07-09-2004 01:38 PM

Re: 64 bit porting
 
In <9bb9619e.0407090458.5fcb34f8@posting.google.com > mohanasundaram@msn.com (Mohanasundaram) writes:

> We are working on porting a product written in C and C++ from 32 bit
>to 64 bit. We need to maintain both 32 bit and 64 bit versions in the
>future.


If you do your job right, you will have only version to maintain, that
will work on both 32 and 64-bit platforms. This is usually called
64-bit clean code.

>1. Change all the long to integers blindly. But not integers to long.


Don't change *anything* blindly. Try to understand *all* the implications
of *each and every* change you make.

> We think this might solve the following problems
> (a) Code written using bitwise operators assuming that the size of
>long is 4 will create problems


Fix such code instead.

> (b) Getting the offsets of the fields in structures by not using
>OFFSET macro will create problems when the structe has longs


Doing that is sheer stupidity in the first place. If you need such
offsets, offsetof() or pointer arithmetic are the ONLY ways to go.

> (c) Manipulating the long data bytewise by breaking them using
>pointers like long i = 1; char a = ((char *)&i)[0];
> etc.....


This is not affected by 32 vs 64 bit issues, but may be affected by byte
order issues. Switching from long to int buys you nothing. And you
really want to use unsigned char for this purpose.

>2. Check for all the library functions which returns long, like atol
>and make sure no code is written assuming that the reaturn value
>is of 4 bytes. Or consider changing atol to atoi or simlar functions.


Much better, remove *all* the dependencies of the C types sizes in the
code, if reasonably possible.

>3. If C style memory allocation is used insted of "new" then there are
>possibility for bugs.
> long *ptr = malloc(4*2);
> in 32 bit compilation the above statement will allocate 8 bytes of
>memory and ptr can be used as an array of two elements.
> But in 64 bit compilation it will allocate 8 bytes and the number of
>elements in the array is one. So if code is written
> assuming that the number of elements is two then it will break. So
>all "malloc"s "calloc"s and "realloc"s should be checked.


Indeed, and the proper fix is:

long *ptr = malloc(2 * sizeof *ptr);

which is correct *everywhere*.

>4. If the pointers are casted to integers anywhere it has to be
>checked.
> For example
> int a = 10;
> int *ptr = &a;
> int b = reinterpret_cast<int>(a);
> The above code will crete problems in 64 bit compilation since
>pointer is 8 bytes and int is 4 bytes. So it has to be
> changed to
> long b = reinterpret_cast<long>(a);

^^^^^^^^^^^^^^^^^^^^^^
This is not valid C syntax, so I don't know what you're talking about.
If you need to convert pointers to integers, the type unsigned long is
the best choice on both 32 (ILP32) and 64-bit (I32LP64) platforms.

>5. Getting the offsets of the structure fields by assuming the size of
>the fields and not using OFFSET macro.


Deja vu (point 1b above).

>How does this stuff work in case of unions or classes


It is not needed for unions (all members have offset 0) and there are
no classes in C.

>6. size_t is a 32 bit quantity in 32 bit compilation wherein it grows
>to 64 in 64 bit compilation.


Why should your code care about the size of size_t?

>7. The format specifiers should be checked for example
> in printfs and scanfs
> long i = <some expression>;
> printf("%d",i);
> will not be a big problem as far as the result is converned but it
>will print wrong values when the value is i is very
> big and exceeds the limit of integer.


This code is already broken and it works by pure accident. If i has type
long, %d is NOT an option. %ld will correctly work on both 32 and 64-bit
platforms.

It looks like your code was severely broken even on 32-bit platforms and
it worked by luck/accident. Once you fix it, if you do the job right,
it will work equally well on both 32 and 64-bit platforms, without needing
separate versions.

If you need to share binary files between 32 and 64-bit platforms, pay
extra attention to the definition of the data that gets written into the
files.

Dan
--
Dan Pop
DESY Zeuthen, RZ group
Email: Dan.Pop@ifh.de

Tim Prince 07-10-2004 04:02 AM

Re: 64 bit porting
 

"Dan Pop" <Dan.Pop@cern.ch> wrote in message
news:ccm759$6mn$1@sunnews.cern.ch...
> In <9bb9619e.0407090458.5fcb34f8@posting.google.com >

mohanasundaram@msn.com (Mohanasundaram) writes:
>


> >6. size_t is a 32 bit quantity in 32 bit compilation wherein it grows
> >to 64 in 64 bit compilation.

>
> Why should your code care about the size of size_t?
>

Some "C/C++" (sic) customers are adamant that there has to be a way to bury
size_t stuff in the middle of a struct without padding or breaking
alignments between platforms, or that any compiler which barfs at storing
int and size_t interchangeably is broken. When that comes up, it's a
probable sign that I should go back to projects which don't have C++ in
them.



Igmar Palsenberg 07-10-2004 04:20 PM

Re: 64 bit porting
 
Mohanasundaram wrote:

> 1. Change all the long to integers blindly. But not integers to long.
> We think this might solve the following problems
> (a) Code written using bitwise operators assuming that the size of
> long is 4 will create problems
> (b) Getting the offsets of the fields in structures by not using
> OFFSET macro will create problems when the structe has longs
> (c) Manipulating the long data bytewise by breaking them using
> pointers like long i = 1; char a = ((char *)&i)[0];
> etc.....


If you're on a *NIX system, use sys/types.h, and use things like
u_int32_t, int32_t, etc, etc. If you're on Windows, use something that
looks like it.

> 2. Check for all the library functions which returns long, like atol
> and make sure no code is written assuming that the reaturn value
> is of 4 bytes. Or consider changing atol to atoi or simlar functions.


Fix the library I would say.

> 3. If C style memory allocation is used insted of "new" then there are
> possibility for bugs.
> long *ptr = malloc(4*2);
> in 32 bit compilation the above statement will allocate 8 bytes of
> memory and ptr can be used as an array of two elements.
> But in 64 bit compilation it will allocate 8 bytes and the number of
> elements in the array is one. So if code is written
> assuming that the number of elements is two then it will break. So
> all "malloc"s "calloc"s and "realloc"s should be checked.


use malloc(sizeof(long) * 2) for that.

> 4. If the pointers are casted to integers anywhere it has to be
> checked.
> For example
> int a = 10;
> int *ptr = &a;
> int b = reinterpret_cast<int>(a);
> The above code will crete problems in 64 bit compilation since
> pointer is 8 bytes and int is 4 bytes. So it has to be
> changed to
> long b = reinterpret_cast<long>(a);


Fix the code. Casting pointers to ints is a sign of problems in the design.

> 5. Getting the offsets of the structure fields by assuming the size of
> the fields and not using OFFSET macro. How does this stuff
> work in case of unions or classes


The compiler knows the offset. Since you can use members of structs
directly, I hardly see a reason to use them.

> 6. size_t is a 32 bit quantity in 32 bit compilation wherein it grows
> to 64 in 64 bit compilation.


size_t can be a 64 bits variable in 32 bits platforms. It is common
these days, since offsets need 64 bits when dealing with large files.


> 7. The format specifiers should be checked for example
> in printfs and scanfs
> long i = <some expression>;
> printf("%d",i);
> will not be a big problem as far as the result is converned but it
> will print wrong values when the value is i is very
> big and exceeds the limit of integer.


Replace long by an type that indicates what variable and length you
actually mean. That saves tons of headaches, an make the code better to
read.


> Thanks a lot for your time.
>
> Regards,
> Mohan.




Igmar

Stephen Sprunk 07-10-2004 09:59 PM

Re: 64 bit porting
 
"Igmar Palsenberg" <igmar@non-existant.local> wrote in message
news:40f01837$0$48959$e4fe514c@news.xs4all.nl...
> > 6. size_t is a 32 bit quantity in 32 bit compilation wherein it grows
> > to 64 in 64 bit compilation.

>
> size_t can be a 64 bits variable in 32 bits platforms. It is common
> these days, since offsets need 64 bits when dealing with large files.


Why would size_t be 64b on a 32b platform? size_t is the maximum size of a
single allocated object _in memory_, so where is the need for it to exceed
the size of the address space?

There's at least one well-known case where size_t is smaller than the
address space size, but on what implementations can it be larger?

S

--
Stephen Sprunk "Those people who think they know everything
CCIE #3723 are a great annoyance to those of us who do."
K5SSS --Isaac Asimov


jacob navia 07-10-2004 10:35 PM

Re: 64 bit porting
 

"Stephen Sprunk" <stephen@sprunk.org> a écrit dans le message de
news:0af4f619f23b619d8b4a327e951e4a06@news.teranew s.com...
> "Igmar Palsenberg" <igmar@non-existant.local> wrote in message
> There's at least one well-known case where size_t is smaller than the
> address space size, but on what implementations can it be larger?


In the bloated ones :-)




Igmar Palsenberg 07-11-2004 10:31 AM

Re: 64 bit porting
 
Stephen Sprunk wrote:

> Why would size_t be 64b on a 32b platform? size_t is the maximum size of a
> single allocated object _in memory_, so where is the need for it to exceed
> the size of the address space?


Never mind the size_t remark : That should be off_t.

> There's at least one well-known case where size_t is smaller than the
> address space size, but on what implementations can it be larger?


None it seems :)



Igmar

Randy Howard 07-11-2004 05:15 PM

Re: 64 bit porting
 
In article <40f1180d$0$93324$e4fe514c@news.xs4all.nl>, igmar@non-existant.local
says...
> Stephen Sprunk wrote:
>
> > Why would size_t be 64b on a 32b platform? size_t is the maximum size of a
> > single allocated object _in memory_, so where is the need for it to exceed
> > the size of the address space?

>
> Never mind the size_t remark : That should be off_t.
>
> > There's at least one well-known case where size_t is smaller than the
> > address space size, but on what implementations can it be larger?

>
> None it seems :)


I suppose if you had a magic compiler that supported PAE extension for
Intel, it would be possible to have "magic pointers" and offsets that
were outside the range of 32-bit registers by themselves. As may, or
may not be known, 32-bit operating systems such as the W2K and W2K3
server platforms plus Linux distributions can see much more than the
expected 4GB limitation, in some cases as much as 64GB of RAM using
the "PAE" hack Intel came up with. However, most of the time, there
is still a 2GB per process address space limitation, so I'm not sure
how this magical compiler could get around such a limit in all cases,
perhaps using something akin to the MS "AWE API" on your behalf.

I suspect such a compiler would always be buggy, and cost more than
anyone cares to imagine.

It's quite a bit easier to buy a motherboard and AMD 64-bit CPU
for < $500 and go on your merry way. :-)

--
Randy Howard
To reply, remove FOOBAR.

Rupert Pigott 07-14-2004 02:14 PM

Re: 64 bit porting
 
Dan Pop wrote:

[SNIP]

>>3. If C style memory allocation is used insted of "new" then there are
>>possibility for bugs.
>> long *ptr = malloc(4*2);


Weird. Size my very first malloc program I've been using sizeof() to
work out how big I want stuff.

/* Single ptr */
long* ptr = malloc( sizeof( long* ) );

>> in 32 bit compilation the above statement will allocate 8 bytes of
>>memory and ptr can be used as an array of two elements.
>> But in 64 bit compilation it will allocate 8 bytes and the number of
>>elements in the array is one. So if code is written
>> assuming that the number of elements is two then it will break. So
>>all "malloc"s "calloc"s and "realloc"s should be checked.

>
>
> Indeed, and the proper fix is:
>
> long *ptr = malloc(2 * sizeof *ptr);


Erm... Wouldn't something like the following be a little safer and
easier explain for an array of two or more pointers ?

/* Array of 2 ptrs */
long* ptr = malloc( sizeof( long*[2] ) );

The rationale for this approach is that you're taking into account
any weird array element alignment stuff that the compiler might
want to do. I did come unstuck with this many moons ago when I
wrote the following :

short* array = malloc( sizeof( short ) * 42 );

.... The compiler liked to pad shorts up to a word boundary (so
there was practically zero point in them). Therefore I had not
allocated enough memory. When I wrote to that array I ended up
corrupting other stuff and the program died a firey death after
some entertaining but wrong results... After much hair pulling
I changed the line to take this into account, it became :

short* array = malloc( sizeof( short[42] ) );

> which is correct *everywhere*.


The big question is : Does ANSI C permit compilers to pad array
elements up to some other size like it does with structures ?
My guess is *no* given the amount of code that runs fine with
the more intuitive "sizeof( type ) * n" approach.

Cheers,
Rupert


Richard Bos 07-14-2004 02:27 PM

Re: 64 bit porting
 
Rupert Pigott <roo@try-removing-this.darkboong.demon.co.uk> wrote:

> Dan Pop wrote:
>
> > Indeed, and the proper fix is:
> >
> > long *ptr = malloc(2 * sizeof *ptr);

>
> Erm... Wouldn't something like the following be a little safer and
> easier explain for an array of two or more pointers ?
>
> /* Array of 2 ptrs */
> long* ptr = malloc( sizeof( long*[2] ) );
>
> The rationale for this approach is that you're taking into account
> any weird array element alignment stuff that the compiler might
> want to do.


The implementation is not allowed to do any weird array alignment stuff,
unless it also does it in an array of one element, aka the base type.
I.e., sizeof (long*[2]) _must_ be 2*sizeof (long)

> I did come unstuck with this many moons ago when I
> wrote the following :
>
> short* array = malloc( sizeof( short ) * 42 );
>
> ... The compiler liked to pad shorts up to a word boundary (so
> there was practically zero point in them). Therefore I had not
> allocated enough memory.


If it did that for the array, but _not_ for individual shorts, it was
not a C compiler.

> The big question is : Does ANSI C permit compilers to pad array
> elements up to some other size like it does with structures ?


No. Not more so than the individual elements.

Richard


All times are GMT. The time now is 05:59 PM.

Powered by vBulletin®. Copyright ©2000 - 2014, vBulletin Solutions, Inc.
SEO by vBSEO ©2010, Crawlability, Inc.