Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > C Programming > 32/64 bit cc differences

Reply
Thread Tools

32/64 bit cc differences

 
 
Ben Bacarisse
Guest
Posts: n/a
 
      01-10-2014
JohnF <(E-Mail Removed)> writes:
<snip>
>> (2) Uninitialised variables that had been zeroed were now random values.

> I don't think I leave any uninitialized variables hanging,
> but could be. In that case it's not even a 32- vs 64-bit problem,
> per se, just that the 64-bit compiler isn't zeroing memory like the
> others.


There is a very useful program called valgrind. You will want to strew
scented petals before the feet of those who wrote it.

However, it does not report the use of any initialised data so it's
value here is negative. Of course, that was only one example run. I'd
be tempted to use it for every run until the issue is fixed.

<snip>
--
Ben.
 
Reply With Quote
 
 
 
 
Ben Bacarisse
Guest
Posts: n/a
 
      01-10-2014
JohnF <(E-Mail Removed)> writes:

> Ben Bacarisse <(E-Mail Removed)> wrote:
>> JohnF <(E-Mail Removed)> writes:
>> <snip>
>>> The code is an en/de-cryption utility forkosh.com/fm.zip,
>>> which is way too many lines to ask anybody to look at.

>>
>> Well, it's only about 1100 lines in one file, and about 740 that are not
>> comment lines. Unfortunately it has inconsistent (and odd) spacing and
>> indentation. It made my head hurt reading it!

>
> Thanks, Ben, and sorry about that. I pretty much figured >100 lines
> was too much to ask anybody to read.
>
>> There are signs that it's not written by someone who knows C well --
>> some odd idioms, unnecessary casts, not using prototype declarations,
>> using signed types for bit operations... These things mean it's going
>> to be harder work that one might hope. You'll need to decide if the
>> pay-off is worth it.

>
> From the "Revision history" comment near top, note that first version
> was written in 1988, ported from an even earlier program.
> Your "signed types for bit operations" is indeed the one problem
> I'd actually worried about (not that the others aren't problems,
> but unlikely to be causing odd observed behavior), and which
> I actually checked with small standalone test program, as noted
> in preceding followup. But that particular problem is easy enough
> to fix (I know exactly what you're referring to in xcryptstr),
> and I'll do that (and kick myself if it fixes the problem).


Actually I was thinking about the bit getting and setting macros myself.
Also, check that the shift amount is never equal to the bit width. The
fact that this is not defined (even for unsigned types) surprises some
people. One way to do this is to add an assert to them.

The other thing I'd do is to check all the casts and remove those that
are not needed. They can mask interesting and useful compiler warnings.
(You are asking for lots of warnings, I hope.)

<snip>
> P.S. I'm amazed you read the code so fast and in enough detail
> with enough understanding to pick up the info for your remarks.
> Scary. Thanks again,


I once had a job which was almost entirely porting C code from one
system to another very peculiar one. The result is that some (sadly not
all) portability problems leap out at me.

--
Ben.
 
Reply With Quote
 
 
 
 
BartC
Guest
Posts: n/a
 
      01-10-2014
"JohnF" <(E-Mail Removed)> wrote in message
news:laoua0$r0m$(E-Mail Removed)...
> BartC <(E-Mail Removed)> wrote:
>> "JohnF" <(E-Mail Removed)> wrote in message


>> You might try also using, temporarily, specific widths for all integers,
>> instead of relying on whatever width 'int' happens to map to. However, if
>> int is 64-bits on one machine, then I think all intermediate results will
>> be
>> widened to 64-bits, whatever you do.)

>
> Yeah, I didn't mention that I'd also tried -m32-bit on the 64-bit box,
> but that compiler (--version is cc (Debian 4.3.2-1.1) 4.3.2) said
> cc1: error: unrecognized command line option "-m32-bit"
> and man cc isn't on that machine.


How big are int, long and long long on your machines?

I seem to remember that gcc 'long' was 32-bit under Windows, and 64-bit
under Linux. (If you expect long to be 64-bits, and ever want to run under
Windows, you might want to upgrade the type.)

(I've looked at your code; I've no idea what the problem might be, but if
you're porting from an int=32, long=64 implementation to an int=64, long=64
one, I'm surprised you haven't got bigger differences. Certainly my own code
would have a load of problems!)

--
Bartc

 
Reply With Quote
 
James Kuyper
Guest
Posts: n/a
 
      01-10-2014
On 01/10/2014 08:21 AM, JohnF wrote:
> glen herrmannsfeldt <(E-Mail Removed)> wrote:
>> JohnF <(E-Mail Removed)> wrote:


> For encrypting, the program fread()'s the input file in randomly-sized
> blocks, processing each block separately. It first adds a random number
> of noise bytes (which can be set to 0 and which I tried),
> then randomly xor or xnor's each data byte with a random byte from
> your key, and finally randomly permutes the bits of the entire block
> (typically a many-thousand-bit permutation). Decrypting reverses
> the procedure.

....
> Of course, I can modify the program to turn off stuff more
> "slowly", and I'm pretty sure that'll eventually zero in on the
> problem. But it's a big pain to do that properly, e.g., the random
> number streams have to be kept in sync reversibly, so that
> encrypted stuff can be decrypted.


The random numbers in your program make it difficult to debug, because
the behavior can be different each time it's run. For debugging
purposes, you should modify the code to use a fixed seed for your random
number generator, so that it generates exactly the same random numbers
each time it is run. Make sure that the particular seed you use is one
that will reproduce the problem!
One possible issue is that the random number generators on the two
systems might be different. You might need to write your own, just to
make sure it generates the same sequence every time, on both systems. It
doesn't have to be a very sophisticated one, so long as it does
reproducibly duplicate the problem you're seeing.
 
Reply With Quote
 
Aleksandar Kuktin
Guest
Posts: n/a
 
      01-10-2014
On Fri, 10 Jan 2014 10:05:47 +0000, JohnF wrote:

> Firstly, both executables "work", i.e., if you encrypt and then decrypt,
> you get back the exact same original file.
> But if you encrypt using the 32-bit executable, scp the encrypted file
> to the 64-bit machine (md5's match) and then decrypt, the result is
> exactly the same length and almost identical except for about one byte
> in a thousand that doesn't diff. Vice versa (encrypt on 64-bit, decrypt
> on 32) gives the same behavior. (By the way, the 32-vs-64-bit encrypted
> files are also ~one-in-a-thousand different, so both stages exhibit this
> small problem.)


Wait, I don't understand this.

Correct me if I'm wrong:
1. same input file;
2. output of 32-bit executable and of 64-bit executable are bit-for-bit
identical;
3. output of executable, when decrypted by same executable that outputted
it, is bit-for-bit identical to the input;
4. output of 32-bit executable, when decrypted on 64-bit produces wrong
output;
5. output of 64-bit executable, when decrypted on 32-bit produces wrong
output.

Obviously, this can not be.
 
Reply With Quote
 
James Kuyper
Guest
Posts: n/a
 
      01-10-2014
On 01/10/2014 11:26 AM, Aleksandar Kuktin wrote:
> On Fri, 10 Jan 2014 10:05:47 +0000, JohnF wrote:
>
>> Firstly, both executables "work", i.e., if you encrypt and then decrypt,
>> you get back the exact same original file.
>> But if you encrypt using the 32-bit executable, scp the encrypted file
>> to the 64-bit machine (md5's match) and then decrypt, the result is
>> exactly the same length and almost identical except for about one byte
>> in a thousand that doesn't diff. Vice versa (encrypt on 64-bit, decrypt
>> on 32) gives the same behavior. (By the way, the 32-vs-64-bit encrypted
>> files are also ~one-in-a-thousand different, so both stages exhibit this
>> small problem.)

>
> Wait, I don't understand this.
>
> Correct me if I'm wrong:
> 1. same input file;
> 2. output of 32-bit executable and of 64-bit executable are bit-for-bit
> identical;
> 3. output of executable, when decrypted by same executable that outputted
> it, is bit-for-bit identical to the input;
> 4. output of 32-bit executable, when decrypted on 64-bit produces wrong
> output;
> 5. output of 64-bit executable, when decrypted on 32-bit produces wrong
> output.
>
> Obviously, this can not be.


My understanding of what he's saying (I could be mistaken) is that
input => 32-bit executable in encrypt mode => encrypted1 => 32-bit
executable in decrypt mode=> output1

input => 64-bit executable in encrypt mode => encrypted2 => 64-bit
executable in decrypt mode => output1

output1 is identical to input, for both runs, but encrypted1 differs
from encrypted2. As a result:

input => 32-bit executable in encrypt mode => encrypted1 => 64-bit
executable in decrypt mode => output2

input => 64-bit executable in encrypt mode => encrypted2 => 32-bit
executable in decrypt mode => output3

Neither output2 nor output3 is the same as output1 - but only one byte
in a thousand is different.

 
Reply With Quote
 
Keith Thompson
Guest
Posts: n/a
 
      01-10-2014
Aleksandar Kuktin <(E-Mail Removed)> writes:
> On Fri, 10 Jan 2014 10:05:47 +0000, JohnF wrote:
>> Firstly, both executables "work", i.e., if you encrypt and then decrypt,
>> you get back the exact same original file.
>> But if you encrypt using the 32-bit executable, scp the encrypted file
>> to the 64-bit machine (md5's match) and then decrypt, the result is
>> exactly the same length and almost identical except for about one byte
>> in a thousand that doesn't diff. Vice versa (encrypt on 64-bit, decrypt
>> on 32) gives the same behavior. (By the way, the 32-vs-64-bit encrypted
>> files are also ~one-in-a-thousand different, so both stages exhibit this
>> small problem.)

>
> Wait, I don't understand this.
>
> Correct me if I'm wrong:
> 1. same input file;
> 2. output of 32-bit executable and of 64-bit executable are bit-for-bit
> identical;


I don't believe that's what he said.

On both 32-bit and 64-bit systems, encrypting a file *and then
decrypting it* yields an identical copy of the original file.
But the content of the encrypted file can vary (probably even from
one run to the next, since the program makes heavy use of random
numbers).

So a 32-bit encrypted file and a 64-bit encrypted file, from the
same input, will differ -- and decrypting a 32-bit encrypted file
using the 64-bit decryption program, or vice versa, yields output
that's *slightly* different from the original input.

JohnF, have I summarized the behavior correctly?

You say "about one byte in a thousand". Is there any pattern to
where the incorect bytes appear in the output? For example, if
the last byte of each 1024-byte block were incorrect, that would
be interesting. If it's seemingly random and varies from one run
to the next, that would also be interesting.

--
Keith Thompson (The_Other_Keith) http://www.velocityreviews.com/forums/(E-Mail Removed) <http://www.ghoti.net/~kst>
Working, but not speaking, for JetHead Development, Inc.
"We must do something. This is something. Therefore, we must do this."
-- Antony Jay and Jonathan Lynn, "Yes Minister"
 
Reply With Quote
 
Stephen Sprunk
Guest
Posts: n/a
 
      01-10-2014
On 10-Jan-14 07:21, JohnF wrote:
> The code's supposed to be quite portable, and was originally
> checked on intel linux, netbsd, freebsd, ms windows, and on
> VAX and alpha OpenVMS. It always generated identical encrypted
> files...until now.


Are all of those systems ILP32, or is the Alpha one ILP64? (I've never
used OpenVMS.) Have you tried it on Win64, which is IL32LLP64, or just
Win32?

Linux/x86-64 is I32LP64, so if that's the first such system you've
ported to, that may indicate where the problem lies. If your code works
on ILP64 systems, type problems seem unlikely, but you could still have
int/long mismatches that wouldn't show up there--or on IL32LLP64.

Also, you're using three different versions of GCC; it's possible that
one of them has a bug (or "feature") that's triggered by some UB in your
code and results in slight output changes. I'd highly suggest using the
same version on all your systems, if possible, to eliminate that as a
potential source of differences.

Aside: Why roll your own encryption algorithm rather than use a proven,
off-the-shelf algorithm, e.g. AES? Depending on OpenSSL's libcrypto is
pretty standard these days; no sense reinventing the square wheel.

S

--
Stephen Sprunk "God does not play dice." --Albert Einstein
CCIE #3723 "God is an inveterate gambler, and He throws the
K5SSS dice at every possible opportunity." --Stephen Hawking
 
Reply With Quote
 
osmium
Guest
Posts: n/a
 
      01-10-2014
"James Kuyper" wrote:

> On 01/10/2014 08:21 AM, JohnF wrote:
>> glen herrmannsfeldt <(E-Mail Removed)> wrote:
>>> JohnF <(E-Mail Removed)> wrote:

>
>> For encrypting, the program fread()'s the input file in randomly-sized
>> blocks, processing each block separately. It first adds a random number
>> of noise bytes (which can be set to 0 and which I tried),
>> then randomly xor or xnor's each data byte with a random byte from
>> your key, and finally randomly permutes the bits of the entire block
>> (typically a many-thousand-bit permutation). Decrypting reverses
>> the procedure.

> ...
>> Of course, I can modify the program to turn off stuff more
>> "slowly", and I'm pretty sure that'll eventually zero in on the
>> problem. But it's a big pain to do that properly, e.g., the random
>> number streams have to be kept in sync reversibly, so that
>> encrypted stuff can be decrypted.

>
> The random numbers in your program make it difficult to debug, because
> the behavior can be different each time it's run. For debugging
> purposes, you should modify the code to use a fixed seed for your random
> number generator, so that it generates exactly the same random numbers
> each time it is run. Make sure that the particular seed you use is one
> that will reproduce the problem!
> One possible issue is that the random number generators on the two
> systems might be different. You might need to write your own, just to
> make sure it generates the same sequence every time, on both systems. It
> doesn't have to be a very sophisticated one, so long as it does
> reproducibly duplicate the problem you're seeing.


As a bystander to this thread, that sounds like a Really Really Good Idea.


 
Reply With Quote
 
Stephen Sprunk
Guest
Posts: n/a
 
      01-10-2014
On 10-Jan-14 11:29, Keith Thompson wrote:
> You say "about one byte in a thousand". Is there any pattern to
> where the incorect bytes appear in the output? For example, if
> the last byte of each 1024-byte block were incorrect, that would
> be interesting. If it's seemingly random and varies from one run
> to the next, that would also be interesting.


My first thought when seeing "one byte error every $blocksize" is that
there's an off-by-one error reading/modifying some buffer, but I can't
think of how porting between 32- and 64-bit variants of essentially the
same implementation would trigger such; it should always be there.

S

--
Stephen Sprunk "God does not play dice." --Albert Einstein
CCIE #3723 "God is an inveterate gambler, and He throws the
K5SSS dice at every possible opportunity." --Stephen Hawking
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
What is the point of having 16 bit colour if a computer monitor can only display 8 bit colour? How do you edit 16 bit colour when you can only see 8 bit? Scotius Digital Photography 6 07-13-2010 03:33 AM
DEVELOP THE WINNING EDGE, SMALL DIFFERENCES IN YOUR PERFORMANCE CANLEAD TO LARGE DIFFERENCES IN YOUR RESULTS Home_Job_opportunity C Programming 0 01-14-2009 03:51 PM
DEVELOP THE WINNING EDGE, SMALL DIFFERENCES IN YOUR PERFORMANCE CANLEAD TO LARGE DIFFERENCES IN YOUR RESULTS Home_Job_opportunity C Programming 0 01-08-2009 04:31 PM
64 bit - Windows Liberty 64bit, Windows Limited Edition 64 Bit, Microsoft SQL Server 2000 Developer Edition 64 Bit, IBM DB2 64 bit - new ! vvcd Computer Support 0 09-17-2004 08:15 PM
64 bit - Windows Liberty 64bit, Windows Limited Edition 64 Bit,Microsoft SQL Server 2000 Developer Edition 64 Bit, IBM DB2 64 bit - new! Ionizer Computer Support 1 01-01-2004 07:27 PM



Advertisments