Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Java > read binary data from C file???

Reply
Thread Tools

read binary data from C file???

 
 
John Adams
Guest
Posts: n/a
 
      09-22-2004
Hi,

I am sorry if this has been asked before but I couldn't find any hint
scheming through the list. So here is my problem:

I have to write a client end in java reading data sent over UDP from
a server (the server is written in C). The data format is a C struct
with all the fields including long, int, float, etc. I got the data in
the java client end in the format of a DatagramPacket, then I convert it
to a byte array. Now the question is, how do I extract each field from
the byte array? I know this is silly but I have tried using classes like
ByteBuffer, Array, etc, without good result (The print out of the
recieiving end differs than that from the sending end).

Thanks for all the advice.
 
Reply With Quote
 
 
 
 
Michael Borgwardt
Guest
Posts: n/a
 
      09-22-2004
John Adams wrote:
> I am sorry if this has been asked before but I couldn't find any hint
> scheming through the list. So here is my problem:
>
> I have to write a client end in java reading data sent over UDP from a
> server (the server is written in C). The data format is a C struct with
> all the fields including long, int, float, etc. I got the data in the
> java client end in the format of a DatagramPacket, then I convert it to
> a byte array. Now the question is, how do I extract each field from the
> byte array? I know this is silly but I have tried using classes like
> ByteBuffer, Array, etc, without good result (The print out of the
> recieiving end differs than that from the sending end).


What do you expect? That we can see your code and find the bugs in it by
telepathy?

Show us what you tried, then maybe we can tell you what seems fishy.

The most obvious thing that can go wrong is endianness and alignment
in the C struct.

The most obvious way to debug it is to put some easily identifiable
values into the struct on the server side and then dump the entire
byte array on the client to see how your data actually arrives.
 
Reply With Quote
 
 
 
 
Will Hartung
Guest
Posts: n/a
 
      09-22-2004

"John Adams" <(E-Mail Removed)> wrote in message
news:415183b6$(E-Mail Removed)...
> Hi,
>
> I am sorry if this has been asked before but I couldn't find any hint
> scheming through the list. So here is my problem:
>
> I have to write a client end in java reading data sent over UDP from
> a server (the server is written in C). The data format is a C struct
> with all the fields including long, int, float, etc. I got the data in
> the java client end in the format of a DatagramPacket, then I convert it
> to a byte array. Now the question is, how do I extract each field from
> the byte array? I know this is silly but I have tried using classes like
> ByteBuffer, Array, etc, without good result (The print out of the
> recieiving end differs than that from the sending end).
>
> Thanks for all the advice.


Take a look at java.io.DataInputStream. It may do what you want, but you may
have issues with endian-ness.

Regards,

Will Hartung
((E-Mail Removed))



 
Reply With Quote
 
Boudewijn Dijkstra
Guest
Posts: n/a
 
      09-23-2004
"John Adams" <(E-Mail Removed)> schreef in bericht
news:415183b6$(E-Mail Removed)...
> Hi,
>
> I am sorry if this has been asked before but I couldn't find any hint
> scheming through the list. So here is my problem:
>
> I have to write a client end in java reading data sent over UDP from
> a server (the server is written in C). The data format is a C struct
> with all the fields including long, int, float, etc. I got the data in
> the java client end in the format of a DatagramPacket, then I convert it
> to a byte array. Now the question is, how do I extract each field from
> the byte array? I know this is silly but I have tried using classes like
> ByteBuffer, Array, etc, without good result (The print out of the
> recieiving end differs than that from the sending end).


Experiment with a few struct values to find out where each byte ends up in the
array. Then use << and | to fill ints and longs. If you want floats and
doubles, use Float.intBitsToFloat() and Double.longBitsToDouble().


 
Reply With Quote
 
Gordon Beaton
Guest
Posts: n/a
 
      09-23-2004
On Wed, 22 Sep 2004 16:53:23 -0400, John Adams wrote:
> I have to write a client end in java reading data sent over UDP
> from a server (the server is written in C). The data format is a C
> struct with all the fields including long, int, float, etc. I got
> the data in the java client end in the format of a DatagramPacket,
> then I convert it to a byte array. Now the question is, how do I
> extract each field from the byte array?


Others have already pointed out how you can read the data, however
it's worth mentioning that if the server gets recompiled with a
different compiler, or moved to a different platform (with no changes
to the data types) it could break your client.

The real solution is to change the server so that it doesn't write the
data in a format that is quite so compiler and platform dependent.

Writing raw binary data directly from a C struct is lazy, fragile and
non-portable. And as you've already discovered, it makes it difficult
and error prone to write a client in a different language, that
doesn't use the same compiler, or that will run on a different
hardware platform.

/gordon

--
[ do not email me copies of your followups ]
g o r d o n + n e w s @ b a l d e r 1 3 . s e
 
Reply With Quote
 
John Adams
Guest
Posts: n/a
 
      09-23-2004
Gordon Beaton wrote:
> On Wed, 22 Sep 2004 16:53:23 -0400, John Adams wrote:
>
>> I have to write a client end in java reading data sent over UDP
>>from a server (the server is written in C). The data format is a C
>>struct with all the fields including long, int, float, etc. I got
>>the data in the java client end in the format of a DatagramPacket,
>>then I convert it to a byte array. Now the question is, how do I
>>extract each field from the byte array?

>
>
> Others have already pointed out how you can read the data, however
> it's worth mentioning that if the server gets recompiled with a
> different compiler, or moved to a different platform (with no changes
> to the data types) it could break your client.
>
> The real solution is to change the server so that it doesn't write the
> data in a format that is quite so compiler and platform dependent.
>
> Writing raw binary data directly from a C struct is lazy, fragile and
> non-portable. And as you've already discovered, it makes it difficult
> and error prone to write a client in a different language, that
> doesn't use the same compiler, or that will run on a different
> hardware platform.
>
> /gordon
>



Hi,

Thank you all for your invaluable advice. To date, I have seemed to
try all the methods mentioned: Array, ByteBuffer, DataInputStream,
ByteArrayInputStream, intBitsToFloat(), etc. I had, ofcourse as Michael
Borgwardt so "brilliantly" pointed out, tried putting some values on the
server side and dumping them out on the client side (That is HOW I know
they are not the same!!!).

Flatform and hardware also are not the concern since I had tried
running the server on the very same intel machine as the client (just to
debug). Endianess is also out of question in this scenario.

This is how I do it:

// The data format from a C server.
// To make thing simple, I make all fiels float
typede struct {
float something;
float some_other_thing; ...
}

// class to convert data on the java end
public class StructureInputStream extends ByteArrayInputStream
{
static int curr = 0;
private DataInputStream dis = null;

public StructureInputStream(byte[] b)
{
super(b);
curr = 1;
dis = new DataInputStream((InputStream)this);
}

public final long readLong() throws IOException, EOFException
{
return dis.readLong();
}

public final double readDouble() throws IOException, EOFException
{
return dis.readDouble();
}

public final int readInt() throws IOException, EOFException
{
return dis.readInt();
}

public final int readUnsignedShort() throws IOException, EOFException
{
return dis.readUnsignedShort();
}

public final float readFloat() throws IOException, EOFException
{
return dis.readFloat();
}

public final short readShort() throws IOException, EOFException
{
return dis.readShort();
}

public final char readChar() throws IOException, EOFException
{
return (char)dis.readByte();
}

public void skipBytes(int n) throws IOException
{
int num_bytes_to_skip = n;
int x = 0;

do {
x = dis.skipBytes(num_bytes_to_skip);
num_bytes_to_skip -= x;
} while (num_bytes_to_skip>0);
}
}


// And how the UDP client gets data
// try - catch blocks are omitted for clarity
DatagramPacket pkt = new DatagramPacket(buffer, buffer.length);
socket.receive(pkt);
byte[] data = pkt.getData();
StructureInputStream t = new StructureInputStream(data);
float a1 = t.readFloat(); t.skipBytes(4);
float a2 = t.readFloat(); t.skipBytes(4);
float a3 = t.readFloat(); t.skipBytes(4);
float a4 = t.readFloat(); t.skipBytes(4);

// print out ...


Again, thank you all for your advice.
Regards,
 
Reply With Quote
 
Paul Lutus
Guest
Posts: n/a
 
      09-23-2004
John Adams wrote:

/ ...

> Thank you all for your invaluable advice. To date, I have seemed to
> try all the methods mentioned: Array, ByteBuffer, DataInputStream,
> ByteArrayInputStream, intBitsToFloat(), etc. I had, ofcourse as Michael
> Borgwardt so "brilliantly" pointed out, tried putting some values on the
> server side and dumping them out on the client side (That is HOW I know
> they are not the same!!!).


Okay, to restate a point made by someone else, why are you trying to
transfer the data in binary form? Why not make the server emit the data in
plain-text form? This is a much more robust solution, one that is immune to
nearly all the issues that are occupying your thinking right now.

In other words, you may spend months debugging this binary transfer
solution, only to see your work entirely unraveled by a revision in a
future compiler, or you can use plain text and be done with it.

--
Paul Lutus
http://www.arachnoid.com

 
Reply With Quote
 
Will Hartung
Guest
Posts: n/a
 
      09-23-2004

"Paul Lutus" <(E-Mail Removed)> wrote in message
news:(E-Mail Removed)...
> Okay, to restate a point made by someone else, why are you trying to
> transfer the data in binary form? Why not make the server emit the data in
> plain-text form? This is a much more robust solution, one that is immune

to
> nearly all the issues that are occupying your thinking right now.
>
> In other words, you may spend months debugging this binary transfer
> solution, only to see your work entirely unraveled by a revision in a
> future compiler, or you can use plain text and be done with it.


The choice of format may well be out of the hands of the guy with the
problem. Could be a legacy or 3rd party application.

The fact that there may be other, more appropriate ways to transfer data
doesn't necessarily lessen the need for someone to do exactly what he's
asking.

Regards,

Will Hartung
((E-Mail Removed))



 
Reply With Quote
 
Paul Lutus
Guest
Posts: n/a
 
      09-23-2004
Will Hartung wrote:

/ ...

> The choice of format may well be out of the hands of the guy with the
> problem. Could be a legacy or 3rd party application.
>
> The fact that there may be other, more appropriate ways to transfer data
> doesn't necessarily lessen the need for someone to do exactly what he's
> asking.


Yes, but we don't know that, and the OP may not know the advantages of plain
text. Therefore someone needed to mention it.

--
Paul Lutus
http://www.arachnoid.com

 
Reply With Quote
 
=?ISO-8859-1?Q?Daniel_Sj=F6blom?=
Guest
Posts: n/a
 
      09-23-2004
John Adams wrote:
> Thank you all for your invaluable advice. To date, I have seemed to
> try all the methods mentioned: Array, ByteBuffer, DataInputStream,
> ByteArrayInputStream, intBitsToFloat(), etc. I had, ofcourse as Michael
> Borgwardt so "brilliantly" pointed out, tried putting some values on the
> server side and dumping them out on the client side (That is HOW I know
> they are not the same!!!).
>
> Flatform and hardware also are not the concern since I had tried
> running the server on the very same intel machine as the client (just to
> debug). Endianess is also out of question in this scenario.


Sigh. Have you even read the documentation for DataInputStream? It reads
big endian data, while your x86 is little endian. The other problem is
that your structs probably contain random padding bytes.

Your problem is that you use C structs as a communication format. This
is inherently unportable (since it is in no way defined what a struct
looks like in memory), even on the same platform if you happen to
compile with a different compiler or even another version of the same
compiler. You will need to actually design and specify the transmission
format. XML or binary data with endianess/padding issues removed, do
whatever you wish, but do not dump raw C structs.

--
Daniel Sj÷blom
Remove _NOSPAM to reply by mail
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
read binary data problems pman C++ 3 08-18-2006 08:57 PM
read binary data problems pman C++ 0 08-17-2006 08:06 PM
Binary data stored in SQL Server: can't read from ASP.NET, *can* read from Access? Doug ASP .Net 3 11-04-2005 07:35 PM
How can I Read/Write multiple sequential Binary/Text data files Albert Tu Python 3 03-10-2005 09:16 PM
How do I read binary file data in a test bench? Pete Fraser VHDL 4 11-04-2004 02:51 AM



Advertisments