Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Java > Low-latency alternative to Java Object Serialization

Reply
Thread Tools

Low-latency alternative to Java Object Serialization

 
 
Giovanni Azua
Guest
Posts: n/a
 
      10-01-2011
Hi again

I have this lite Client-Server framework based on Blocking IO using classic
java.net.* Sockets (must develop it myself for a grad course project). The
way I am using to pass data over the Sockets is via Serialization i.e.
ObjectOutputStream#writeObject(...) and ObjectInputStream#readObject(...) I
was wondering if anyone can recommend a Serialization framework that would
outperform the vanilla Java default Serialization?

Three years ago I worked for a "high frequency trading" company and they
avoided default Java Serialization like "the devil to the cross" this is a
Spanish idiom btw ... due to its latency. However, I must say that their
remoting framework dated back to the Java stone age and my guess is that the
default Serialization must have improved over the years; I don't have hard
numbers to judge though. I remember JBoss Middleware implementation having
some Serialization framework for this very same reason ... have to check
that too.

Can anyone advice what would be best than Java Serialization without
requiring an unreasonably heavy dependency footprint?

Many thanks in advance,
Best regards,
Giovanni

 
Reply With Quote
 
 
 
 
Lew
Guest
Posts: n/a
 
      10-01-2011
Giovanni Azua wrote:
> I have this lite Client-Server framework based on Blocking IO using classic
> java.net.* Sockets (must develop it myself for a grad course project). The
> way I am using to pass data over the Sockets is via Serialization i.e.
> ObjectOutputStream#writeObject(...) and ObjectInputStream#readObject(...)I
> was wondering if anyone can recommend a Serialization framework that would
> outperform the vanilla Java default Serialization?
>
> Three years ago I worked for a "high frequency trading" company and they
> avoided default Java Serialization like "the devil to the cross" this is a
> Spanish idiom btw ... due to its latency. However, I must say that their
> remoting framework dated back to the Java stone age and my guess is that the
> default Serialization must have improved over the years; I don't have hard
> numbers to judge though. I remember JBoss Middleware implementation having
> some Serialization framework for this very same reason ... have to check
> that too.
>
> Can anyone advice what would be best than Java Serialization without
> requiring an unreasonably heavy dependency footprint?


Side bar: What exactly do you mean by "latency" here?

Serialization assumes no knowledge on the restoring end about the structures to restore, so all knowledge has to reside in the serialization format.

Circular dependencies, inheritance chains, the whole megillah has to be encoded into the serialized stream.

Serialization is designed to store and restore object graphs, not the data in them.

Take a page from web services and create an XML schema to represent the *data* you wish to transfer. This assumes knowledge on both ends of the structures used to hold the data, unlike object serialization, hence much less information must flow between the participants.

Use JAXB to generate the classes used to process that schema and incorporate those classes into the protocol at both ends.

Fast, standard and fairly low effort and low maintenance, assuming you haveversion control and continuous integration (CI).

By "fast" I mean both to develop and to operate.

You will write custom code to jam the data into your JAXB-generated structures and retrieve them therefrom.

But you will be transmitting data via a format that omits the object graph overhead and focuses on just the data to share. The object-graph knowledgeis coded into the application and need not be transferred.

XML is awesome for this kind of task.

--
Lew

 
Reply With Quote
 
 
 
 
markspace
Guest
Posts: n/a
 
      10-01-2011
On 10/1/2011 5:46 AM, Giovanni Azua wrote:
> Three years ago I worked for a "high frequency trading" company and they
> avoided default Java Serialization like "the devil to the cross"



Just because "avoid serialization" was a requirement for your previous
work, doesn't mean that it should be a requirement for every project
after that.

Frequently, the low-developer cost of Java serialization overrides all
other concerns. The increase in CPU costs and network bandwidth it
requires is very cheap. DO NOT work around Java serialization unless
you are sure you need to. I.e., after careful analysis (and profiling)
of a working app or prototype.

If you do need to work around Java serialization, look at
Externalizable interface.

http://java.sun.com/developer/techni...serialization/

Note the sections on "gotchas" in that article. Esp. both the caching
and the performance considerations.

Totally rolling your own protocol is possible too if you need the utmost
performance. 'Tain't hard. 'Tain't easy either. Data IO Streams are a
good compromise between higher level serialization and raw sockets.

<http://download.oracle.com/javase/7/docs/api/java/io/DataOutputStream.html>

 
Reply With Quote
 
Robert Klemme
Guest
Posts: n/a
 
      10-01-2011
On 10/01/2011 06:19 PM, Lew wrote:
> Giovanni Azua wrote:
>> I have this lite Client-Server framework based on Blocking IO using classic
>> java.net.* Sockets (must develop it myself for a grad course project). The
>> way I am using to pass data over the Sockets is via Serialization i.e.
>> ObjectOutputStream#writeObject(...) and ObjectInputStream#readObject(...) I
>> was wondering if anyone can recommend a Serialization framework that would
>> outperform the vanilla Java default Serialization?
>>
>> Three years ago I worked for a "high frequency trading" company and they
>> avoided default Java Serialization like "the devil to the cross" this is a
>> Spanish idiom btw ... due to its latency. However, I must say that their
>> remoting framework dated back to the Java stone age and my guess is that the
>> default Serialization must have improved over the years; I don't have hard
>> numbers to judge though. I remember JBoss Middleware implementation having
>> some Serialization framework for this very same reason ... have to check
>> that too.
>>
>> Can anyone advice what would be best than Java Serialization without
>> requiring an unreasonably heavy dependency footprint?

>
> Side bar: What exactly do you mean by "latency" here?
>
> Serialization assumes no knowledge on the restoring end about the structures to restore, so all knowledge has to reside in the serialization format.
>
> Circular dependencies, inheritance chains, the whole megillah has to be encoded into the serialized stream.
>
> Serialization is designed to store and restore object graphs, not the data in them.
>
> Take a page from web services and create an XML schema to represent the *data* you wish to transfer. This assumes knowledge on both ends of the structures used to hold the data, unlike object serialization, hence much less information must flow between the participants.
>
> Use JAXB to generate the classes used to process that schema and incorporate those classes into the protocol at both ends.
>
> Fast, standard and fairly low effort and low maintenance, assuming you have version control and continuous integration (CI).
>
> By "fast" I mean both to develop and to operate.
>
> You will write custom code to jam the data into your JAXB-generated structures and retrieve them therefrom.
>
> But you will be transmitting data via a format that omits the object graph overhead and focuses on just the data to share. The object-graph knowledge is coded into the application and need not be transferred.
>
> XML is awesome for this kind of task.


http://www.json.org/ might also be a good alternative which - depending
on format etc. - can be less verbose. See http://json.org/example.html

Kind regards

robert
 
Reply With Quote
 
Robert Klemme
Guest
Posts: n/a
 
      10-02-2011
On 01.10.2011 21:35, jebblue wrote:
> On Sat, 01 Oct 2011 21:13:40 +0200, Robert Klemme wrote:
>
>> On 10/01/2011 06:19 PM, Lew wrote:
>>> Giovanni Azua wrote:
>>>> I have this lite Client-Server framework based on Blocking IO using
>>>> classic java.net.* Sockets (must develop it myself for a grad course
>>>> project). The way I am using to pass data over the Sockets is via
>>>> Serialization i.e. ObjectOutputStream#writeObject(...) and
>>>> ObjectInputStream#readObject(...) I was wondering if anyone can
>>>> recommend a Serialization framework that would outperform the vanilla
>>>> Java default Serialization?
>>>>
>>> But you will be transmitting data via a format that omits the object
>>> graph overhead and focuses on just the data to share. The object-graph
>>> knowledge is coded into the application and need not be transferred.
>>>
>>> XML is awesome for this kind of task.

>>
>> http://www.json.org/ might also be a good alternative which - depending
>> on format etc. - can be less verbose. See http://json.org/example.html
>>

>
> JSON is convenient for JavaScript heads, it is not human readable,
> this is one reason why XML exists in the first place.


I am not sure why you say JSON is not human readable while XML is.
Remember: for network transfer you would use the most compressed format
of either which means that for XML you would not have line breaks and
indentation. I'd say an XML on one line with a reasonable complex
structure is not human readable.

> JSON was
> a mistake, instead of coming up with an arcane hacked syntax
> to replace XML; JavaScript should have been improved to handle
> XML.


That sounds like opinion to me. Can you provide any real arguments why
XML should be chosen for as a data transfer format over JSON?

XML does have some overhead and often uses more bytes to represent the
same structure.

There's also an interesting discussion at stackoverflow:
http://stackoverflow.com/questions/2...nd-xml#2636380

Kind regards

robert

--
remember.guy do |as, often| as.you_can - without end
http://blog.rubybestpractices.com/
 
Reply With Quote
 
Robert Klemme
Guest
Posts: n/a
 
      10-02-2011
On 01.10.2011 14:46, Giovanni Azua wrote:
> Hi again
>
> I have this lite Client-Server framework based on Blocking IO using classic
> java.net.* Sockets (must develop it myself for a grad course project). The
> way I am using to pass data over the Sockets is via Serialization i.e.
> ObjectOutputStream#writeObject(...) and ObjectInputStream#readObject(...) I
> was wondering if anyone can recommend a Serialization framework that would
> outperform the vanilla Java default Serialization?
>
> Three years ago I worked for a "high frequency trading" company and they
> avoided default Java Serialization like "the devil to the cross" this is a
> Spanish idiom btw ... due to its latency. However, I must say that their
> remoting framework dated back to the Java stone age and my guess is that the
> default Serialization must have improved over the years; I don't have hard
> numbers to judge though. I remember JBoss Middleware implementation having
> some Serialization framework for this very same reason ... have to check
> that too.
>
> Can anyone advice what would be best than Java Serialization without
> requiring an unreasonably heavy dependency footprint?


Btw, there is a completely different option not mentioned so far: CORBA
with IIOP which was specifically designed for remote communication. Of
course this would mean that you had to exchange your complete
communication layer - but I wanted to mention it because I believe CORBA
is used too rarely because it somehow seems out of fashion. But if you
look at network bandwidth used I believe CORBA is a pretty good
contender compared to SOAP for example.

Kind regards

robert

--
remember.guy do |as, often| as.you_can - without end
http://blog.rubybestpractices.com/
 
Reply With Quote
 
Tom Anderson
Guest
Posts: n/a
 
      10-03-2011
On Sat, 1 Oct 2011, Giovanni Azua wrote:

> I was wondering if anyone can recommend a Serialization framework that
> would outperform the vanilla Java default Serialization?


Swords not words:

https://github.com/eishay/jvm-serializers/wiki/

I sent them a patch to add JBoss Serialization a while ago, but they
haven't taken it. I should try again now the project is on GitHub.

> I remember JBoss Middleware implementation having some Serialization
> framework for this very same reason ... have to check that too.


It's pretty good. More or less plug-compatible with JDK serialization at
the API level (as in, it doesn't need schema generation or weird
interfaces or anything), and much faster. From what i remember of my
benchmarks, it was faster than any of the textual formats, and only a bit
slower than the schema-based binary formats like Protocol Buffers.

tom

--
Now I am thoroughly confused. -- Colin Brace sums up RT3090 support
in Linux
 
Reply With Quote
 
Tom Anderson
Guest
Posts: n/a
 
      10-03-2011
On Sat, 1 Oct 2011, Lew wrote:

> Giovanni Azua wrote:
>
>> Can anyone advice what would be best than Java Serialization without
>> requiring an unreasonably heavy dependency footprint?

>
> Serialization assumes no knowledge on the restoring end about the
> structures to restore, so all knowledge has to reside in the
> serialization format.
>
> Circular dependencies, inheritance chains, the whole megillah has to be
> encoded into the serialized stream.
>
> Serialization is designed to store and restore object graphs, not the
> data in them.
>
> Take a page from web services and create an XML schema to represent the
> *data* you wish to transfer. This assumes knowledge on both ends of the
> structures used to hold the data, unlike object serialization, hence
> much less information must flow between the participants.
>
> Use JAXB to generate the classes used to process that schema and
> incorporate those classes into the protocol at both ends.
>
> Fast, standard and fairly low effort and low maintenance, assuming you
> have version control and continuous integration (CI).
>
> By "fast" I mean both to develop and to operate.


Interesting. I do not believe this to be true. Specifically, i believe
that: (a) developing an XML-based transfer format using JAXB will take
considerably more effort than using standard serialization, or an equally
convenient library such as JBoss Serialization, although still not a large
amount of effort, certainly; (b) the data will be larger than
with standard serialization (because the "object graph overhead" is not
actually that large, and XML is much less space-efficient than
serialization's binary format); and (c) the speed of operation, even
assuming an infinitely fast network, will be lower.

One get-out clause: for very short streams (one or a few objects), XML
might beat standard serialization for space and speed. Standard
serialization does have some per-class overhead, which is
disproportionately expensive for short streams.

tom

--
Now I am thoroughly confused. -- Colin Brace sums up RT3090 support
in Linux
 
Reply With Quote
 
Roedy Green
Guest
Posts: n/a
 
      10-03-2011
On Sat, 01 Oct 2011 14:35:17 -0500, jebblue <(E-Mail Removed)> wrote, quoted or
indirectly quoted someone who said :

>JSON is convenient for JavaScript heads, it is not human readable,

JSON is reads much like Java source code. I find it easier
understand than XML even though I know XML much better.

You might have been looking at some compressed JSON, or encrypted SSL
traffic. XML would be just as inscrutable if you compressed it. It
compresses well. (This is not a compliment).
--
Roedy Green Canadian Mind Products
http://mindprod.com
It should not be considered an error when the user starts something
already started or stops something already stopped. This applies
to browsers, services, editors... It is inexcusable to
punish the user by requiring some elaborate sequence to atone,
e.g. open the task editor, find and kill some processes.

 
Reply With Quote
 
Roedy Green
Guest
Posts: n/a
 
      10-04-2011
On Mon, 3 Oct 2011 19:24:20 +0100, Tom Anderson <(E-Mail Removed)>
wrote, quoted or indirectly quoted someone who said :

>Specifically, i believe
>that: (a) developing an XML-based transfer format using JAXB will take
>considerably more effort than using standard serialization


Serialisation handles complex data structures, even loops. XML is
limited to trees.

Serialisation handles any imaginable data type without extra work. XML
requires inventing an external character representation and a way of
converting to chars and back.

Serialisation is hard to upgrade. XML is easy. Serialisation pretty
much requires everyone to stay in sync with identical software. XML
allows clients with out of date software, software in other languages,
or even no software at all.
--
Roedy Green Canadian Mind Products
http://mindprod.com
It should not be considered an error when the user starts something
already started or stops something already stopped. This applies
to browsers, services, editors... It is inexcusable to
punish the user by requiring some elaborate sequence to atone,
e.g. open the task editor, find and kill some processes.

 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
how to move from java object serialization to xml serialization? Dimitri Ognibene Java 4 09-02-2006 07:32 AM
Object serialization XML vs java serialization plasticfloor@gmail.com Java 3 06-14-2006 03:45 AM
Serialization Problems and books on serialization? sinleeh@hotmail.com Java 8 01-02-2005 02:40 PM
avoiding XML serialization, different WSDL generation, soap serialization Ramunas Urbonas ASP .Net Web Services 1 07-27-2004 09:57 PM
alternative XML serialization - how? Shone ASP .Net Web Services 3 07-19-2004 02:43 PM



Advertisments