Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > XML > Memory Consumption in JDOM

Reply
Thread Tools

Memory Consumption in JDOM

 
 
Vijay Anandh
Guest
Posts: n/a
 
      05-13-2006
Hi All,
I am new to JDOM and I am currently coding a program to build xml
files using JDOM.
The size of the xml file which i am goin to build will be in the order
of a few GB's( 1 to 5 GB).
Will I be able to build such huge documents with JDOM??
Also if there are any resources on jdom benchmarking please let me
know the links.
( BTW, I am using the latest JDOM build ).

Thanks in Advance for your help
Vijay Anandh

 
Reply With Quote
 
 
 
 
Vijay Anandh
Guest
Posts: n/a
 
      05-13-2006
JDOM experts........, help me please on my above question.
I am eagerly waiting...,

 
Reply With Quote
 
 
 
 
=?ISO-8859-1?Q?J=FCrgen_Kahrs?=
Guest
Posts: n/a
 
      05-13-2006
Vijay Anandh wrote:

> I am new to JDOM and I am currently coding a program to build xml
> files using JDOM.
> The size of the xml file which i am goin to build will be in the order
> of a few GB's( 1 to 5 GB).
> Will I be able to build such huge documents with JDOM??


No, probably not. In this newsgroup, we have
explained many many times what a DOM does:
Read the complete XML file into memory.
This procedure needs at least so much memory
(RAM) as the XML file takes on disk.

So, unless you have many GB of RAM, the answer is no.
Use Google to find all the other details that
have been posted in this newsgroup earlier.
 
Reply With Quote
 
Joe Kesselman
Guest
Posts: n/a
 
      05-13-2006
Jürgen Kahrs wrote:
> So, unless you have many GB of RAM, the answer is no.


Slight quibble: Most systems these days support virtual memory, so it is
theoretically possible to build models that exceed the size of physical
memory if you're willing to spill to disk. But that costs performance
and Java itself may impose some memory-size limits.

There are two possible approaches. One is to do active memory
management. Another is to replace the JDOM (which is a rather naive
design, I believe, consuming an object per node) or DOM with a more
compact data model; this is part of why we invented the DTM model for Xalan.

If you don't want to do a lot of coding to reinvent those solutions, you
may want to investigate XML databases. Managing huge amounts of data is
*supposed* to be what they're tuned for, and they should have applied
these sorts of tricks. (Note that I say "should"; I can't advise you on
which ones are good or not other than to say that I'm favorably
impressed with IBM's new XML support in the latest version of DB2. But
I'm an IBMer, so I may be biased.)


--
() ASCII Ribbon Campaign | Joe Kesselman
/\ Stamp out HTML e-mail! | System architexture and kinetic poetry
 
Reply With Quote
 
Vijay Anandh
Guest
Posts: n/a
 
      05-13-2006
Thanks for your replies..
I have tried benchmarking my JDOM application
on our solaris 8 server( more RAM now ).

sizes of files to create 10M 20M 30M 40M
increase heap size to none none 128M 128M
~ time taken in seconds 2 4 7 11


And fortunately the maximum file size I will be creating is ~70M.
So I will right now persist with JDOM ( while increasing my JVM heap
size )
and test its limits and then plan for XML databases

Vijay Anandh

 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
memory consumption in JDOM vanandh22@gmail.com Java 5 05-15-2006 05:53 PM
java.lang.NoSuchMethodError: org.jdom.Element: method getParent()Lorg/jdom/Element Tinker Java 4 10-09-2005 03:12 PM
JDOM: java.lang.NoClassDefFoundError: org/jdom/Content Bernd Oninger Java 4 06-21-2004 09:08 PM
JDOM: java.lang.NoClassDefFoundError: org/jdom/Content Bernd Oninger XML 3 06-21-2004 09:08 PM
Help with JDOM, turn org.jdom.Document -> org.w3c.dom.Document? Wendy S Java 1 08-04-2003 11:48 PM



Advertisments