«

Hi All
I am new to JDOM and I am currently coding a program to build xml
files using JDOM.
The size of the xml file which i am goin to build will be in the order
of a few GB's( 1 to 5 GB).
Will I be able to build such huge documents with JDOM??
Also if there are any resources on jdom benchmarking please let me
know the links.

Thanks in Advance for your help
Vijay Anandh

Any updates on this issue....,
I am eagerly waiting for the answers.

wrote:

> I am new to JDOM and I am currently coding a program to build xml
> files using JDOM.
> The size of the xml file which i am goin to build will be in the order
> of a few GB's( 1 to 5 GB).

I'm not sure what the typical overhead of a JDOM structure is (how much RAM it
takes compared to how much disk-space the equivalent XML would take up), and
it's application-dependent anyway. Assume, for the sake of argument, that it's
3:1.

Under that assumption you'll need around 3 to 15 GB of addressable RAM just to
hold the JDOM structure. So you will have to be running on a 64-bit JVM before
you can even consider this.

Then there's the space required for the JVM itself. And possibly the space
required for the data from which you build your JDOM tree. I would hope that
you don't try to convert the entire tree into a String before writing it out to
file (or to the network), but maybe you would have to do so. So, when you are
considering how much real RAM you need, you have to think in terms of at least
1 GB more than the above estimate, and maybe even (in the worst case) more than
double the estimate.

Only you know whether your application can justify using those kinds of
resources.

Personally, I would think /very/ hard about why I cannot just build the XML
incrementally (writing it out to file in one pass without ever holding all the
data in memory at once). With luck, some thought would show that it is
possible after all.

-- chris

On 12 May 2006 05:03:05 -0700, wrote:
> I am new to JDOM and I am currently coding a program to build xml
>files using JDOM.
> The size of the xml file which i am goin to build will be in the order
>of a few GB's( 1 to 5 GB).
> Will I be able to build such huge documents with JDOM??
> Also if there are any resources on jdom benchmarking please let me
>know the links.

At a time, I found a few JDOM benchmarks, mostly bechmarking speed.
JDOM scored pretty well compared to purely SAX tools, it is one of the
fastest.

As for the maximum size of an XML file... why not test it yourself?

I've just run this simple test case:


import org.jdom.Document;
import org.jdom.Element;

public class JDOM {
// Timeout set to 60 seconds
private static long timeout = 60 * 1000;

public static void main(String[] args) {
Element element = new Element("root");
Document document = new Document(element);
long i = 1;
long startTime = System.currentTimeMillis();

try {
for (;; i++) {
Element currElement = new Element("element" +
String.valueOf(i));
element.addContent(currElement);
element = currElement;
if (System.currentTimeMillis() - startTime > timeout){
break;
}
}
} catch (Exception ex) {
System.out.println("Exception: " + ex.getMessage());
}

System.out.println("Number of created elements: " + i);
}
}

It was able to create 28885 elements in 60 seconds. I left it running
for a while without a timeout, to see if it would throw
OutOfMemoryException, but it seems that the memory overhead for
element is pretty low, so it would take forever to drain the heap.

Domchi

--
Ouroboros ltd. - http://www.ouroboros.hr
Antispam: to reply, remove extra monkey from reply-to address.

On Sun, 14 May 2006 10:00:43 +0100, "Chris Uppal"
<> wrote:
>I'm not sure what the typical overhead of a JDOM structure is (how much RAM it
>takes compared to how much disk-space the equivalent XML would take up), and
>it's application-dependent anyway. Assume, for the sake of argument, that it's
>3:1.

Yes, I'd say that's about right. On the one hand, Java stores Strings
as Unicode, so one character takes 2 bytes - one byte more than on
disk; on the other hand, if you have:

<longxmlelementname>foo</longxmlelementname>

You have to store only two Strings in memory:

longxmlelementname foo

And of course, there's JDOM element overhead. There's the good article
on Java object overhead and memory consumption:

http://www.javaworld.com/javaworld/j...avatip130.html

>Under that assumption you'll need around 3 to 15 GB of addressable RAM just to
>hold the JDOM structure. So you will have to be running on a 64-bit JVM before
>you can even consider this.

It seems that the speed limit is the one you hit before the memory
limit. If it takes a hours to fill even 500 MB of memory with XML
data...

>Personally, I would think /very/ hard about why I cannot just build the XML
>incrementally (writing it out to file in one pass without ever holding all the
>data in memory at once). With luck, some thought would show that it is
>possible after all.

Yes, some kind of SAX parser which doesn't load XML to memory would
probably be more appropriate.

Domchi

--
Ouroboros ltd. - http://www.ouroboros.hr
Antispam: to reply, remove extra monkey from reply-to address.

wrote:
> Hi All
> I am new to JDOM and I am currently coding a program to build xml
> files using JDOM.

A better alternative to JDOM that you might want to consider is XOM
(www.xom.nu) which is much cleaner and easier to use.
--
Dale King