Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Java > Re: OutOfMemoryError when using SAX to process an XML document

Reply
Thread Tools

Re: OutOfMemoryError when using SAX to process an XML document

 
 
Robert Olofsson
Guest
Posts: n/a
 
      07-02-2003
: I was wondering if anyone has encountered an OutOfMemoryError while using
: SAX to parse an XML file and if so, how did you resolve the problem? The
: file itself is quite large (about 29717K) and I've tried increasing the swap
: space on the JVM memory to 1GB, but it still hangs after a certain point.

Seems like your leaking memory. When parsing documents with dom4j using
_DOM_ memory seems to be around 6 times the file size (50MB file=>300MB
memory usage).
With SAX I would expect that you dont use very much memory unless you build
huge data structures.

Grab a profiler and find out what references you keep around.

How did you increase the jvm memory? with the -Xmx1024m flag or?

/robo
 
Reply With Quote
 
 
 
 
Sunitha Kumar
Guest
Posts: n/a
 
      07-02-2003
"Robert Olofsson" <(E-Mail Removed)> wrote in message
news:bdttg8$5bg$(E-Mail Removed)...
> Seems like your leaking memory. When parsing documents with dom4j using
> _DOM_ memory seems to be around 6 times the file size (50MB file=>300MB
> memory usage). With SAX I would expect that you dont use very much memory

unless you build
> huge data structures.


I'm not really creating a huge data structure - just a simple class with
Strings references and one ArrayList object - I call its toString method in
the endElement method, so there isn't really anything hanging around.

> Grab a profiler and find out what references you keep around.


Any profiler you recommend? I'm using Eclipse.

> How did you increase the jvm memory? with the -Xmx1024m flag or?


Yes, with the -Xms512m -Xmx1024m flag.

> /robo

Thanks for your suggestions - I'm googling for an Eclipse profiler as I
write this.


 
Reply With Quote
 
 
 
 
Robert Olofsson
Guest
Posts: n/a
 
      07-02-2003
Sunitha Kumar ((E-Mail Removed)) wrote:
: I'm not really creating a huge data structure - just a simple class with
: Strings references and one ArrayList object - I call its toString method in
: the endElement method, so there isn't really anything hanging around.

: > Grab a profiler and find out what references you keep around.
: Any profiler you recommend? I'm using Eclipse.

Well, I do develop an open source profiler called jmp. It works nice
for me:
http://www.khelekore.org/jmp/

On sourceforge you will also find a few other jvmpi-based free profilers.
mjp, http://mjp.sourceforge.net/, may work for you, I have not
used it myself

I also know that there are a few eclipse plugins for profiling available
but havent really tried any since I use emacs (and sometimes vi).

/robo
 
Reply With Quote
 
Sunitha Kumar
Guest
Posts: n/a
 
      07-03-2003
> Well, I do develop an open source profiler called jmp. It works nice
> for me:
> http://www.khelekore.org/jmp/
> On sourceforge you will also find a few other jvmpi-based free profilers.
> mjp, http://mjp.sourceforge.net/, may work for you, I have not
> used it myself
>
> I also know that there are a few eclipse plugins for profiling available
> but havent really tried any since I use emacs (and sometimes vi).


Robo, thanks for your advice - I've never used a profiler before, but I'll
make sure to try your implementation out.: )
BTW, for the record, it isn't a memory leak - I tried parsing the document,
without the callbacks and found to my dismay that the SAX parser wasn't able
to parse the document completely. In the process, I ran across an altogether
new error:

Error: org.xml.sax.SAXParseException: Parser has reached the entity
expansion limit "64,000" set by the Application.

A google search turned this document up:
http://java.sun.com/webservices/docs...ma-limitations,
but I'm not sure if changing the entity expansion limit will really do
anything for me.

Any suggestions, comments, advice?

Sunitha/.


 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Different results parsing a XML file with XML::Simple (XML::Sax vs. XML::Parser) Erik Wasser Perl Misc 5 03-05-2006 10:09 PM
Print XML parsing to JspWriter (out) Class org.xml.sax.helpers.NewInstance can not access a member of class javax.xml.parsers.SAXParser with modifiers "protected" Per Magnus L?vold Java 0 11-15-2004 02:27 PM
Help on including one XML document within another XML document using XML Schemas Tony Prichard XML 0 12-12-2003 03:18 PM
Is it possible to get element type specified by schema while parsingthe xml document using SAX/DOM? Jari Kujansuu XML 2 09-30-2003 03:14 PM



Advertisments