Velocity Reviews

Velocity Reviews (http://www.velocityreviews.com/forums/index.php)
-   Perl Misc (http://www.velocityreviews.com/forums/f67-perl-misc.html)
-   -   How to avoid Out of Memory Errors when dealing with a large XML file? (http://www.velocityreviews.com/forums/t914468-how-to-avoid-out-of-memory-errors-when-dealing-with-a-large-xml-file.html)

Saqib Ali 01-10-2011 10:50 PM

How to avoid Out of Memory Errors when dealing with a large XML file?
 
Hi.

I'm reading a large (57 MB) XML file Using XML::XPath::XMLParser()

I keep getting this error:

"Callback called exit at XML/XPath/Node/Element.pm at line 144 during
global destruction."

I'm using Windows XP. So I watched the task-management memory meter
during the execution of this process. The PERL process chewed up a lot
of the "available memory". But when the process died, it still showed
about 216MB available memory.

Is there anything I can do to work-around this problem? From reading
responses to other similar questions, the only option may be to use a
XML stream parser instead of one that builds the entire DOM tree
internally.

Chris Nehren 01-11-2011 08:36 AM

Re: How to avoid Out of Memory Errors when dealing with a large XMLfile?
 
On 2011-01-10, Saqib Ali scribbled these curious markings:
> Hi.
>
> I'm reading a large (57 MB) XML file Using XML::XPath::XMLParser()
>
> I keep getting this error:
>
> "Callback called exit at XML/XPath/Node/Element.pm at line 144 during
> global destruction."
>
> I'm using Windows XP. So I watched the task-management memory meter
> during the execution of this process. The PERL process chewed up a lot
> of the "available memory". But when the process died, it still showed
> about 216MB available memory.


Firstly, as someone or something has inflicted Windows on you, are you
using Strawberry Perl like every sane Windows user (bit of an oxymoron,
that)? Second, calling it "PERL" makes you look either ignorant or
trollish, so don't do that. :)

> Is there anything I can do to work-around this problem? From reading
> responses to other similar questions, the only option may be to use a
> XML stream parser instead of one that builds the entire DOM tree
> internally.


Yes, you'll want a SAX parser. Strawberry Perl ships with XML::LibXML,
which provides a SAX interface via XML::LibXML::SAX.

--
Thanks and best regards,
Chris Nehren

Saqib Ali 01-14-2011 09:31 PM

Re: How to avoid Out of Memory Errors when dealing with a large XML file?
 
FYI, someone posted a workaround here: http://osdir.com/ml/perl-beginners/2.../msg00182.html
It worked for me.


I'm using Perl provided by ActiveState.




On Jan 11, 3:36*am, Chris Nehren <apei...@isuckatdomains.net.invalid>
wrote:
> On 2011-01-10, Saqib Ali scribbled these curious markings:
>
> > Hi.

>
> > I'm reading a large (57 MB) XML file Using XML::XPath::XMLParser()

>
> > I keep getting this error:

>
> > "Callback called exit at XML/XPath/Node/Element.pm at line 144 during
> > global destruction."

>
> > I'm using Windows XP. So I watched the task-management memory meter
> > during the execution of this process. The PERL process chewed up a lot
> > of the "available memory". But when the process died, it still showed
> > about 216MB available memory.

>
> Firstly, as someone or something has inflicted Windows on you, are you
> using Strawberry Perl like every sane Windows user (bit of an oxymoron,
> that)? Second, calling it "PERL" makes you look either ignorant or
> trollish, so don't do that. :)
>
> > Is there anything I can do to work-around this problem? From reading
> > responses to other similar questions, the only option may be to use a
> > XML stream parser instead of one that builds the entire DOM tree
> > internally.

>
> Yes, you'll want a SAX parser. Strawberry Perl ships with XML::LibXML,
> which provides a SAX interface via XML::LibXML::SAX.
>
> --
> Thanks and best regards,
> Chris Nehren




All times are GMT. The time now is 11:26 PM.

Powered by vBulletin®. Copyright ©2000 - 2014, vBulletin Solutions, Inc.
SEO by vBSEO ©2010, Crawlability, Inc.