Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Perl > Perl Misc > How to avoid Out of Memory Errors when dealing with a large XML file?

Reply
Thread Tools

How to avoid Out of Memory Errors when dealing with a large XML file?

 
 
Saqib Ali
Guest
Posts: n/a
 
      01-10-2011
Hi.

I'm reading a large (57 MB) XML file Using XML::XPath::XMLParser()

I keep getting this error:

"Callback called exit at XML/XPath/Node/Element.pm at line 144 during
global destruction."

I'm using Windows XP. So I watched the task-management memory meter
during the execution of this process. The PERL process chewed up a lot
of the "available memory". But when the process died, it still showed
about 216MB available memory.

Is there anything I can do to work-around this problem? From reading
responses to other similar questions, the only option may be to use a
XML stream parser instead of one that builds the entire DOM tree
internally.
 
Reply With Quote
 
 
 
 
Chris Nehren
Guest
Posts: n/a
 
      01-11-2011
On 2011-01-10, Saqib Ali scribbled these curious markings:
> Hi.
>
> I'm reading a large (57 MB) XML file Using XML::XPath::XMLParser()
>
> I keep getting this error:
>
> "Callback called exit at XML/XPath/Node/Element.pm at line 144 during
> global destruction."
>
> I'm using Windows XP. So I watched the task-management memory meter
> during the execution of this process. The PERL process chewed up a lot
> of the "available memory". But when the process died, it still showed
> about 216MB available memory.


Firstly, as someone or something has inflicted Windows on you, are you
using Strawberry Perl like every sane Windows user (bit of an oxymoron,
that)? Second, calling it "PERL" makes you look either ignorant or
trollish, so don't do that.

> Is there anything I can do to work-around this problem? From reading
> responses to other similar questions, the only option may be to use a
> XML stream parser instead of one that builds the entire DOM tree
> internally.


Yes, you'll want a SAX parser. Strawberry Perl ships with XML::LibXML,
which provides a SAX interface via XML::LibXML::SAX.

--
Thanks and best regards,
Chris Nehren
 
Reply With Quote
 
 
 
 
Saqib Ali
Guest
Posts: n/a
 
      01-14-2011
FYI, someone posted a workaround here: http://osdir.com/ml/perl-beginners/2.../msg00182.html
It worked for me.


I'm using Perl provided by ActiveState.




On Jan 11, 3:36*am, Chris Nehren <apei...@isuckatdomains.net.invalid>
wrote:
> On 2011-01-10, Saqib Ali scribbled these curious markings:
>
> > Hi.

>
> > I'm reading a large (57 MB) XML file Using XML::XPath::XMLParser()

>
> > I keep getting this error:

>
> > "Callback called exit at XML/XPath/Node/Element.pm at line 144 during
> > global destruction."

>
> > I'm using Windows XP. So I watched the task-management memory meter
> > during the execution of this process. The PERL process chewed up a lot
> > of the "available memory". But when the process died, it still showed
> > about 216MB available memory.

>
> Firstly, as someone or something has inflicted Windows on you, are you
> using Strawberry Perl like every sane Windows user (bit of an oxymoron,
> that)? Second, calling it "PERL" makes you look either ignorant or
> trollish, so don't do that.
>
> > Is there anything I can do to work-around this problem? From reading
> > responses to other similar questions, the only option may be to use a
> > XML stream parser instead of one that builds the entire DOM tree
> > internally.

>
> Yes, you'll want a SAX parser. Strawberry Perl ships with XML::LibXML,
> which provides a SAX interface via XML::LibXML::SAX.
>
> --
> Thanks and best regards,
> Chris Nehren


 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Re: How include a large array? Edward A. Falk C Programming 1 04-04-2013 08:07 PM
Dealing with large memory under linux or How to Build 64bit Python proteusguy@gmail.com Python 1 01-22-2009 08:30 PM
dealing with nested xml within nested xml within...... Ultrus Python 3 07-09-2007 09:00 PM
Out-of-memory errors and caching errors. George1776 ASP .Net 2 09-14-2006 03:34 PM
Errors, errors, errors Mark Goldin ASP .Net 2 01-17-2004 08:05 PM



Advertisments
 



1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57