Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Perl > Perl Misc > ExpatXS: 'Can't call method "read" on an undefined value' after ca.500 XML files

Reply
Thread Tools

ExpatXS: 'Can't call method "read" on an undefined value' after ca.500 XML files

 
 
Arvin Portlock
Guest
Posts: n/a
 
      01-14-2006
In using ExpatXS to parse large batches of XML files
something curious happens. After parsing something close
to 500 files the program crashes with the error:

Can't call method "read" on an undefined value at
/xxx/perl5.8/lib/site_perl/5.8.7/sun4-solaris-thread-multi/
XML/SAX/ExpatXS.pm line 155.

Of course I examined the files where the errors occurred
and there is nothing wrong with them. I can do this on any
batch of files and get the same problem--permissions are
always set correctly, the files are confirmed well-formed,
etc.

Sounds like something's leaking.

I reduced my big complex program down to something very
simple which doesn't actually do anything. It still crashes
after (successfully) parsing around 500 files. Can somebody
test this script on a large batch of files, or otherwise tell
me what I'm (or ExpatXS) is doing wrong? Maybe it's a problem
with XML::Filter?

# $Id: ExpatXS.pm,v 1.39 2005/11/10 09:38:31 cvspetr Exp $

This is perl, v5.8.7 built for sun4-solaris-thread-multi

Simple program follows:

use XML::SAX;
use XML::Filter::BufferText;
use File:ath;
use strict 'vars';

my $dir = '/home/xmldocs/lib1';
opendir (DIR, $dir);
my @xmlfiles = grep (/\.xml/, readdir (DIR));
closedir (DIR);

foreach my $file (sort @xmlfiles) {
my $fullpath = "$dir/$file";
print STDERR "$fullpath\n";
my $handler = new MySAXHandler;
my $filter = new XML::Filter::BufferText (Handler => $handler);
my $parser = XML::SAX:arserFactory->parser(Handler => $filter);
$parser->parse_uri($fullpath);
undef $parser;
undef $filter;
undef $handler;
}

package MySAXHandler;
use base qw(XML::SAX::Base);

sub start_element {
}

sub end_element {
}

 
Reply With Quote
 
 
 
 
Matt Garrish
Guest
Posts: n/a
 
      01-15-2006

"Arvin Portlock" <(E-Mail Removed)> wrote in message
news:dq9gb7$219p$(E-Mail Removed)...
> In using ExpatXS to parse large batches of XML files
> something curious happens. After parsing something close
> to 500 files the program crashes with the error:
>
> Can't call method "read" on an undefined value at
> /xxx/perl5.8/lib/site_perl/5.8.7/sun4-solaris-thread-multi/
> XML/SAX/ExpatXS.pm line 155.
>


You aren't checking whether you succesfully get an object. I would doubt
it's a memory leak or your system would have ground to a halt. I don't see
where you're trying to call a read method anywhere in your code, but I would
suggest wrapping the objects in an eval and see if that gives you more info.
At the very least it should isolate which module is causing the problem:

my ($filter, $parser);

eval { $filter = new XML::Filter::BufferText (Handler => $handler); };

if ($@) {
die "Couldn't get a new Buffer Object: $@\n";
}

eval { $parser = XML::SAX:arserFactory->parser(Handler => $filter); };


if ($@) {
die "Couldn't get a new ParserFactory Object: $@\n";
}


Matt


 
Reply With Quote
 
 
 
 
Arvin Portlock
Guest
Posts: n/a
 
      01-15-2006
Matt Garrish wrote:

> "Arvin Portlock" wrote in message
> news:dq9gb7$219p$(E-Mail Removed)...
>
> >In using ExpatXS to parse large batches of XML files
> >something curious happens. After parsing something close
> >to 500 files the program crashes with the error:
> >
> >Can't call method "read" on an undefined value at
> >/xxx/perl5.8/lib/site_perl/5.8.7/sun4-solaris-thread-multi/
> >XML/SAX/ExpatXS.pm line 155.
> >

>
>
> I don't see
> where you're trying to call a read method anywhere in your code


I don't understand what you mean by a read method. Isn't that
what $parser->parse_uri($fullpath) does? I'm doing something
right at least because the program works perfectly (reading and
manipulating xml files) until somewhere around the 500th file.

> , but I would
> suggest wrapping the objects in an eval and see if that gives you more
> info.
> At the very least it should isolate which module is causing the problem:
>
> my ($filter, $parser);
>
> eval { $filter = new XML::Filter::BufferText (Handler => $handler); };


I wrapped an eval around the $parser->parse_uri and it simply
gave me the same error message "Can't call method" about a
thousand times, for each of the subsequent files after the
first failure. In other words, it didn't give me an error for
a single file then happily continue working with the rest. Once
it fails it fails for good.

I tried your suggestion with the other objects but no error
messages there. And thank you for the idea, I'll try to go into
ExpatSX.pm and try it a few places in there to see if I can
get a little closer to the source of the problem.


use XML::SAX;
use XML::Filter::BufferText;
use File:ath;
use strict 'vars';

my $dir = '/home/xmldocs/lib1';
opendir (DIR, $dir);
my @xmlfiles = grep (/\.xml/, readdir (DIR));
closedir (DIR);

foreach my $file (sort @xmlfiles) {
my $fullpath = "$dir/$file";
print STDERR "$fullpath\n";
my $handler = new MySAXHandler;
my $filter = new XML::Filter::BufferText (Handler => $handler);
my $parser = XML::SAX:arserFactory->parser(Handler => $filter);
$parser->parse_uri($fullpath);
undef $parser;
undef $filter;
undef $handler;
}

package MySAXHandler;
use base qw(XML::SAX::Base);

sub start_element {
}

sub end_element {
}

 
Reply With Quote
 
Arvin Portlock
Guest
Posts: n/a
 
      01-15-2006
Matt Garrish wrote:

> "Arvin Portlock" wrote in message
> news:dq9gb7$219p$(E-Mail Removed)...
>
> >In using ExpatXS to parse large batches of XML files
> >something curious happens. After parsing something close
> >to 500 files the program crashes with the error:
> >
> >Can't call method "read" on an undefined value at
> >/xxx/perl5.8/lib/site_perl/5.8.7/sun4-solaris-thread-multi/
> >XML/SAX/ExpatXS.pm line 155.
> >

>
>
> I would doubt it's a memory leak or your system would have ground to a
> halt.


That's true. Now that I think about it it sounds more like
file handles aren't being closed and at some point I simply
run out. I would assume all that is taken care of during
parser destruction when it goes out of scope or is explicitely
deleted as I have done. Now I have something more concrete to
check.

Arvin.

 
Reply With Quote
 
xhoster@gmail.com
Guest
Posts: n/a
 
      01-15-2006
"Matt Garrish" <(E-Mail Removed)> wrote:
> "Arvin Portlock" <(E-Mail Removed)> wrote in message
> news:dq9gb7$219p$(E-Mail Removed)...
> > In using ExpatXS to parse large batches of XML files
> > something curious happens. After parsing something close
> > to 500 files the program crashes with the error:
> >
> > Can't call method "read" on an undefined value at
> > /xxx/perl5.8/lib/site_perl/5.8.7/sun4-solaris-thread-multi/
> > XML/SAX/ExpatXS.pm line 155.
> >

>
> You aren't checking whether you succesfully get an object. I would doubt
> it's a memory leak or your system would have ground to a halt.


But it could easily be a file-handle leak. Around 500 would be a common
per-process limit on open file-handles.

Xho

--
-------------------- http://NewsReader.Com/ --------------------
Usenet Newsgroup Service $9.95/Month 30GB
 
Reply With Quote
 
Matt Garrish
Guest
Posts: n/a
 
      01-15-2006

<(E-Mail Removed)> wrote in message
news:20060115160930.889$(E-Mail Removed)...
> "Matt Garrish" <(E-Mail Removed)> wrote:
>> "Arvin Portlock" <(E-Mail Removed)> wrote in message
>> news:dq9gb7$219p$(E-Mail Removed)...
>> > In using ExpatXS to parse large batches of XML files
>> > something curious happens. After parsing something close
>> > to 500 files the program crashes with the error:
>> >
>> > Can't call method "read" on an undefined value at
>> > /xxx/perl5.8/lib/site_perl/5.8.7/sun4-solaris-thread-multi/
>> > XML/SAX/ExpatXS.pm line 155.
>> >

>>
>> You aren't checking whether you succesfully get an object. I would doubt
>> it's a memory leak or your system would have ground to a halt.

>
> But it could easily be a file-handle leak. Around 500 would be a common
> per-process limit on open file-handles.
>


That seems more likely. I just wonder what the read method is that is being
called on what undefined object. It's not a particularly helpful error
message. It could be coming right from Expat from the sound of it.

Matt


 
Reply With Quote
 
Arvin Portlock
Guest
Posts: n/a
 
      01-15-2006
Matt Garrish wrote:

> wrote in message
> news:20060115160930.889$(E-Mail Removed)...
>
> >"Matt Garrish" wrote:
> >
> >>"Arvin Portlock" wrote in message
> >>news:dq9gb7$219p$(E-Mail Removed)...
> >>
> >>>In using ExpatXS to parse large batches of XML files
> >>>something curious happens. After parsing something close
> >>>to 500 files the program crashes with the error:
> >>>
> >>>Can't call method "read" on an undefined value at
> >>>/xxx/perl5.8/lib/site_perl/5.8.7/sun4-solaris-thread-multi/
> >>>XML/SAX/ExpatXS.pm line 155.
> >>>
> >>
> >>You aren't checking whether you succesfully get an object. I would doubt
> >>it's a memory leak or your system would have ground to a halt.

> >
> >But it could easily be a file-handle leak. Around 500 would be a common
> >per-process limit on open file-handles.
> >

>
>
> That seems more likely. I just wonder what the read method is that is
> being
> called on what undefined object. It's not a particularly helpful error
> message. It could be coming right from Expat from the sound of it.
>
> Matt


The line in question in ExpatXS is this:

$result = $args->{ParseFunc}->($args->{Parser},
$args->{ParseFuncParam});

In some context that's:

sub _parse_systemid {
my $self = shift;
my $fh = IO::File->new(shift);
$self->{ParseOptions}->{ParseFunc} = \&ParseStream;
$self->{ParseOptions}->{ParseFuncParam} = $fh;
$self->_parse;
}

sub _parse {
my $self = shift;
my $args = bless $self->{ParseOptions}, ref($self);
....
my $result;
$result = $args->{ParseFunc}->($args->{Parser},
$args->{ParseFuncParam});

ParserFree($args->{Parser});
....
}

See that ParseFuncParam is a filehandle. I don't know
enough about IO::File to know exactly what's going on
but I think some more evals might be helpful.

Arvin

 
Reply With Quote
 
Arvin Portlock
Guest
Posts: n/a
 
      01-15-2006
http://www.velocityreviews.com/forums/(E-Mail Removed) wrote:

> Arvin Portlock wrote:
>
> >In using ExpatXS to parse large batches of XML files
> >something curious happens. After parsing something close
> >to 500 files the program crashes with the error:
> >
> >Can't call method "read" on an undefined value at
> >/xxx/perl5.8/lib/site_perl/5.8.7/sun4-solaris-thread-multi/
> >XML/SAX/ExpatXS.pm line 155.
> >

>
> I've used your program to parse 10_000 files. No problem.


Thank you. That was very nice.

> Maybe I am using the Pure perl method. Can you force it to use the
> pure-perl method and see what happens?


Everything works just fine with Pure perl (and it's not even
as slow as I thought it would be). Haven't had a chance to
try other parsers since installation is a bear for me.

Arvin

 
Reply With Quote
 
robic0
Guest
Posts: n/a
 
      01-16-2006
On Fri, 13 Jan 2006 16:24:06 -0800, Arvin Portlock <(E-Mail Removed)> wrote:

Use my SAX parse that I'm about to release
 
Reply With Quote
 
xhoster@gmail.com
Guest
Posts: n/a
 
      01-16-2006
Arvin Portlock <(E-Mail Removed)> wrote:
>
> # $Id: ExpatXS.pm,v 1.39 2005/11/10 09:38:31 cvspetr Exp $


I've now tried your program with this exact version of ExpatXS.pm, and
it also works for at least 10_000 files.

> This is perl, v5.8.7 built for sun4-solaris-thread-multi


This is perl, v5.8.7 built for x86_64-linux
This is perl, v5.8.3 built for x86_64-linux-thread-multi
This is perl, v5.8.0 built for i386-linux-thread-multi

(I would have replied to myself, but my first post hasn't showed up for
me yet. I hope it does eventually.)

Xho

--
-------------------- http://NewsReader.Com/ --------------------
Usenet Newsgroup Service $9.95/Month 30GB
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Undefine Method Only Marks the Method Undefined Su Zhang Ruby 3 04-18-2011 12:11 AM
undefined method `call' for nil:NilClass (NoMethodError) Iain Barnett Ruby 2 07-20-2010 02:01 PM
Help using Spreadsheet::ParseExcel Module - Can't call method "value"on an undefined value perl Newbie Perl Misc 2 05-06-2009 09:43 AM
Cant' call method "EOF" on an undefined value... Marc Baker Perl Misc 2 01-19-2006 09:58 AM
Can't call method "uri" on an undefined value chad phillips Perl Misc 0 09-15-2003 04:29 AM



Advertisments