Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > Sequential XML parsing with xml.sax

Reply
Thread Tools

Sequential XML parsing with xml.sax

 
 
peter@hardy.dropbear.id.au
Guest
Posts: n/a
 
      08-23-2005
Hi hi.

I'm trying to do sequential decompression of a bzipped XML file and
feed it to a SAX parser with the following code.

remotefh = urllib.urlopen('file:///home/peter/catalog.rdf.bz2')
decompressor = bz2.BZ2Decompressor()
handler = CatalogueDocumentHandler(sys.stdout)
chunksize = 2048
data = remotefh.read(chunksize)
while data != '':
out = decompressor.decompress(data)
if out != '':
xml.sax.parseString(out, handler)
data = remotefh.read(chunksize)

This fails with the first chunk of decompressed data passed to
xml.sax.parseString. I'm suspecting because it's an incomplete fragment
of XML. I've tried with a number of different chunk sizes, putting the
break in different places, but it always fails on the first call. For
reference, the traceback looks like:

xml.sax.parseString(out, handler)
File "/usr/lib/python2.4/site-packages/_xmlplus/sax/__init__.py",
line 47, in parseString
parser.parse(inpsrc)
File "/usr/lib/python2.4/site-packages/_xmlplus/sax/expatreader.py",
line 109, in parse
xmlreader.IncrementalParser.parse(self, source)
File "/usr/lib/python2.4/site-packages/_xmlplus/sax/xmlreader.py",
line 125, in parse
self.close()
File "/usr/lib/python2.4/site-packages/_xmlplus/sax/expatreader.py",
line 226, in close
self.feed("", isFinal = 1)
File "/usr/lib/python2.4/site-packages/_xmlplus/sax/expatreader.py",
line 220, in feed
self._err_handler.fatalError(exc)
File "/usr/lib/python2.4/site-packages/_xmlplus/sax/handler.py", line
38, in fatalError
raise exception
xml.sax._exceptions.SAXParseException: <unknown>:15132:63: no element
found

(line 15132 is the last, incomplete line feed to parseString. FWIW,
it's:
<pgterms:friendlytitle rdfarseType="Literal">Searchlights o
)

The API reference isn't clear on whether parseString can only handle
discrete bits of valid XML, or if it's designed to be called in this
way. So I'm not sure if I'm misusing the function, or if I've done
something else wrong.

Any pointers?
Thanks,
--
Pete

 
Reply With Quote
 
 
 
 
Fredrik Lundh
Guest
Posts: n/a
 
      08-23-2005
http://www.velocityreviews.com/forums/(E-Mail Removed) wrote:

> The API reference isn't clear on whether parseString can only handle
> discrete bits of valid XML


the documentation says that "parse" expects an XML document,
and that "parseString" is the same thing, but parses from a buffer.

it's probably easier to pass a BZ2File instance to "parse", but if you
insist on doing incremental SAX parsing, the IncrementalParser class
might be what you need:

http://www.python.org/doc/current/li...xmlreader.html
http://www.python.org/doc/current/li...r-objects.html

</F>



 
Reply With Quote
 
 
 
 
peter@hardy.dropbear.id.au
Guest
Posts: n/a
 
      08-24-2005
Hi.

Fredrik Lundh wrote:
> (E-Mail Removed) wrote:
>
> > The API reference isn't clear on whether parseString can only handle
> > discrete bits of valid XML

>
> the documentation says that "parse" expects an XML document,
> and that "parseString" is the same thing, but parses from a buffer.


OK, so it sounded a lot more ambiguous at 4am.

> it's probably easier to pass a BZ2File instance to "parse",


It is easier to retrieve a remote file, and decompress and parse as
separate steps. But I've been wondering if it would be faster / more
efficient to do it without caching.

> but if you
> insist on doing incremental SAX parsing, the IncrementalParser class
> might be what you need:


That'll do the trick nicely. Thanks.

Cheers,
--
Pete

 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
What libraries should I use for MIME parsing, XML parsing, and MySQL ? John Levine Ruby 0 02-02-2012 11:15 PM
Different results parsing a XML file with XML::Simple (XML::Sax vs. XML::Parser) Erik Wasser Perl Misc 5 03-05-2006 10:09 PM
Signals and variables, concurrent and sequential assignments Taras_96 VHDL 5 04-14-2005 03:07 AM
Print XML parsing to JspWriter (out) Class org.xml.sax.helpers.NewInstance can not access a member of class javax.xml.parsers.SAXParser with modifiers "protected" Per Magnus L?vold Java 0 11-15-2004 02:27 PM
Sequential Machines john VHDL 6 11-05-2004 05:24 AM



Advertisments