Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > Limited XML tidy

Reply
Thread Tools

Limited XML tidy

 
 
Toby White
Guest
Posts: n/a
 
      08-23-2005

I have a program which produces well-formed XML documents,
but takes several hours if not days to do so. It would
be useful to be able to take the incomplete output and
manipulate it as XML.

Clearly, however, the incomplete output will not be well-
formed, so before being able to manipulate it I need to
make it wellformed.

This is essentially fairly simple. I know it's well-formed
up to now - all I need to do is close unclosed tags. So,
I 've made a short function that will do this:

#!/usr/bin/python

import sys
import xml.sax

tagStack = []

closingTags = ""

class DodgyHandler(xml.sax.ContentHandler):
def startElement(self, tag, attributes):
tagStack.append(tag)

def endElement(self, tag):
tagStack.pop()


class DodgyErrorHandler(xml.sax.ErrorHandler):
def fatalError(self,exception):
global closingTags
tagStack.reverse()
for tag in tagStack:
closingTags += "</%s>" % tag
return closingTags


def finishXML(text):
p = xml.sax.make_parser()
p.setContentHandler(DodgyHandler())
p.setErrorHandler(DodgyErrorHandler())

for line in text:
p.feed(line)
p.close()

text.append(closingTags)


However - while this works for 90% of the cases I need,
it fails in the case where my incomplete output stops in the
middle of a tag (not to mention some other more arcane
places I don't really care about).

The problem is that when the sax handler raises an exception,
I can't see how to find out why. What I want to do is for
DodgyErrorHandler to do something different depending on
where we are in the course of parsing. Is there anyway
to get that information back from xml.sax (or indeed from
any other sax handler?)

Toby

--
Dr. Toby White
Dept. of Earth Sciences, Downing Street, Cambridge CB2 3EQ. UK
Email: <(E-Mail Removed)>

 
Reply With Quote
 
 
 
 
Magnus Lie Hetland
Guest
Posts: n/a
 
      08-25-2005
In article <(E-Mail Removed)>, Toby White wrote:
>

[snip]

I do similar stuff in the new (upcoming) version of Atox
(atox.sf.net), which works with potentially ill-formed, partial XML
(in the form of PYX events) internally, and can take partial,
ill-formed XML as input.

>The problem is that when the sax handler raises an exception,
>I can't see how to find out why. What I want to do is for
>DodgyErrorHandler to do something different depending on
>where we are in the course of parsing. Is there anyway
>to get that information back from xml.sax (or indeed from
>any other sax handler?)


What I ended up doing was using an SGML parser (sgmlop) instead. It's
highly forgiving (even of illegal entities and the like) but gives me
the information I need. Might be worth a look in your app too?

>Toby


--
Magnus Lie Hetland
http://hetland.org
 
Reply With Quote
 
 
 
 
uche.ogbuji@gmail.com
Guest
Posts: n/a
 
      08-25-2005
> The problem is that when the sax handler raises an exception,
I can't see how to find out why. What I want to do is for
DodgyErrorHandler to do something different depending on
where we are in the course of parsing. Is there anyway
to get that information back from xml.sax (or indeed from
any other sax handler?)

You can get raw location information, yes. See:

http://www.xml.com/pub/a/2004/11/24/py-xml.html

But I don't think this is enough for you. You also need recovery,
which you're implementing in crude form.

I tend to agree with Magnus that using an SGML parser might be your
best bet. You might even be able to turn that SGML into XML using a
tool such as James Clark's SX:

http://www.jclark.com/sp/sx.htm

--
Uche
http://copia.ogbuji.net

 
Reply With Quote
 
Toby White
Guest
Posts: n/a
 
      08-26-2005
http://www.velocityreviews.com/forums/(E-Mail Removed) writes:

>> The problem is that when the sax handler raises an exception,

> I can't see how to find out why. What I want to do is for
> DodgyErrorHandler to do something different depending on
> where we are in the course of parsing. Is there anyway
> to get that information back from xml.sax (or indeed from
> any other sax handler?)
>
> You can get raw location information, yes. See:
>
> http://www.xml.com/pub/a/2004/11/24/py-xml.html
>
> But I don't think this is enough for you. You also need recovery,
> which you're implementing in crude form.


(If you're referring to the Locator objects), yes I'm aware
that's possible. But what I want is not my location in the
document, but for the parser to say "this is an error because
I am in the middle of a tag & the document ended", or "I
was in the middle of a text section and the document ended", or
"I was in the middle of an attribute value and the document
ended", etc, so that I can then construct a simple end to the
document, inserting quote marks, finishing the tag, and closing
all unclosed tags as appropriate.

I have just realised that I might be able to grab the message
that the exception gives me, look at the expat source code
and work out what parsing events cause which error messages.
Which is a bit round the houses, but I think ought to work.


> I tend to agree with Magnus that using an SGML parser might be your
> best bet. You might even be able to turn that SGML into XML using a
> tool such as James Clark's SX:
>
> http://www.jclark.com/sp/sx.htm


If I can't get my scheme above to work, I'll have a go. But I was
hoping to do this without requiring additional packages. And in
any case, it doesn't need to be perfectly robust. As long as it
handles 99% of cases, I'll be happy.

--
Dr. Toby White
Dept. of Earth Sciences, Downing Street, Cambridge CB2 3EQ. UK
Email: <(E-Mail Removed)>
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Tidy; how to make it XML-conform? <BR> needs to be closed Ragnar XML 9 10-27-2006 09:16 AM
limited connectivity for limited users =?Utf-8?B?aG9yc2VmbHk=?= Wireless Networking 1 03-24-2006 05:17 PM
Using ElementTree to tidy up an XML string to my liking peterbe@gmail.com Python 6 02-27-2006 03:53 PM
Limited XML tidy Toby White Python 0 08-23-2005 04:35 PM
HTML to XML Conversion - Difficulty with Tidy and TagSoup Eric Java 0 12-30-2003 01:54 AM



Advertisments