«

I searched online, but couldn't really find a standard package for
working with Python and XML -- everybody seems to suggest different
ones.

Is there a standard xml package for Python? Preferably high-level, fast
and that can parse in-file, not in-memory since I have to deal with
potentially MBs of data.

Thanks.

Gleb Rybkin wrote:

> I searched online, but couldn't really find a standard package for
> working with Python and XML -- everybody seems to suggest different
> ones.
>
> Is there a standard xml package for Python? Preferably high-level, fast
> and that can parse in-file, not in-memory since I have to deal with
> potentially MBs of data.

cElementTree and lxml (which is API-compatible to the former). cElementTree
has an incremental parser, which allows for lager-than-memory-files to be
processed.

Diez

Diez B. Roggisch wrote:
> Gleb Rybkin wrote:
>
>> I searched online, but couldn't really find a standard package for
>> working with Python and XML -- everybody seems to suggest different
>> ones.
>>
>> Is there a standard xml package for Python? Preferably high-level, fast
>> and that can parse in-file, not in-memory since I have to deal with
>> potentially MBs of data.
>
> cElementTree and lxml (which is API-compatible to the former). cElementTree
> has an incremental parser, which allows for lager-than-memory-files to be
> processed.

In Python 2.5, cElementTree and ElementTree will be available in the
standard library as xml.etree.cElementTree and xml.etree.ElementTree.
So learning them now is a great idea.

STeVe

Okay, thanks!

Steven Bethard wrote:
> Diez B. Roggisch wrote:
> > Gleb Rybkin wrote:
> >
> >> I searched online, but couldn't really find a standard package for
> >> working with Python and XML -- everybody seems to suggest different
> >> ones.
> >>
> >> Is there a standard xml package for Python? Preferably high-level, fast
> >> and that can parse in-file, not in-memory since I have to deal with
> >> potentially MBs of data.
> >
> > cElementTree and lxml (which is API-compatible to the former). cElementTree
> > has an incremental parser, which allows for lager-than-memory-files to be
> > processed.
>
> In Python 2.5, cElementTree and ElementTree will be available in the
> standard library as xml.etree.cElementTree and xml.etree.ElementTree.
> So learning them now is a great idea.
>
> STeVe

Hi Gleb,

Gleb Rybkin wrote:
> I searched online, but couldn't really find a standard package for
> working with Python and XML -- everybody seems to suggest different
> ones.
>
> Is there a standard xml package for Python? Preferably high-level, fast
> and that can parse in-file, not in-memory since I have to deal with
> potentially MBs of data.
>
> Thanks.

Another option is Amara; also quite high-level and also allows for
incremental parsing. I would say Amara is somewhat higher level than
ElementTree since it allows you to access your XML nodes as Python
objects (with some extra attributes and some minor warts), as well as
giving you XPath expressions on the object tree.

URL:

http://uche.ogbuji.net/tech/4suite/amara/

Best version currently available is version 1.1.7

It does work together with py2exe on windows if the need ever arises
for you but you have to fiddle a bit with it (ask for details on this
list if you ever need to do that)

Cheers,

--Tim

Tim N. van der Leeuw wrote:
> Another option is Amara; also quite high-level and also allows for
> incremental parsing. I would say Amara is somewhat higher level than
> ElementTree since it allows you to access your XML nodes as Python
> objects (with some extra attributes and some minor warts), as well as
> giving you XPath expressions on the object tree.

Then you should definitely give lxml.objectify a try. It combines the ET API
with the lxml set of features (XPath, RelaxNG, XSLT, ...) and hides the actual
XML behind a Python object interface. That gives you everything at the same time.

http://codespeak.net/lxml/objectify.html

It's part of the lxml distribution:
http://codespeak.net/lxml/

Stefan

Steven Bethard <> writes:
[...]
> In Python 2.5, cElementTree and ElementTree will be available in the
> standard library as xml.etree.cElementTree and
> xml.etree.ElementTree. So learning them now is a great idea.

Only some of the original ElementTree software is going into 2.5,
apparently. So you can get more on the effbot.org site than you get
from just downloading Python 2.5. Probably future Python releases
will add more of Fredrik's XML code.


John

Gleb Rybkin schrieb:
> I searched online, but couldn't really find a standard package for
> working with Python and XML -- everybody seems to suggest different
> ones.
>
> Is there a standard xml package for Python? Preferably high-level, fast
> and that can parse in-file, not in-memory since I have to deal with
> potentially MBs of data.

It seems that everybody is proposing libraries that use in-memory
representations. There is a standard xml package for Python, it's
called "xml" (and comes with the standard library). It contains a
SAX interface, xml.sax, which can parse files incrementally.

Regards,
Martin

Martin v. Löwis wrote:
> Gleb Rybkin schrieb:
>> I searched online, but couldn't really find a standard package for
>> working with Python and XML -- everybody seems to suggest different
>> ones.
>>
>> Is there a standard xml package for Python? Preferably high-level, fast
>> and that can parse in-file, not in-memory since I have to deal with
>> potentially MBs of data.
>
> It seems that everybody is proposing libraries that use in-memory
> representations. There is a standard xml package for Python, it's
> called "xml" (and comes with the standard library). It contains a
> SAX interface, xml.sax, which can parse files incrementally.

To use ElementTree and keep your memory consumption down, consider using
the iterparse function:

http://effbot.org/zone/element-iterparse.htm

Then you can get more SAX-like memory consumption while still enjoying
the high-level interface of ElementTree.

STeVe

Martin v. Löwis wrote:
>
> It seems that everybody is proposing libraries that use in-memory
> representations. There is a standard xml package for Python, it's
> called "xml" (and comes with the standard library). It contains a
> SAX interface, xml.sax, which can parse files incrementally.

What about xml.dom.pulldom? It quite possibly resembles ElementTree's
iterparse, or at least promotes event-style handling of XML information
using some kind of mainloop...

import xml.dom.pulldom

for etype, node in xml.dom.pulldom.parseString(s):
if etype == xml.dom.pulldom.START_ELEMENT:
print node.nodeName, node.attributes

....instead of callbacks (as happens with SAX):

import xml.sax

class CH(xml.sax.ContentHandler):
def startElement(self, name, attrs):
print name, attrs

xml.sax.parseString(s, CH())

Paul