Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > XML > XQuery on big files

Reply
Thread Tools

XQuery on big files

 
 
Pascal Sartoretti
Guest
Posts: n/a
 
      04-10-2007
Hello,

I want to process big files (e.g. 100 MB) with XQuery. Are there XQuery
processors that work in a "SAX-like" way (and not DOM-like), i.e. which
don't assume that they have everything in memory ?

Thanks for any help

Pascal
 
Reply With Quote
 
 
 
 
Philippe Poulard
Guest
Posts: n/a
 
      04-10-2007
Pascal Sartoretti wrote:
> Hello,
>
> I want to process big files (e.g. 100 MB) with XQuery. Are there XQuery
> processors that work in a "SAX-like" way (and not DOM-like), i.e. which
> don't assume that they have everything in memory ?
>
> Thanks for any help
>
> Pascal


hi,

with RefleX, you can process big files with XPath ; unlike many
streaming-XPath processors, RefleX doesn't restrict the XPath syntax,
but you must be aware that using XPath queries such as //* on the root
would cause caching the complete document causing an OutOfMemoryError ;
however, if you intend to use backwards axis, you'd rather "cast" small
chunks of your input in DOM subtrees, and use the full capabilities of XPath

an example here :
http://reflex.gforge.inria.fr/tutorial.html#N801C30
(the input file is not as big as yours, but big enough to make a simple
DOM parsing fail ; it was the case at the time I wrote this example)

in this example, you'll see how an inline-filter can split a big file in
about 13000 small files with 2 flavours : SAX to SAX and SAX to DOM

enjoy !

--
Cordialement,

///
(. .)
--------ooO--(_)--Ooo--------
| Philippe Poulard |
-----------------------------
http://reflex.gforge.inria.fr/
Have the RefleX !
 
Reply With Quote
 
 
 
 
Martin Honnen
Guest
Posts: n/a
 
      04-10-2007
Pascal Sartoretti wrote:

> I want to process big files (e.g. 100 MB) with XQuery. Are there XQuery
> processors that work in a "SAX-like" way (and not DOM-like), i.e. which
> don't assume that they have everything in memory ?


SAXON-SA for XSLT (not for XQuery it seems) has a streaming processing
mode, see
<http://www.saxonica.com/documentation/sourcedocs/serial.html>


--

Martin Honnen
http://JavaScript.FAQTs.com/
 
Reply With Quote
 
Tony Lavinio
Guest
Posts: n/a
 
      04-10-2007
DataDirect XQuery is optimized for very large files. It supports both
SAX and StAX modes. We've tested it with multi-gigabyte files.

See http://www.xquery.com

There is also an embedded version inside of Stylus Studio that you
can try out. See http://www.stylusstudio.com



Pascal Sartoretti wrote:
> Hello,
>
> I want to process big files (e.g. 100 MB) with XQuery. Are there XQuery
> processors that work in a "SAX-like" way (and not DOM-like), i.e. which
> don't assume that they have everything in memory ?
>
> Thanks for any help
>
> Pascal


--
Tony Lavinio <> DataDirect <> Stylus Studio XML <> http://www.velocityreviews.com/forums/(E-Mail Removed)
XQuery, XSLT, XML Schema and EDI Toolset <> http://www.stylusstudio.com/
<> There is no problem that brute force and ignorance cannot overcome <>
 
Reply With Quote
 
Philippe Poulard
Guest
Posts: n/a
 
      04-10-2007
....and it's free of charge

--
Cordialement,

///
(. .)
--------ooO--(_)--Ooo--------
| Philippe Poulard |
-----------------------------
http://reflex.gforge.inria.fr/
Have the RefleX !
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
GIDS 2009 .Net:: Save Big, Win Big, Learn Big: Act Before Dec 29 2008 Shaguf ASP .Net 0 12-26-2008 09:29 AM
GIDS 2009 .Net:: Save Big, Win Big, Learn Big: Act Before Dec 29 2008 Shaguf ASP .Net Web Controls 0 12-26-2008 06:11 AM
GIDS 2009 Java:: Save Big, Win Big, Learn Big: Act Before Dec 29 2008 Shaguf Python 0 12-24-2008 07:35 AM
GIDS 2009 Java:: Save Big, Win Big, Learn Big: Act Before Dec 29 2008 Shaguf Ruby 0 12-24-2008 05:07 AM
XQUERY frontend for consulting XML files Alfaeco XML 0 10-25-2004 10:00 AM



Advertisments