XML streaming with XProc.

Question

Hi all, I'm playing with xproc, the XML pipeline language and http://xmlcalabash.com/. I'd like to find an example for streaming large xml documents. for example, given the following huge xml document:

<Books>
 <Book>
   <title>Book-1</title>
 </Book>
 <Book>
   <title>Book-2</title>
 </Book>
 <Book>
   <title>Book-3</title>
 </Book>

<!-- many many.... -->
 <Book>
   <title>Book-N</title>
 </Book>
</Books>

How should I proceed to loop (streaming) over x->N documents like

<Books>
 <Book>
   <title>Book-x</title>
 </Book>
</Books>

and treat each document with a xslt ? is it possible with xproc ?

Answer 1

I remember a recent discussion on the XProc Dev list related to streaming. It seems that Calabash does not attempt streaming, see Norman Walsh message here.

Saxon SA, supports streaming for XSLT and XQuery, for details see: http://www.saxonica.com/documentation/sourcedocs/serial.html

Answer 2

Yes, much as I'd like to support streaming, my real goals for XML Calabash were completeness and correctness.

I have some ideas for reworking the internals of XML Calabash to use more of the push/pull streaming features of Saxon, but there are a lot of other things on my "todo" list too :-/

Answer 3

EMC's Calumet (http://developer.emc.com/xmltech) doesn't do streaming either. The main focus until now has been compliance with the XProc specification together with integrability with other our XML-related tools, such as the xDB native XML database. Support for streaming is on my radar, although I can't tell when I will be able to get to that right now.