How did stackoverflow create a xml data dump of those large tables?

Question

Inspired by the xml data dump stackoverflows makes I wondered how to make them myself. I use the same tooling as they do (SqlServer and C#) but when the tables are so big I guess you wouldn't do a 'select * from questions' and iterate over it while writing it to xml via an XMLWriter class.

Answer 1

Why not? There is probably some processing involved to cleanse the data. Still an XmlWriter can stream the output to the disk without having to hold the full dataset in memory (like an XmlSerializer approach would have to).

Answer 2

And like the XMLWriter doesn't need to hold the full dataset neither does your application or the sql server. A forward-only cursor will do.

Answer 3

Use SQL server's command line BCP tool to export to XML.

> bcp "SELECT * FROM MyTable FOR XML EXPLICIT" queryout data.xml -w -r "" -S sqlserver -T