What is the best way to export a large table with many records from SQLite to a custom delimited text file?

Question

The table I'm dealing with it potentially larger than available memory (let's say 10GB) and some of the fields can have at most 100MB of text. So a simple SELECT query probably isn't going to cut it. I've seen some command line solutions but I need to be able to do this through a C# program.

Answer 1

A select should be fine. Last time I checked, the entire recordset (and all its data) isn't loaded into memory when you query a DB.

If that is somehow not the case, and it is taking up all the RAM in the known universe, do your query to just get IDs and then tick through the IDs getting induvidual records. Much, much slower, but it should limit the RAM usage.