I have around 2 million strings with different lengths that I need to compress and put into MongoDb GridFS as files.
The strings are currently stored in MS SQL TEXT field of a table. I wrote a sample app to read each row, compress it and store it as a GridFS file.
There is one reader and a thread pool of 50 threads storing the results. It works but it is very slow (100 records per second on average).
I was wondering if there is any way for faster import into GridFS?
I'm using MongoDb 1.6 on Windows with MongoCSharp driver in C# and .NET.