Dataset size limit as an xml file.

Question

Hi We are currently using DataSet for loading and saving our data to an xml file using Dataset and there is a good possibility that the size of the xml file could get very huge. Either way we are wondering if there is any limit on the size for an xml file so the Dataset would not run into any issues in the future due to the size of it. Please advise. Thanks N

Answer 1

well, OS max file size in one thing to consider. (although modern os won't have this problem). old OS support only 2 GB per file if I recall right. also - the time that you will need to waist on updating the file is enourmouse. if you're going for a very very large file, use a small DB (mysql or sqlexpress or sqLite)

Answer 2

DataSets are stored in memory so the limit should be somewhere around the amount of memory the OS can address for custom processes.

Answer 3

While doing work for a prior client of reading and parsing large XML files over 2 gig per XML file, the system choked while trying to use an XML reader. While working w/Microsoft, we ultimately were passed on to the person who wrote the XML engine. His recommendation was for us to read and process it in smaller chunks, that it couldn't handle loading the entire thing into memory at one time. However, if you are trying to WRITE XML as a stream to a final output .XML file, you should be good to go on most current OS that support over 2 gig.