maximum size for xml files

Question

whats your rule of thumb for maximum size for xml files

Answer 1

My rule is that if it's too slow to do what I want, then it's too big, and your data probably needs to be moved to some other format... database or such.

Traversing XML nodes or using XPath can be a dog.

Answer 2

There isn't any. There are maximum sizes for files that depend on the file system you are using, though.

Answer 3

I don't think you should have a rule of thumb for maximum size for data, be it XML or anything else. If you need to store multiple gigabytes of data, then you store that data. What makes a difference is what API you use to process that data. However, XML may not be your best bet if your data set is very large. In those cases a relational or XML database will probably work better than a single XML file.

Answer 4

I think it depends on the context, where the file comes from/is generated from, what you are going to do with it, the bandwidth of any connection it has to pass through, system RAM size etc?

what is your context?

Answer 5

This may be not the thing you want to hear, but... If you're thinking about the size of your XML files, chances are you should use a database instead of files (even if they are not flat files but structured like XML). Databases are highly optimized for efficient storage of huge masses of data. The best algorithms for retrieving data are in the code base of databases.

Answer 6

Even though not all parsers read the whole file into memory, if you really need a rule of thumb I would say not bigger than half of your available ram. Anything bigger will likely be way too slow :)