I am considering to use BDB XML, as in memory application data store. It is XML like data with definition records and data records, i have provided XPath like quiring for data access. Its running well except for a memory overflow problem, that occurs in some cases which can't be avoided (so, just ant to avoid mem-overflow), and data is still needed for further processing (so, it can't be got to output stream yet). So, i was going to cache data to disk using delimited or XMl files, also was trying BOOST serialization. And thats when i thought of using some embedded db which can give me on-disk caching (when needed), encrypted on disk caching (i still do some encrypted XML caching to disk in certain cases but they are not related to the memory overflow thing). Now, I need:
- No performance degradation and reliability
- encrypted on-disk caching of in-memory data when needed (or when dataset reaches a limit)
- on demand encrypted on-disk caching
- XPath compatibility for data access (access should not need to know whether data is in-memory of on-disk)
- Don't want to create on-disk db container (in case of BDB XML) as its not needed after execution is complete, deletions will often if and might have some toll.
- Solution needs to be platform independent
so, should i use BDB XML ? it is most prominent solution and provides C++ API (my application is in platform independent C++), it provided XPath 2.0 for access (its needed), but i want to build and XML in it node by node and mostly access node by node and want to use it without an explicit on-disk container which is needed to be cleared out often during exec and deleted afterwards, but need it to not use more memory than specified buffer and use disk when needed.