How to hook up Boost serialization & iostreams to serialize & gzip an object to string?

Question

Hi all; I've been using the Boost serialization library, which is actually pretty nice, and lets me make simple wrappers to save my serializable objects to strings, like so:

template <class T> inline std::string saveString(const T & o) {
 std::ostringstream oss;
 boost::archive::binary_oarchive oa(oss);
 oa << o;
 return oss.str();
}
template <class T> inline void saveFile(const T & o, const char* fname) {
 std::ofstream ofs(fname, std::ios::out|std::ios::binary|std::ios::trunc);
 boost::archive::binary_oarchive oa(ofs);
 oa << o;
}
template <class T> inline void loadFile(T & o, const char* fname) {
 std::ifstream ifs(fname, std::ios::in|std::ios::binary);
 assert(ifs.good()); // XXX catch if file not found
 boost::archive::binary_iarchive ia(ifs);
 ia >> o;
}

The thing is, I just found the need to compress my serialized data, too, so I'm looking at doing that with the filters in boost::iostreams. I figured out how to do it successfully with files:

template <class T> inline void saveGZFile(const T & o, const char* fname) {
 std::ofstream ofs(fname, std::ios::out|std::ios::binary|std::ios::trunc);
 boost::iostreams::filtering_streambuf<boost::iostreams::output> out;
 out.push(boost::iostreams::gzip_compressor());
 out.push(ofs);
 boost::archive::binary_oarchive oa(out);
 oa << o;
}
template <class T> inline void loadGZFile(T & o, const char* fname) {
 std::ifstream ifs(fname, std::ios::in|std::ios::binary);
 assert(ifs.good()); // XXX catch if file not found
 boost::iostreams::filtering_streambuf<boost::iostreams::input> in;
 in.push(boost::iostreams::gzip_decompressor());
 in.push(ifs);
 boost::archive::binary_iarchive ia(in);
 ia >> o;
}

But can't figure out how to save correctly to a compressed string. The problem is that I'm not flushing the chain of filters, but I've tried popping and syncing and nothing seems to work. Here's my broken code:

template <class T> inline std::string saveGZString(const T & o) {
 std::ostringstream oss;
 boost::iostreams::filtering_streambuf<boost::iostreams::output> out;
 out.push(boost::iostreams::gzip_compressor());
 out.push(oss);
 boost::archive::binary_oarchive oa(out);
 oa << o;
 // XXX out.pop() twice?  out.strict_sync()??  oss.flush()??
 return oss.str();
}

As a result some data gets stuck in the stream buffer somewhere, and I always end up with a a few complete blocks (16K or 32K) of compressed data when I know it should be 43K or so given the (valid) output I get from using my saveGZFile method. Apparently hooking up the ofstream closes and flushes properly, but hooking up the ostringstream doesn't.

Any help? (This is my first stackoverflow question — help me, guys, you're my only hope!)

Answer 1

I haven't run the code myself, but my best guess is to use out.strict_sync() which applies flush() to every filter/device in the pipeline. I can't seem to tell, though, if gzip_compressor is flushable. If it is not, then strict_sync() will return false, and sync() would be more appropriate.