looking for a MemoryStream in C++

Question

In the wonderful world of C# i can create a memory stream without specifying its size, write into it and then just take the underlying buffer.

How can i do the same in c++? basicly i need to do:

memory_stream  ms(GROW_AS_MUCH_AS_YOU_LIKE);

ms << someLargeObjects << someSmallObjects << someObjectsWhosSizeIDontKnow;

unsigned char* buffer = ms.GetBuffer();
int bufferSize = ms.GetBufferSize();

rawNetworkSocket.Send(buffer, bufferSize);

By the way I have boost in my project though I'm not all that familiar with it.

Thank you.

Answer 1

You may want to look at std::stringstream for that purpose. The stream will grow as required. Unless you want to leave the objects in binary instead of ASCII, in which case you could take a look at the streambuf objects and implementations.

Note that C++ does not have reflection or double/multiple dispatch, so you will have to provide support for the unknown sized object yourself:

class unknown_base {
   virtual void dump( std::ostream & ) const;
};
std::ostream& operator<<( std::ostream& o, unknown_base const & obj ) {
   obj.dump( o );
   return o;
}
std::string serialize( std::vector<unknown_base*> const & data ) {
   std::ostringstream st;
   for ( std::vector<unknown_base*>::const_iterator it = data.begin(), end = data.end();
         it != end; ++it ) {
      st << **it; // double dereference: iterator, pointer
   }
   return st.str();
}

Answer 2

Since you are talking about network, it seems awfully like you want to create some kind of message and send it over the wire.

There are libraries to create messages and generate APIs for these messages, the most famous being Google Protocol Buffers (protobuf for short). It lets you describe the syntax of your message in a short file (custom format) and then automatically generate the API to decode this message in C++ / Python / Java

Advantages include:

• interoperability, which also mean the possibility of inspecting a message with a scripting language here, handy when debugging.
• version handling (backward and forward compatibility), because we know you'll soon modify the message but perhaps not upgrade everything at once
• text / binary output. Text is handy for debugging, Binary is required when every bit count

Also, it's possible to use the text output and compress it with LZO or such to gain some space :)

Answer 3

On Boost side there is Iostreams which is very similiar.

Answer 4

#include <sstream>

std::ostringstream  buffer; // no growth specification necessary
buffer << "a char buffer" << customObject << someOtherObject;

std::string contents = buffer.str();
size_t bufferSize = contents.size();

rawNetworkSocket.Send(contents); // you can take the size in Send

Using this approach you will have to parse the result where you receive it (as the code above just transforms your data into an unstructured string.

Another problem with it is that since C++ doesn't support reflection, you will have to define operator << for your objects. This is the code for a Custom class:

template<typename C, typename T>
std::basic_ostream<C,T>& operator << (
    std::basic_ostream<C,T>& out, const Custom& object)
{
    out << object.member1 << "," << object.member2 /* ... */ << object.memberN;
    return out;
}

If you want structured serialization, have a look at boost::serialization.