Stream from std::string without making a copy?

Question

I have a network client with a request method that takes a std::streambuf*. This method is implemented by boost::iostreams::copy-ing it to a custom std::streambuf-derived class that knows how to write the data to a network API, which works great. This means I can stream a file into the request without any need to read it all into memory.

There are some cases, however, where large blocks of data must be sent which are not in a file, so I included an overload that takes a string. To avoid duplicating all the network code in the stream, it seemed obvious that I should set up a streambuf representing the string and call the other method. The only way I could figure out to make this work was something like:

std::istringstream ss(data);
send(ss.rdbuf());

Unfortunately, istringstream makes a copy of the data, which in some cases is several megabytes. It makes perfect sense in the general case, of course, if you hand a const reference to some object you don't want that object assuming it can continue using that reference.

I worked around this with the following:

struct zerocopy_istringbuf
    : public std::stringbuf
{
    zerocopy_istringbuf(std::string const* s)
        : std::stringbuf(std::ios::in)
    {
        char* p = const_cast<char*>(s->c_str());
        setg(p, p, p + s->length());
    }
};

...

send(&zerocopy_istringbuf(data));

This seems to work just fine, but I wonder if it's really necessary. Why doesn't std::istringstream have an overload taking a std::string const *? Is there a better way to do this?

Answer 1

The reason you're having these problem is that std::string isn't really suited to what you're doing. A better idea is to use vector of char when passing around raw data. If its possible, I would just change everything to use vector, using vector::swap and references to vectors as appropriatte to eliminate all your copying. If you like the iostreams/streambuf api, or if you have to deal with something that takes a streambuf, it would be trivial to create your own streambuf that uses a vector, like yours. It would effectively do the same thing that you do with the same issues as listed in the other answers, but you wouldn't be violating the class's contract.

Otherwise, I think what you've got is probably the best way forward short of passing around an istringstream everywhere.

Answer 2

imho, best choice is a deprecated class std::strstream