tags:

views:

606

answers:

3

Google's Protocol buffer uses the C++ standard string class std::string as variable size byte array (see here) similar to Python where the string class is also used as byte array (at least until Python 3.0).

This approach seems to be good:

  • It allows fast assignment via assign and fast direct access via data that is not allowed with vector<byte>
  • it allows an easier memory management and const references than using byte*.

But I am curious: Is that the preferred way for a byte arrays in C++? What are the drawbacks of this approach (more than a few static_casts)

+2  A: 

I "think" that using std::vector is a better approach, because it was intended to be used as an array. It is true that all implementations(I know and heard of) store string "elements" in contiguous memory, but that doesn't make it standard. i.e. the code that uses std::string like a byte array, it assumes that the elements are contiguous where they don't have to be according to the standards.

AraK
In the original C++ standard, `std::vector` didn't require contiguous storage, but the 2003 standard does require it (§23.2.4/1). C++ 0x will add a similar requirement for `std::string` (§[string.require]/5 in N3000).
Jerry Coffin
+7  A: 

Personally, I prefer std::vector, since std::string is not guaranteed to be stored contiguously and std::string::data() need not be O(1). std::vector will have data member function in C++0x.

avakar
Mike Seymour
You can force `std::string` to store contiguously by calling its `c_str()` method. Crude, but effective. This way you force contiguous memory only when you need it.
Mike DeSimone
Mike, `data()` is more suited here than `c_str()` as it doesn't bother appending the terminating 0. The latter can be even slower than the former.
avakar
@Mike: No, `c_str()` forces the string to give you a contiguous representation, but that doesn't force it to "store contiguously". The clue is that `data()` and `c_str()` don't invalidate iterators or references, so the implementation cannot "throw away" any previous non-contiguous storage it was using. It's undefined behaviour to modify the array you get back from `c_str()` or `data()`, whereas with vector you can modify the array you get from `-)
Steve Jessop
No, scratch that thing about the "clue". There's a difference here between SGI's original STL and the standard, and I was looking at the STL. `data()` and `c_str()` can invalidate references. But it still doesn't mean they're required to convert the "real" storage of the string. They both return `const char*`, so unlike vectors, strings are not required by the standard to have any way of providing contiguous, modifiable data. This is what you'd need for instance to use a C-style reading API.
Steve Jessop
+1  A: 

std::strings may have a reference counted implementation which may or may not be a advantage/disadvantage to what you're writing -- always be careful about that. std::string may not be thread safe. The potential advantage of std::string is easy concatenation, however, this can also be easily achieved using STL.

Also, all those problems in relation to protocols dissapear when using boost::asio and it's buffer objects.

As for drawbacks of std::vector:

  1. fast assign can be done by a trick with std::swap
  2. data can be accessed via &arr[0] -- vectors are guaranteed (?) to be continious (at least all implementations implement them so)

Personally I use std::vector for variable sized arrays, and boost::array for static sized ones.

Kornel Kisielewicz
avakar
std::vector is (effectively) guaranteed not to be ref counted - however, that doesn't make it any more thread-safe than std::string.
anon
@Niel, we've went through hell because of GCC's thread unsafety of strings while working on a server. Switching to vectors did help.
Kornel Kisielewicz
@Niel, @avakar -- both of you are right, yet no standard will save you once a server starts leaking, crashing and Valgrind going crazy with errors. I usually prefer to look at the implementation, then trust a standard, that not everyone (*cough* MS *cough*) follows ;>
Kornel Kisielewicz
I have no idea why `std::vector` would not be threadsafe unless you're talking about multiple threads hitting the same vector. `std::string`, because of copy-on-modify implementations, can be thread-unsafe even if you do all the right things regarding thread safety, such as enforcing that each string has one owner.
Mike DeSimone
Ok, why is std::string not threadsafe as long as I don't access/modify it from multiple threads?
dmeister
Access from multiple threads is what thread safety is all about - it's easy to be thread safe if you only have one thread!
anon
@Neil, it may be possible that the string implementation modifies a shared state without proper locking. Ok, the question should be: Is std::string threadsafe as long as I don't access/modify the same instance from multiple threads, but I access other instances of std::string concurrently.?
dmeister
avakar
The problem with a RC string is that even if you carefully pass a copy of a string to another thread, so each thread has its own string object to work with, the reference counted one can still really have two references to a single shared object. With std::vector, a pass by value forces a copy of the data, preventing sharing of the data across threads.
Jerry Coffin