views:

166

answers:

2

I am writing a UTF-8 library for C++ as an exercise as this is my first real-world C++ code. So far, I've implemented concatenation, character indexing, parsing and encoding UTF-8 in a class called "ustring". It looks like it's working, but two seemingly equivalent ways of declaring a new ustring behave differently. The first way:

ustring a;
a = "test";

works, and the overloaded "=" operator parses the string into the class (which stores the Unicode strings as an dynamically allocated int pointer). However, the following does not work:

ustring a = "test";

because I get the following error:

test.cpp:4: error: conversion from ‘const char [5]’ to non-scalar type ‘ustring’ requested

Is there a way to workaround this error? It probably is a problem with my code, though. The following is what I've written so far for the library:

#include <cstdlib>
#include <cstring>
class ustring {
  int * values;
  long len;
  public:
  long length() {
    return len;
  }
  ustring * operator=(ustring input) {
    len = input.len;
    values = (int *) malloc(sizeof(int) * len);
    for (long i = 0; i < len; i++)
      values[i] = input.values[i];
    return this;
  }
  ustring * operator=(char input[]) {
    len = sizeof(input);
    values = (int *) malloc(0);
    long s = 0;                                                                 // s = number of parsed chars
    int a, b, c, d, contNeed = 0, cont = 0;
    for (long i = 0; i < sizeof(input); i++)
      if (input[i] < 0x80) {                                                    // ASCII, direct copy (00-7f)
        values = (int *) realloc(values, sizeof(int) * ++s);
        values[s - 1] = input[i];
      } else if (input[i] < 0xc0) {                                             // this is a continuation (80-bf)
        if (cont == contNeed) {                                                 // no need for continuation, use U+fffd
          values = (int *) realloc(values, sizeof(int) * ++s);
          values[s - 1] = 0xfffd;
        }
        cont = cont + 1;
        values[s - 1] = values[s - 1] | ((input[i] & 0x3f) << ((contNeed - cont) * 6));
        if (cont == contNeed) cont = contNeed = 0;
      } else if (input[i] < 0xc2) {                                             // invalid byte, use U+fffd (c0-c1)
        values = (int *) realloc(values, sizeof(int) * ++s);
        values[s - 1] = 0xfffd;
      } else if (input[i] < 0xe0) {                                             // start of 2-byte sequence (c2-df)
        contNeed = 1;
        values = (int *) realloc(values, sizeof(int) * ++s);
        values[s - 1] = (input[i] & 0x1f) << 6;
      } else if (input[i] < 0xf0) {                                             // start of 3-byte sequence (e0-ef)
        contNeed = 2;
        values = (int *) realloc(values, sizeof(int) * ++s);
        values[s - 1] = (input[i] & 0x0f) << 12;
      } else if (input[i] < 0xf5) {                                             // start of 4-byte sequence (f0-f4)
        contNeed = 3;
        values = (int *) realloc(values, sizeof(int) * ++s);
        values[s - 1] = (input[i] & 0x07) << 18;
      } else {                                                                  // restricted or invalid (f5-ff)
        values = (int *) realloc(values, sizeof(int) * ++s);
        values[s - 1] = 0xfffd;
      }
    return this;
  }
  ustring operator+(ustring input) {
    ustring result;
    result.len = len + input.len;
    result.values = (int *) malloc(sizeof(int) * result.len);
    for (long i = 0; i < len; i++)
      result.values[i] = values[i];
    for (long i = 0; i < input.len; i++)
      result.values[i + len] = input.values[i];
    return result;
  }
  ustring operator[](long index) {
    ustring result;
    result.len = 1;
    result.values = (int *) malloc(sizeof(int));
    result.values[0] = values[index];
    return result;
  }
  char * encode() {
    char * r = (char *) malloc(0);
    long s = 0;
    for (long i = 0; i < len; i++) {
      if (values[i] < 0x80)
        r = (char *) realloc(r, s + 1),
        r[s + 0] = char(values[i]),
        s += 1;
      else if (values[i] < 0x800)
        r = (char *) realloc(r, s + 2),
        r[s + 0] = char(values[i] >> 6 | 0x60),
        r[s + 1] = char(values[i] & 0x3f | 0x80),
        s += 2;
      else if (values[i] < 0x10000)
        r = (char *) realloc(r, s + 3),
        r[s + 0] = char(values[i] >> 12 | 0xe0),
        r[s + 1] = char(values[i] >> 6 & 0x3f | 0x80),
        r[s + 2] = char(values[i] & 0x3f | 0x80),
        s += 3;
      else
        r = (char *) realloc(r, s + 4),
        r[s + 0] = char(values[i] >> 18 | 0xf0),
        r[s + 1] = char(values[i] >> 12 & 0x3f | 0x80),
        r[s + 2] = char(values[i] >> 6 & 0x3f | 0x80),
        r[s + 3] = char(values[i] & 0x3f | 0x80),
        s += 4;
    }
    return r;
  }
};
+15  A: 

Your problem is that ustring a = "test" actually invokes the constructor, not the assignment operator. yay, welcome to c++ :)

You'll need to define yourself both a default constructor and one that takes a const char*, because once you define a constructor, you need to define all your constructors.

A few other things:

  • pass your input ustring by reference
  • pass const char * instead of char[] (you don't modify the input and char* is more common)
  • sizeof isn't doing what you think it's doing, it doesn't work properly for array parameters. It is returning you sizeof(char*), not sizeof(array).
  • return reference to this from your operators.
  • you can use vector<int> values; to manage all your memory for you.
  • encode() should probably return a string. With string:
    • it manages its own memory, so the caller doesn't need to free or delete it.
    • you can use s.append(c); instead of using realloc.
    • you can use printf("%s", s.c_str());, but in c++ you usually use cout << s;
  • consider defining a copy constructor as well.

Like this:

class ustring {
 public:
  // Default constructor, allows you to create your class with no arguments.
  ustring() { ...; }
  // Allows you to create your class from string literals.
  ustring(const char *input) { ...; }
  // Copy constructor, allows you to create your class from other instances.
  ustring(const ustring &input) { ...; }

  // Assignment operators.
  ustring &operator=(const ustring &input) { ...; return *this; }
  ustring &operator=(const char *input) { ...; return *this; }
};

int main() {
  ustring s, t;  // invokes default constructor.
  s = t;         // invokes ustring assignment op.
  s = "test";    // invokes const char* assignment op.
  ustring u = "test";  // invokes const char* constructor.
  ustring v("test");   // invokes const char* constructor.
  ustring x(u);  // invokes copy constructor.
}

If this is c++, why are you doing all this malloc/realloc stuff? I haven't fully parsed that code, but I'd imagine there's a simpler way... see the comment about using vector.

As @Michael Aaron Safyan mentioned in the comments, if you do any memory allocation for the ustring class, you will want to deallocate it in the destructor. However, I think by switching to memory managed containers - vector & string - you'll avoid any of your own memory management and can avoid writing a destructor.

Stephen
@Stephen, don't forget the destructor! Especially since the OP is doing memory allocation and needs to deallocate it in his destructor. You might also want to suggest swap as an easy, clean way to perform assignment.
Michael Aaron Safyan
Thank you very much for your answer; it was very helpful and I'm now giving your advice a go. However, when you say that "encode() should probably return a string", I think that returning a char* is better because you can then use the UTF-8 string as follows: printf("%s\n", mystring.encode());
Delan Azabani
Also, another dilemma... if I 'pass const char * instead of char[]', then I won't be able to sizeof() to get the string length. If so, should I just stop at null?
Delan Azabani
@Michael Aaron Safyan, @Delan Azabani : edited my answer in response to your comments. Thanks!
Stephen
@Delan: If `encode` returned a `std::string`, you could do `printf("%s\n", mystring.encode().c_str())`. Hardly a big problem. Plus it saves the user from having to keep track of the ownership semantics of the encoded string. Don't use `char*`. You're making things harder for yourself and the user of your class.
jalf
@Delan, since UTF-8 does not allow the embedding of NULs, there's no reason why you couldn't store a UTF-8 string in an std::string. In fact, the encoding of std::string is unspecified... it could, on some systems, be interpreted as storing a UTF-8 string.
Michael Aaron Safyan
+1  A: 

These are two operations:

ustring a; // construct a new object using constructor
a = "test"; // assign value to object using operator=

This is one operation:

ustring a = "test"; // construct with a value, aka value-intialization

In the interest of runtime efficiency and allowing semantic freedom, C++ does not extrapolate the default constructor ustring::ustring() and the assignment operator ustring::operator=(const char*) into a constructor ustring::ustring(const char *).

But, for most reasonable string classes, this will work:

ustring::ustring(const char *str)
 : /* initialize ustring::ustring() does */ {
    /* do whatever ustring::ustring() does */
    *this = str; // assign value.
}

It is better to call the assignment operator from the constructor than to attempt it the other way around.

Of course, you can probably improve efficiency by considering the length of the given string while performing initialization.

Potatoswatter