Protocol Buffers: Should I use int64 or fixed64 to represent a .NET DateTime value?

Question

I need to serialize a .NET DateTime value in a protocol buffers message.

My plan is to use the DateTime.ToBinary() and then pass the 64-bit returned value in the message. But I'm not sure what to choose as a protocol buffers data type to represent that.

I guess I'm confused about when the fixed64 (or sfixed64) data types should be used.

I'm assuming in this scenario I would use the signed types since the values returned by DateTime.ToBinary() can be negative as well as positive.

Answer 1

You should use a signed 64-bit number, not because the DateTime can be negative, but because the ToBinary method returns an Int64, which is a signed 64-bit number.

Answer 2

Well, you definitely want either int64 or sfixed64 to cope with the value being signed.

Having just done a quick test, DateTime.Now.ToBinary() is encoded in 10 bytes using int64 whereas sfixed64 will always use 8 bytes. Basically, the variable length encoding is great for small numbers, but becomes bigger than the fixed encoding for large numbers. (It's the same kind of tradeoff as using UTF-8 instead of UTF-16 - ASCII characters can be encoded in UTF-8 in a single byte, but later on code points end up being encoded as 2 and then 3 bytes, whereas UTF-16 always uses a 2 bytes.)

My guess is that DateTime.ToBinary() values are likely to be quite large (without knowing the details of exactly what it does) so sfixed64 is more appropriate.

Does that make sense?

Answer 3

In protobuf-net, I use a graduated scale approach (and indeed, it handles all this for you if you simply use DateTime) - the equivalent .proto is something like this:

message DateTime {
  optional sint64 value = 1; // the offset (in units of the selected scale)
                             // from 1970/01/01
  optional TimeSpanScale scale = 2 [default = DAYS]; // the scale of the
                                                     // timespan
  enum TimeSpanScale {
    DAYS = 0;
    HOURS = 1;
    MINUTES = 2;
    SECONDS = 3;
    MILLISECONDS = 4;

    MINMAX = 15; // dubious
  }
}

i.e. if the DateTime can be expressed in whole days, I just send the number of days since 1970, etc - plus a small marker to the scale. This means that dates can be sent a bit more efficiently, but it doesn't really cost much more for other scales.

Personally, I wouldn't use ToBinary() - I would explicitly use an offset of a known scale from a known epoch (such as the unix epoch). This makes it more portable between platforms. But if you are sending (for example) just the millisecond offset, then a fixed scale would usually be more efficient than a variant-length scale. Whether you need signed or unsigned depends on whether you need dates before your epoch ;-p