Protocol buffer polymorphism

Question

I have a C++ program that sends out various events, e.g. StatusEvent and DetectionEvent with different proto message definitions to a message service (currently Active MQ, via activemq-cpp APU). I want to write a message listener that receives these messages, parses them and writes them to cout, for debugging purposes. The listener has status_event_pb.h and detection_event_pb.h linked.

My question is: How can I parse the received event without knowing its type? I want to do something like (in pseudo code)

receive event
type = parseEventType(event);
if( type == events::StatusEventType) { 
    events::StatusEvent se = parseEvent(event);
    // do stuff with se
}
else {
    // handle the case when the event is a DetectionEvent
}

I looked at this question but I'm not sure if extensions are the right way to go here. A short code snippet pointing the way will be much appreciated. Examples on protobuf are so rare!

Thanks!

Answer 1

It seems extensions are indeed the way to go but I've got one last point to clear up. Here's the proto definition that I have so far:

// A general event, can be thought as base Event class for other event types.
message Event {
    required int64 task_id = 1;     
    required string module_name = 2;    // module that sent the event

    extensions 100 to 199;               // for different event types
}

// Extend the base Event with additional types of events.
extend Event {
    optional StatusEvent statusEvent = 100;
    optional DetectionEvent detectionEvent = 101;
}

// Contains one bounding box detected in a video frame, 
// representing a region of interest.
message DetectionEvent {
    optional int64 frame = 2;   
    optional int64 time = 4;
    optional string label = 6;
}

// Indicate status change of current module to other modules in same service.
// In addition, parameter information that is to be used to other modules can
// be passed, e.g. the video frame dimensions.
message StatusEvent {
    enum EventType { 
        MODULE_START = 1; 
        MODULE_END = 2; 
        MODULE_FATAL = 3; 
    }
    required EventType type = 1;        
    required string module_name = 2;    // module that sent the event

    // Optional key-value pairs for data to be passed on.
    message Data {
        required string key = 1;
        required string value = 2;
    }
    repeated Data data = 3; 
}

My problem now is (1) how to know which specific event that the Event message contains and (2) make sure that it contains only one such event (according to the definition, it can contain both a StatusEvent and a DetectionEvent).

Answer 2

I would not use Protocol Buffers for that, but that's perhaps a combination of little use and other habits.

Anyway, I think I would use an abstract class here, to ease general handling and to contain routing information. Class that would not be defined using protobuf, and would contain a protobuf message.

class Message
{
public:
  Type const& GetType() const;

  Origin const& GetOrigin() const;
  Destination const& GetDestination() const;

  // ... other informations

  template <class T>
  void GetContent(T& proto) const
  {
    proto.ParseFromIstream(&mContent); // perhaps a try/catch ?
  }

private:
  // ...

  std::stringstream mContent;
};

With this structure, you have both general and specific handling at the tip of your fingers:

void receive(Message const& message)
{
  LOG("receive - " << message.GetType() << " from " << message.GetOrigin()
                   << " to " << message.GetDestination());

  if (message.GetType() == "StatusEvent")
  {
    StatusEvent statusEvent;
    message.Decode(statusEvent);
    // do something
  }
  else if (message.GetType() == "DetectionEvent")
  {
    DetectionEvent detectionEvent;
    message.Decode(detectionEvent);
    // do something
  }
  else
  {
    LOG("receive - Unhandled type");
  }
}

Of course, it would be prettier if you used a std::unordered_map<Type,Handler> instead of a hardcoded if / else if + / else chain, but the principle remains identical:

1. Encode the type of message sent in the header
2. Decode only the header upon reception and dispatch based on this type
3. Decode the protobuf message in a part of the code where the type is known statically