How do you assess/evaluate a messaging architecture?

Question

I'm working on a project that relies on integrating with SMS/MMS messaging aggregator companies for deploying applications to cellphones as well as performing mobile payments through SMS. Many concepts in such architectures are closely related to messaging pattersn in enterprise integration and SOA worlds. I'm currently in the process of evaluating different SMS messaging vendors and would like to know if there is some special criteria that an architect looks for in messaging architectures that should be of concern.

My approach is to use the architecture "ilities" (performance, availability, scalability, security...etc) to come up woth a scoring model for each vendor's system. However, does anyone recommend other approaches or criteria to look for when integrating with such architectures?

Thanks a bunch.

Answer 1

To score `illities' correctly, you should take into consideration what the requirements are.

You could try to score all `illities' as objectively as possible, and then weight each score according to the requirements.

P.S. Shouldn't cost (€'s) be part of the equation too. For users and providers of course!

Answer 2

Do you really intend to evaluate the architecture, ie. the internal structure of the aggregator? If I was doing that I'd be very interested in factors such as the decomposition into components, their degree of cohesiveness and their relative decoupling. These would be important for a number of reasons, not least the future felxibility of the product.

Another place where the internal structure of the product becomes interesting is in the area of scalability and availability. If claims are made for such "ilities" I'd very much want to know "How does the architecture achieve that?"

I think your approach of taking an external view, stacking up your non-functional requirements against the products "ilities", is probably the most pragmatic approach. I'd also be interested in the "ilities" of the vendor themselves - what's the installed base, do we believe in the ongoing support of the product, what leve of product support, is it local to your timemezone etc.

Answer 3

Full Disclosure: I worked for one of the largest SMS/MMS brokers so I might be a bit biased.

I think the things that are important to you will be hard to get reliable (i.e. non-sales spin) numbers out of the brokers. The things we focused on at the broker I worked for were:

1. Scalability: we chose horizontal scalability) using commodity hardware for most things

2. Throughput: how many connections to each operator does the broker maintain live/fallback

3. Failover methodology: is the broker connected to redundant SMSC/MMSCs on each operators that they are available for?

4. Connection Methodology: is the broker connected directly via SMPP/MM4|7 or do they utilize the SS7 backbone directly, i.e. they have a point code.

5. Queueing capacity: the how long can the aggregator queue messages in the event of operator outages before messages get lost?

6. Direct vs. Peer connections: How many of your potential delivery MNOs are directly connected to the broker and how many are reached via peers?

7. End-to-end transit time

8. QOS