Distributed Message Ordering

Question

I have an architectural question on handling message ordering. For purposes of this question, the transport is irrelevant, so I'm not going to specify one.

Say we have three systems, a website, a CRM and an ERP. For this example, the ERP will be the "master" system in terms of data ownership. The website and the CRM can both send a new customer message to the ERP system. The ERP system then adds a customer and publishes the customer with the newly assigned account number so that the website and CRM can add the account number to their local customer records. This is a pretty straight forward process.

Next we move on to placing orders. The account number is required in order for the CRM or website to place an order with the ERP system. However the CRM will permit the user to place an order even if the customer lacks an account number. (For this example assume we can't modify the CRM behavior) This creates the possibility that a user could create a new customer, and place an order before the account number gets updated in the CRM.

What is the best way to handle this scenario? Would it be best to send the order message sans account number and let it go to an error queue? Would it be better to have the CRM endpoint hold the message and wait until the account number is updated in the CRM? Maybe something completely different that I haven't thought of?

Thanks in advance for any help.

Answer 1

Could you not encapsulate the "create new customer & place order" operation into a single message that is simple deconstructed and played in sequence in the ERP system? So you'd have a "create new customer" message, a "place order" message and a new "create customer & place order" message.

Answer 2

I suppose CRM has its own unique customer id for the newly created customer. This CRM customer id is an external key for ERP and it should be present in the ERP->CRM update, otherwise CRM cannot correlate and update its own user record. If this assumption is right, you can put a middleman between CRM and ERP that keeps the orders without account numbers waiting in a queue, until it catches the account number update from ERP. The middleman would use the CRM customer id to correlate the waiting order request with the account number update, then enrich the order with the account id and forward the order to ERP.

If an order is stuck in the middleman's queue for too long, it should be moved to an error/escalation queue.

The middleman may be implemented within CRM or in ERP or in some integration platform.

Answer 3

The CRM system should have its own Account Numbers, which are used internally. This gives you a mechanism for processing orders before the ERP system has created the master account. The CRM app holds the ERP Account Number as a unique key on its Customer Record; basically CRM Account Number is a synthetic key and the ERP Account Number is a business key.

In this design the CRM sends a New Customer message and a New Order message to the ERP. Both messages send the CRM Customer Number and a blank ERP Account Number. Assuming the ERP system has just the one message queue the usual laws of physics apply so the ERP system should process the New Customer message first; it sends back a New Account message to the CRM system, tying the CRM Customer Number to a new ERP Account Number. The ERP system needs a data store which correlates its Customer numbers with the CRM account numbers, so that it can process any New Order messages which were sent before before it issued the ERP Account Number.

If you have a situation in which a New Order message arrives for processing before the New Customer message has been processed you will need to route it to a pending queue. When the ERP system processes a New Customer message then it would need to check the pending queue. Actually perhaps queue is the wrong term, as the ERP system is going to have riffle through all the New Order messages looking for ones which match that CRM Customer Number.