All,
I just need some pointers in a problem I have. Basically 100 C++ engine instances write to an SQL Server 2005 and it is creating some problems for us. As we don't really need to write the data in the database at that point we are thinking to stop doing it and use either messaging to send the data to a service for furher processing or something like cache. We need the data fast but not in terms of milliseconds. The biggest problem is the data is massive. We are talking about 90Gb of data in a span of 4-5 hours. Can a queuing mechanism (MSMQ,Tibco, etc) handle such a volume ? What other alternatives we have? Maybe Coherence? The plan is to send this data either on a queue and have a service (in C#.NET) process it or have a cache to store it and the service read it from there.
Any pointers are very welcome.
Many Thanks,
MK