Integrations are typically an integral part of an enterprise CRM application. In a happy day scenario the system will always be up and not encounter issues. Unfortunately we know for a fact that this will not always be the case. Ultimately there are 3 major data integration styles which I’ll describe and discuss in concept.
Some question you have to ask yourself when designing your integration mechanisms are:
- What happens if one or more of the integrated systems is not available or worst case scenario the entire network goes down?
- How easy is it to recover from such outages and can you maintain the integrity of the data?
- What do you have to do in order to recycle your CRM application and what are the implications?
Point-to-Point Integration
In a point to point integration scenario the implied assumption is that all systems will be running and available at all times. There is no buffer for when things are not working correctly. If you have processes that try once and move on, for example when synchronizing information then this is a very bad position to be in. Hopefully in this case there are logs that at least can be used to determine what failed and what needs to be synchronized during the outage, but this is at a minimum not a fun task to be in charge of especially if the data velocity is very high.
Hub & Spoke Integration
Hub and spoke integration patters can provide some isolation depending on the design. If the hub fails to talk to a particular end point then it may be able to retry having a log of the transactions the failed. This is a better position to be in from a point to point style integration and allows for greater scalability as server resources can be scaled up or out. Data integration providers continue to make enhancements to their offerings and have even started subscription type services to cloud based data processing. Hub and spoke styles are typically implemented using something like SQL Server Integration services which comes with SQL Server or other third-party products such as Scribe, Pervasive or Informatica.
Message Queuing Integration
The best place to be in is having message queues sitting in between your integration/network boundaries. For example, when a network connection goes down between systems data can be queued and retry mechanisms put in place ready to go when the network comes back up. The same applies for integrated systems. If a system goes down then queues provide a buffer for system downtime. If you have to shut down a bunch of processes when say recycling your CRM application servers else experience impacts to data integrity then you have to consider if there is a better way to manage this inevitable situation.
The caveat to message queuing integration is that is does require more work and forethought on because there are additional complexities to consider such as canonical data formats, choosing between message queue architectures/platforms and message management. Ultimately this can be the most robust and scalable architecture to have and should be investigated as part of an ongoing system strategy.
