How to Fix Legacy Messaging Models with Modern Streaming
Bill McLaneCTO Cloud, DataStax
A number of years ago I had a friend who was restoring his 100-year old farm house, and uttered the phrase “if it ain’t broke, don’t fix it”, referring to whether he should upgrade the ancient electrical system. Unfortunately for my friend this was not an ideal time to use this phrase, as he was soon dealing with a significant electrical fire about a month into his renovation.
My name is Bill McLane, CTO at DataStax. I think of this story as I hear CTOs deprioritize modernization, only to witness major disasters within their organization, and entire business operations to utterly fail. To help you minimize the risk of wrongly stating “if it ain’t broke…” I’ve written the following article to help you strengthen your approach to modernizing your application communication infrastructure to support legacy and modern data distribution patterns.
The brittleness of our legacy based approach
Like an ancient electrical system, many enterprises have out-dated solutions for integration and messaging. Now we aren’t talking about iMessage or Whatsapp here, we are talking about the enterprise communication layer that powers things like integration services, application-to-application communication, and event driven architectures.
Messaging has been around for over 50 years, in some form or another, and it has evolved over time. There are many common frameworks for application communication, from IBM MQ Series that started on the mainframe, to TIBCO Rendezvous that revolutionized wall street, to standardization with things like Java Message Service (JMS) with solutions like TIBCO Enterprise Message Service or IBM MQ JMS and AMQ/AMQP with solutions like RabbitMQ, to more recent solutions like MQTT for IoT and Kafka for streaming.
Like my friend's farm house, many enterprises have continued to build and add on to this communications infrastructure with the mentality of “if it ain’t broke, don’t fix it”, so as new applications come along they adopt new technologies, wiring them together with legacy solutions. Unsurprisingly, with every addition they are one step closer to disaster.
Eventually this patchwork-wiring of different infrastructures grows into a highly complex, brittle mess of servers, virtual machines, protocols, and operational models that with one false move can crumble, taking the entire hierarchy of mission critical applications to the ground.
The Legacy Messaging Approach
To grow we need to innovate, and innovation requires a focused strategy on where to invest. Do we invest in the things that make one room in our house look pretty by slapping some paint on the walls or do we transform our house from a century old farmhouse into a beautiful modern home that is built for today's technologies and is capable of providing all the innovation of today without sacrificing the beauty of what was built so many years ago?
Should I upgrade or replace my enterprise messaging solution?
At DataStax we recognize the value of upgrading to a cloud-based system as opposed to scrapping an entire architecture. We empower our users who share this strategy with a platform that delivers the benefits of modern architectures for streaming without sacrificing those benefits that messaging solutions like JMS and other standards provide.
There is no one solution that meets all the different patterns an enterprise communication framework requires. Apache Kafka, for example, is great for streaming but struggles with some of the standard messaging patterns like queuing and long term data retention. JMS and AMQP tried to standardize the interface for messaging so have been great for building things like Enterprise Service Buses (ESB) but struggle with message replay and real-time streaming for things like event driven architectures. We need to approach the problem differently where the messaging/stream infrastructure can be decoupled from the applications interfaces used to distribute the data. We need an architecture that has been built to scale and perform to modern data patterns introduced by cloud applications. We need a solution that is:
- Cloud native by design
- Can replace legacy infrastructure
- Messaging, streaming and eventing all in one
- Multi-tenant with built in data security and segmentation
- Globally distributed
- Highly scalable
This is why DataStax Streaming is powered by Apache Pulsar, it provides the common infrastructure that can supply all the features and functionality needed for modern streaming and event driven architectures. It also provides the common features and functions applications built to leverage traditional messaging models like JMS and AMQP require.
With Apache Pulsar, DataStax streaming has the infrastructure for a truly universal approach to enterprise data distribution. But while there are many solutions available, the reality is that a single approach to application communication rarely works. If we are truly going to invest in upgrading and modernizing our data communications infrastructure it needs to seamlessly interact with a number of protocols and patterns that have already been proven and established.
The DataStax streaming approach
DataStax streaming natively incorporates DataStax Starlight, an open source approach to enabling solutions built to use JMS, AMQP and even Kafka to leverage Apache Pulsar as the broker infrastructure, without having to change the application code or development model.
So if you have a brittle patch work of protocols, and you need your infrastructure to communicate across JMS, Kafka, and AMQP applications all in a single solution (powered by open source technologies) DataStax streaming is the right solution to ensure that “it’s never broke” and you won’t need to worry about someone asking “do we need to fix it”.
Key takeaways
- Decouple messaging distribution: A single data distribution architecture never scales to every enterprise demand and it can introduce significant risk to your organization, approach messaging and streaming with an architecture that decouples the distribution of messages and streams from the interfaces used to access the data.
- Use an upgrade-first mindset: Messaging modernization can be challenging, to reduce risk, you may want to upgrade as opposed to replacing your entire system.. Don’t sacrifice your investments in JMS or AMQP, build on them with a common infrastructure.
- Keep these attributes in mind: To modernize your enterprise communication system your ideal solution should:
- Meet your future messaging, streaming, and eventing needs
- Work with your existing solution for JMS, AMQP, Kafka, and Pulsar
- Support your organization as a managed service
- Deploy into self-managed infrastructure on-premise, in public, or a private cloud
Be sure to check out DataStax Streaming as we’re confident that it will meet all of your messaging requirements. Or complete this form and to speak to our expert.