Infinite Scale without Fail with Starlight for RabbitMQ

Yesterday's data architecture can't meet today's business requirements for speed, flexibility, and growth. To harness the power of their data, enterprises are turning to modern streaming platforms, like the cloud-native, horizontally scalable Apache Pulsar. Yet many still rely on traditional message brokers like RabbitMQ, a once-popular open-source message queueing platform that is struggling to keep up with the scale and performance requirements of today’s technology solutions. RabbitMQ suffers from many of the same shortcomings as other legacy messaging platforms - a monolithic architecture that presents serious obstacles to scale and inherent brittleness. As a result, many organizations are seeking an alternative that allows them to continue to leverage their existing investment in RabbitMQ, but without the foundational shortcomings in that platform.

That’s why DataStax is pleased to announce Starlight for RabbitMQ, an open-source, fast-messaging compatibility layer for Pulsar. This solution provides wire-level compatibility with the RabbitMQ API and enables enterprises to take advantage of Pulsar’s incredible performance and horizontal scalability without requiring them to rewrite their RabbitMQ applications.

Starlight for RabbitMQ streamlines enterprise operations

By combining the industry-standard AMQP 0.9.1 (RabbitMQ) API with Pulsar, this new API provides a powerful way to modernize your RabbitMQ infrastructure while improving performance and reducing costs. 

Here are a few key benefits of Starlight for RabbitMQ: 

• Blazing fast RabbitMQ performance Starlight for RabbitMQ scales horizontally up or down without any operational hassles. 

• Drop-in replacement for existing RabbitMQ applications Starlight for RabbitMQ provides a drop-in replacement to enable existing RabbitMQ applications to run on Pulsar. It supports commonly used features of RabbitMQ via compatibility with its network protocol, AMQP 0.9.1.

• Message retention and replay Pulsar supports offloading old messages to object storage for practically infinite storage capacity. Applications using Starlight for RabbitMQ can revert to or play previously consumed messages for auditing, testing, and failure recovery purposes. Older messages that are read less frequently can be moved to lower-cost storage like HDFS or AWS S3.

• No message loss Pulsar durably persists messages to disk before accepting them. Once a message is accepted it is guaranteed to be delivered. 

• Geo-replication for low latency and disaster recovery Pulsar is designed for asynchronous replication across multiple, geographically separate regions. This enables lower latency because clients are writing to and reading from brokers in their local region, and traffic can be rerouted to surviving regions in the event of an outage.

• Open source to avoid lock-in Starlight for RabbitMQ is 100% open source. You can run it on-premises or in any cloud provider, in Kubernetes.

• Future-proof with a reduced total cost of ownership: Pulsar supports traditional RabbitMQ messaging workloads, but it can also support modern streaming use cases such as log collection, microservices integration, event streaming, and event sourcing. New workloads can run alongside legacy RabbitMQ applications, and because Pulsar is natively multi-tenant, you can do this without the complexity of managing multiple clusters across different lines of business.

• Adds Pulsar functions to RabbitMQ: RabbitMQ provides no native stream processing features; Pulsar functions, on the other hand, enable the creation of complex processing logic on a per-message basis. Starlight for RabbitMQ eliminates the need to deploy additional systems for lightweight computations. Fewer systems or components eliminate the need for ETL processes and reduce time to market

Start streaming with Starlight for RabbitMQ

Want to start improving your data infrastructure and focus on building new event-driven streaming applications— all while reducing costs and simplifying operations?

Starlight for RabbitMQ is included in DataStax’s Luna Streaming Enterprise support for Apache Pulsar. You can also dig into the Starlight for RabbitMQ source code on GitHub (available under the Apache Software License v2) to learn more about how you can easily leverage the power of Pulsar for your RabbitMQ applications. 

Follow DataStax on Medium to keep up with exciting announcements and developer resources on all things Apache, Cassandra, Kubernetes, streaming, and more. 

Other Resources

1. GitHub repo for Starlight for RabbitMQ for Apache Pulsar
2. DataStax Luna Streaming 
3. Apache Pulsar
4. RabbitMQ