Accelerate Application Modernization with the Zero Downtime Migration Suite

Migrating production databases has always been challenging. That’s why DataStax has released the open source Zero Downtime Migration (ZDM) suite for Apache Cassandra®. Migrations are hard, even for the open source NoSQL database behind some of the largest applications in the world. There's a lot to think about, it can be a risky process, and downtime is almost always not an option.

In the past, structured data migrations were a difficult and expensive process. Hardware splitters would be used to mirror LUNs (logical units) between proprietary storage arrays. Often swing networks would need to be created, causing changes to network topology and introducing security risks. Some organizations would even go to the length of dropping dark fiber runs. Throughout the process, observability would be obscured, making it difficult to judge performance and production applications would often take a performance hit. With the DataStax limitless data stack and market-leading cloud economics, the story has changed.  While Apache Cassandra and DataStax have always been leaders in highly available data architectures, migrating between two clusters with zero downtime has often required creative and bespoke effort.

The Zero Downtime Migration suite was created with the aim to simplify migrations for any Cassandra Query Language (CQL) compatible datastore. This is a proven technology that has successfully migrated dozens of large, production workloads. Users can now easily migrate applications between a variety of Cassandra cluster types and deployment scenarios, whether that be Apache Cassandra,  DataStax Enterprise (DSE), DataStax Astra DB in your cloud of choice, or any other CQL-compatible datastores.

One successful Zero Downtime Migration was performed at Hornet, a social networking community and app for the LGBTQ+ community with more than 35 million members worldwide. “What was the big win for us? Nothing changed for our users. We delivered the same great response times during and after the migration, and we achieved this with no downtime,” said Nate Mitchell, Lead DevOps Engineer at Hornet.

Daniel Fake from WellAware described their migration process: “We're in the business of industrial IoT so our customers rely on continuous feeds of data and we can't afford any downtime. DataStax described a way to do a zero downtime migration process with the Zero Downtime Migration (ZDM) Proxy and other migration tools. It sounded promising, but we were still a little hesitant about putting something between our production applications and the database. However, we found the proxy worked flawlessly for us in production.”

Scenarios and Use Cases

The Zero Downtime Migration suite will move your application’s data with no downtime and with minimal configuration changes. Clusters will be kept in sync at all times by a dual-write logic implemented by ZDM Proxy. Observability platforms can pull in health metrics, such as Astra DB Metrics, and ensure performance requirements are met at the destination before cut over. Most importantly, an easy rollback is possible at any point for complete peace of mind.

There are numerous use cases for ZDM. Users can migrate from a self-managed Cassandra cluster to DataStax Astra in a cloud of choice. Migrate from Astra Classic to Astra Serverless. Perform simple on-site upgrades for Cassandra to a new version or hardware platform. Use ZDM to consolidate multiple applications or split applications across different clusters.

ZDM Architecture

The primary component of ZDM is the ZDM Proxy, a fully open-source and lightweight proxy that resides between the application and the source cluster. Astra DB or any CQL-compatible datastore can act as the destination. Both the source and destination will receive write operations and ensure write consistency on both clusters.

The ZDM Proxy is designed to be highly available and can be configured with a minimum of three nodes. Rolling restarts for configuration changes are available.

Get Started

The ZDM suite is a fully-featured tool for mitigating the risk of migrating Apache Cassandra datastores. The ZDM Proxy is a fully open-source, lightweight proxy that can handle a plethora of migration scenarios. Contact your DataStax representative at cloud@datastax.com for more information, including migration services and support options.

Resources

• ZDM Official Documentation
• DataStax Documentation Home
• ZDM Proxy is a fully open-source, lightweight proxy
• ZDM Utility
• DSBulk Migrator
• Cassandra Data Migrator