DSE NodeSync: Operational Simplicity at its Best

Introduction

We’ve got something really special for administrators in DataStax Enterprise (DSE) 6: DSE NodeSync, designed with operational simplicity in mind, can virtually eliminate manual efforts required to run repair operations in a DataStax cluster.

NodeSync

To understand NodeSync, let’s talk about how we got here. One of the most important mechanisms for an administrator to run in Apache Cassandra™ is anti-entropy repair. Despite its name, repair is a process that should always be running in a cluster to ensure that data between nodes are consistent.

The fundamentals of repair haven’t changed since it was initially introduced many years ago.It’s designed as a single-process bulk operation that continuously runs for a long time which means when failure occurs, you must begin the repair over again. Repair is also computationally and network intensive as it creates merkle trees and streams them between nodes.  

The longer classic repair runs, the more failure prone it is.

To help mitigate some of these problems, complex tools were built to help orchestrate and add some structure and resiliency to repair. These tools try to split the repair process in multiple, more manageable pieces in an effort to improve operational simplicity, but in the end, these client-side tools were built to solve issues with a server-side mechanism. There’s only so much that can be done with tooling. Enter NodeSync: NodeSync is a ground-up rethinking of how we do entropy resolution in a DataStax cluster. Once you install DSE 6, NodeSync automatically starts running in the background. You simply tell it which keyspace or tables you’d like managed with NodeSync, and it handles the rest. No more compute-intensive tasks, no more complex tooling, just hands-off repair operations.

Enabling nodesync on a table is as easy as an alter table command.

NodeSync is designed to be simple and reliable. It divides the work it must complete into small tasks. These tasks are always tracked so it knows which data has been synchronized and which hasn’t. It also acts as a checkpoint mechanism so that if a node goes down, NodeSync knows exactly where to start again. NodeSync is also self-managing in that it will prioritize what to synchronize based on the last time the data was synced and whether it failed or not.

Easily enable/disable nodesync on tables through OpsCenter While NodeSync is designed to be as hands-off as possible, we know how important it is for administrators to understand what’s happening in the cluster so we’ve also updated OpsCenter to monitor NodeSync progress for you.

OpsCenter 6.5 lets you monitor NodeSync progress

Conclusion

We know our customers are going love NodeSync as it’s designed to make operations simpler with DataStax. Eliminating the need to orchestrate and manage repair means that administrators spend less time managing their DataStax clusters and more time doing other important tasks. To download DSE 6, and to get more information about NodeSync, please check out this page.